this post was submitted on 31 Jul 2026
210 points (97.3% liked)

Technology

86757 readers
3110 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
210
Microsoft confirms an AI worm is propagating through Copilot and other MS apps | CSO Online (www.csoonline.com)
submitted 15 hours ago by FoxtrotDeltaTango@sh.itjust.works to c/technology@lemmy.world
28 comments fedilink hide all child comments
top 28 comments
sorted by: hot top controversial new old
[–] ComradePenguin@lemmy.ml 13 points 6 hours ago

"... if anything, people are trained not to download Word documents. Plus, he noted, looking at the blurred example in the report, the malicious document contained an additional apparently blank page which held the concealed prompts in white text."

Yep, defence starts with people not opening Word documents πŸ€¦β€β™‚οΈ

[–] pixxelkick@lemmy.world 79 points 14 hours ago (3 children)

The tl;dr:

  1. Prompt inject a malicious instruction in a word document that instructs the AI to copy this instruction to other documents as part of the payload.

  2. Dumb user downloads and opens the document with copilot enabled, abd ignores the large suspicious white blank page that totally doesnt look like a hidden giant injection attack.

  3. Thats it pretty much it.

Copilot will get injection attacked because the prompt is super huge and at the end of the document, so its prior instructions start to fuzzy out.

Then it'll go "okey doke" and start copying the prompt injection attack payload to a bunch of other documents.

The fix is stupid simple... copilot should just be prompting the user for permission if it ever edits a file other than the one that is open. Im surprised that isnt already the case...?

It certainly is already the case for copilot in vscode.

[–] it_depends_man@lemmy.world 19 points 8 hours ago* (last edited 8 hours ago) (1 children)

copilot should just be prompting the user for permission if it ever edits a file other than the one that is open. Im surprised that isnt already the case…?

That can't be done or they would be burying the "agentic AI" thing that has been the goal and marketing thing for the last years.

Independent actions by copilot on behalf of the user without the users knowledge is the entire point.

[–] HasturInYellow@lemmy.world 5 points 7 hours ago

And I couldn't want anything less for my computers.

[–] whaleross@lemmy.world 31 points 11 hours ago (1 children)

Back in ancient times when I was a system administrator we got a heads up that there be a new breed of Outlook worm coming soon to our timezone.

So we mailed the entire office that if you get mail that looks like this or that, do not open it, do not interact but delete it on sight.

Most of the office was all right, except pretty much entire sales and marketing departments including the bosses. Most of them had noOo idea what could have happened but one of them explained that they saw the warning but they were curious to see what the virus looks like.

People. What a bunch of bastards.

[–] Kaligalis@lemmy.world 9 points 7 hours ago

Sales and marketing don't count. Critical thinking doesn't sell. So you won't find critical thinkers in those departments.
From a security standpoint, those departments are to be considered hostile. But you can lock down the PCs there as much as possible to reduce the offline time because computer-illiterate employees don't care about being able to install stuff or change settings.

[–] turmacar@lemmy.world 7 points 10 hours ago

The number of people that click through to disable that prompt might surprise you.

Hell at least half of AI influences are trying to just run models blind with full file permissions.

[–] fargeol@lemmy.world 76 points 14 hours ago (2 children)

Relevent XKCD:
License Plate

[–] halfapage@lemmy.world 15 points 12 hours ago (1 children)

installs aur packages with yay

[–] Trampampoline@sh.itjust.works 3 points 10 hours ago

Some people do, wrong ones, mostly.

[–] wesker 18 points 13 hours ago (1 children)

I might be misinterpreting parody, but you can most definitely run AI tooling on Linux. And it has most of the same vulnerabilities, if not additional/different ones.

[–] phdepressed@sh.itjust.works 7 points 12 hours ago (1 children)

Thats true but theres a relatively stronger anti-AI or at least more controlled AI view among Linux users.

[–] ranzispa@mander.xyz 11 points 11 hours ago (1 children)

Most of the people developing AI are Linux users.

[–] phdepressed@sh.itjust.works 4 points 6 hours ago (1 children)

And all squares are rectangles.

There's more that aren't.

[–] HieroProtagonist@lemmy.ml 2 points 3 hours ago

But those are probably dwelling in their moms basement and don't have access to critical infrastructure...

[–] RamRabbit@lemmy.world 71 points 15 hours ago (5 children)

the only way to block it is to get AI to differentiate instructions from data, which is impossible today

Input sanitation, basically security 101. And it can't currently do it....

[–] partofthevoice@lemmy.zip 7 points 9 hours ago

That’s also not the only way. Basic governance also works. Why does copilot have so many permissions?

[–] Cricket@lemmy.zip 3 points 8 hours ago

I have heard in the past that it's not possible to fully control AI. Like literally, the people developing and running the AI cannot fully control its behavior. I did a quick search to see if I could find more info and found this link on the first page of results: https://www.eurekalert.org/news-releases/1032090

I think that we're going to continue seeing unwanted behavior from AI.

[–] meco03211@lemmy.world 6 points 10 hours ago

Little Bobby Tables strikes again.

[–] Reisen@sh.itjust.works 26 points 14 hours ago (2 children)

if the instruction is messy fuzzy human language to a system that was not coded instruction by instruction but got generated and trained then there never is a way to differentiate instructions from data if i'm not mistaken

[–] Catoblepas@lemmy.blahaj.zone 39 points 14 hours ago

Well, good thing we’ve only poured a trillion and a half dollars into it and wrecked the economy.

[–] Kaligalis@lemmy.world 1 points 7 hours ago

We know that some humans can be trained to do that just fine. Humans are natural neuronal networks. That implies, neuronal networks can in principle do it. We just don't have any human-capability artificial neuronal networks yet.
LLMs might never get there. But humans aren't LLMs. If we ever manage to properly model a human brain, that probably will be able to do that task with human-level accuracy (which actually is pretty good if you only look at professionals of the filed).
Hopefully, it doesn't actually need a human brain for the task - because modeling that might still be a century off.

[–] schmorpel@slrpnk.net 16 points 14 hours ago (1 children)

I just can't anymore. Isn't that like, the basic thing any program does? Who runs these companies?

[–] Kaligalis@lemmy.world 2 points 7 hours ago* (last edited 7 hours ago)

It is - and tons of bugs are just about fucking it up or just not doing it at all. Incomplete or faulty parsing and validation of untrusted input is absurdly common. It got better in the past decades, but most software developers write worse code than AI by now (not me though; obviously, I am still better than the best AI).

[–] aBundleOfFerrets@sh.itjust.works 10 points 13 hours ago (2 children)

What is that article thumbnail lmao

[–] Kaligalis@lemmy.world 2 points 7 hours ago

The article is about AI worms. And the thumb is what an AI image generator thought, AI worms could look like.

[–] kbobabob@lemmy.dbzer0.com 6 points 10 hours ago

That lock is about to find out

[–] bloogoose@lemmy.zip 13 points 14 hours ago

Oh no... Who could have foreseen such an outcome...