There's a lot of layers here. Personally, I fucking hate AI and I'm in favor, sure. But I can easily see there being legal ramifications for this too, it's something you have to tread lightly. And as much as I hate AI, what are the odds that some idiot somewhere using AI on my team gets bit by this and I still suffer the consequences?

Ultimately I don't think this is particularly a good idea, but it's worth being aware of just how easy this is to do and how likely it is to be overlooked. Relying on everyone to behave politely around a tech that they are increasingly growing disgruntled over is not a gamble I'm really willing to take. The fact that this kind of well-poisoning could end up being quite effective is only going to further incentivize it. This for sure isn't the last time we're gonna hear about something like this and honestly I'm surprised it took this long in the first place.

[–] expr@piefed.social 5 points 1 day ago

Good shit. Even if this kind of thing hit my work codebase (not that we use Java, but a different one), so be it. Maybe then they would realize how profoundly stupid all of this is.

[+] obelisk_complex@piefed.ca -22 points 1 day ago (7 children)

Yeah, I don't care what their reasons are, they added malware to their own project to fuck over a portion of their user base. That's not okay.

Let's analogise: an artist discovers their art has been printed out and hung on the wall in the home of someone they don't like. Does that artist have the right to try and burn down that person's house in retribution?

[–] ech@lemmy.ca 9 points 1 day ago (1 children)

[equivalent to burning] down that person's house

Lol, no. Not even close to the same thing. Go hyperbolize somewhere else.

[–] obelisk_complex@piefed.ca 0 points 11 hours ago* (last edited 9 hours ago)

I'll try to make my point clearer:

Damaging other people's shit because you don't like the way they live is not okay.

[–] deliriousdreams@fedia.io 12 points 1 day ago (2 children)

Simple English is not malware. They didn't do their due diligence either.

No code was executed. Failure to sandbox the AI agent you use is not the problem of the person who is providing free code databases to the internet at large.

When it was the guy doing this on LinkedIn and forcing AI agents that interacted with his profile to speak to him in old English and address him as "my lord" was that malware?

[–] obelisk_complex@piefed.ca 0 points 11 hours ago* (last edited 11 hours ago)

Someone actively trying to sabotage their users is not okay, regardless of the excuses you want to trot out.

[–] howrar@lemmy.ca -2 points 1 day ago (2 children)

It's malware as much as a zip bomb is malware. Both serve as input to another program to make it do a specific thing. That thing causes harm on the end user's device. Asking an agent to speak in old English causes no such harm.

[–] obelisk_complex@piefed.ca 2 points 11 hours ago

I appreciate you seeing this for what it is and saying so, where other people are twisting themselves into pretzels to pretend like it's anything else.

[–] deliriousdreams@fedia.io 4 points 1 day ago (2 children)

Malware (a portmanteau of malicious software) is any software intentionally designed to cause disruption to a computer, server, client, or computer network, leak private information, gain unauthorized access to information or systems, deprive access to information, or interfere with the user's computer security and privacy without their knowledge.

Calling this malware is like calling a DDOS attack hacking.

People have been warned about the fact that there is the potential for an AI LLM to take direction from text without the user's knowledge since this tech hit the mainstream. Additionally, it's also not malware when Alexa responds to its name while you're watching a YouTube video.

These people infected their own computers with software that could be externally controlled on purpose. The LLM might be considered malware since it's working as designed, but the plain English isn't.

[–] obelisk_complex@piefed.ca 1 points 11 hours ago (1 children)

intentionally designed to cause disruption to a computer, server, client, or computer network

Actively and deliberately adding text with the explicit purpose of attempting to delete other peoples' work fits this quite nicely, thank you for including it to illustrate my point.

[–] deliriousdreams@fedia.io 1 points 1 hour ago

What work?

[–] howrar@lemmy.ca 1 points 1 day ago (1 children)

Yes, I acknowledge that it isn't malicious software. That's why I compare it to a zip bomb. The important part isn't "software". It's the "malicious". None of what Obelisk said relies on this thing being software.

[–] deliriousdreams@fedia.io 0 points 20 hours ago (1 children)

They called it "malware" when this already has a name and it's "prompt injection".

We've been telling everyone for years not to download and run code you don't understand.

We've been telling people since the advent of the AI LLM that prompt injection is a thing and people who use AI LLM's should protect against this.

These AI companies are constantly stealing things from FOSS projects as well as artists and other devs/creators.

Nobody sees a problem with that when it's poisoning LLM's themselves to protect artwork. But now that some idiot who doesn't check code before they execute it etc is crying about it and it's a big deal. I don't see much of a difference here.

If you want the FOSS community to be there for you, don't go out of your way to use the corporate BS that's actively destroying it.

[–] obelisk_complex@piefed.ca 1 points 11 hours ago

If you want the FOSS community to be there for you, don't go out of your way to use the corporate BS that's actively destroying it.

Heheh, and another one of you thought to call me hyperbolic.

[–] h_ramus@piefed.social 20 points 1 day ago* (last edited 1 day ago)

Release Notes

Warning: Do not use this release with an „AI“ Coding Agent of any form. The tool‘s output may confuse the agent and make it do unwanted things. See the paragraph in the user guide for details.

It's like lighting a cigarette in a petrol station and being annoyed that they didn't make it fire risk proof.

Also, what kind of security is applied nowadays that text can become an executable and no one bothered to put safeguards in place. This is like 90s internet, raw dogging data, no firewall, no encryption just pure unaltered and blind faith that it's all dandy.

[–] Alk@sh.itjust.works 7 points 1 day ago

No because that puts human life in danger and is not comparable.

[–] mlatu@moist.catsweat.com 6 points 1 day ago (1 children)

you might want to hear about this guy... banksy and this picture that was auctioned off only to be shredded as the auction was over....and maybe stew on that story for a minute...

[–] obelisk_complex@piefed.ca 1 points 11 hours ago

I don't see how that's relevant here. Banksy didn't then go and shred every other piece of art the buyer owned. That would be pretty fucked up.

[–] EvergreenGuru@lemmy.world 5 points 1 day ago (1 children)

You’re a flame retardant.

[–] obelisk_complex@piefed.ca 1 points 11 hours ago (1 children)

Classy.

[–] EvergreenGuru@lemmy.world 2 points 3 hours ago

😉

[–] Tyrq@lemmy.dbzer0.com 1 points 1 day ago

When cartograhpers made up places to protect their business