105

in absolutely the funniest outcome so far, you can send data to an LLM that pops a Remote Code Execution vulnerability (mastodon.social)

submitted 7 months ago* (last edited 7 months ago) by dgerard@awful.systems to c/techtakes@awful.systems

13 comments fedilink hide all child comments

courtesy @self

preprint: https://arxiv.org/pdf/2309.02926
blackhat abstract: https://www.blackhat.com/asia-24/briefings/schedule/index.html#llmshell-discovering-and-exploiting-rce-vulnerabilities-in-real-world-llm-integrated-frameworks-and-apps-37215
Tong Liu's related research: https://scholar.google.com/citations?hl=en&user=egWPi_IAAAAJ

can't wait for the crypto spammers to hit every web page with a ChatGPT prompt. AI vs Crypto: whoever loses, we win

you are viewing a single comment's thread
view the rest of the comments

[-] cm0002@lemmy.world 5 points 7 months ago

pretty fucking embarrassing mistakes for supposedly world-class researchers

I'd argue it's not the job of the AI researchers, I'd say for this it's more on the devs and engineers that built all the support for the AI to bring it to production. So basically the UI, the underlying hardware, OS, VMs etc.

[-] self@awful.systems 14 points 7 months ago

all of the developers I know at AI-related startups identify as researchers, regardless of their actual role

the underlying hardware, OS, VMs etc.

no, let’s not blame unaffiliated systems engineers for this dumb shit, thanks

[-] cm0002@lemmy.world 0 points 7 months ago

no, let’s not blame unaffiliated systems engineers for this dumb shit, thanks

Oh, yea sorry I forgot AI models actually run in a vacuum and needs no supporting code or infrastructure to make it usable to the average user so it doesn't even need non-AI best security practices! Process isolation? OS hardening? Pfft who needs it

[-] mawhrin@awful.systems 10 points 7 months ago

i wouldn't touch the llm stuff with a barge pole unless i was expressly told to do so, and if i've been told to do it, i'd look for another employer (which i'm currently doing, for tangentially-related reasons).

and it's not that i don't care about the llms. i do care very much about them all ending in fiery pit of the deepest of hells.

[-] self@awful.systems 6 points 7 months ago

great thanks

this post was submitted on 25 Apr 2024

105 points (100.0% liked)

TechTakes

1442 readers

34 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 1 year ago

MODERATORS

dgerard@awful.systems