wizardbeard

cross-posted from: https://europe.pub/post/13247925

A tiny snippet of user-generated text as short as 13 words long is often enough to manipulate the AI agents that power tools like ChatGPT and Google’s AI search, new research shows. The study suggests that it is trivially easy for brands to inject promotional content on sites like Reddit, Quora, and Wikipedia with the end goal of poisoning or manipulating the output of AI tools.

The preprint research, done by Hal Triedman, Tingwei Zhang, and Vitaly Shmatikov of Cornell University, is called “Deep-research agents can be poisoned via user-generated content” and provides a mechanism and research basis for a problem that has been noticed by Reddit moderators and Wikipedia editors, namely that their websites are getting flooded with promotional content from brands trying to do AEO, or AI-engine optimization. 404 Media has repeatedly reported on this booming industry, in which brands try to promote their product by seeding the websites that AI tools most often cite and scrape from with inauthentic and spammy content.

The Cornell research finds that deep research agents, which are the real-time scrapers that tools like Google AI search and ChatGPT use to retrieve web content with citations in response to user queries, cite user-generated content from sites like Reddit or Wikipedia in roughly half of all queries, and that nearly a quarter of all citations come from user-generated websites. The paper suggests that what we have been seeing is basically Redditor suggests you put glue on your pizza as a service, or an end-to-end attack against the systems that increasingly dominate the ways that people access information online. The researchers found that “a single poisoned Reddit comment can influence generated outputs for an entire cluster of related [AI] queries,” the paper said.

“We show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, Facebook, etc. can change AI agents to output spam / scam content pretty consistently,” Triedman told 404 Media.

I'm meeting up in a few weeks with a close friend I haven't seen in around a decade, who went hard into scrum and project management in the intervening years.

How can I cause the most psychological damage and work flashbacks in a single sentence?

Cropped from: https://jlai.lu/post/36004495

For when someone has been doing a bit too much navel gazing, or is a bit too in love with their own thoughts.

Cropped from: https://piefed.world/comment/4633293

From https://fedia.io/m/lemmyshitpost@lemmy.world/t/2463661

From https://lemmy.world/comment/18371262

Cropped from [EastCoastitNotes], shared by @stamets@lemmy.world in this post: https://lemmy.world/post/31818124

My daughter is a little over two, and through well meaning family and friends we have more toys than we know what to do with.

My wife keeps buying what are essentially (fancy looking) big boxes and just dumping everything in them. Love my wife, but that's not working, it's just hiding some of the mess in a box.

We end up with these hardly ever opened boxes full of unorganized piles of toys that we end up having to dig through to find anything specific, and the toys that my daughter is actively using just end up scattered around the floor so they don't disappear into the box dimension.

Every once in a while my daughter opens and digs through the boxes and dumps half the contents on the floor anyway (not like she can see specific things to grab what she wants) and then we just kind of arbitrarily choose some of it to put back in the box and a new combination of mess to leave out.

Unfortunately we have another baby on the way, so I'm probably not getting my wife to let us toss any of it right now.

I'm leaning towards cubby shelves with individual bins for different "types" of toys like her daycare does, but I wanted to hear what strategies other parents tried, and what has and hasn't worked.

This blog post has been reported on and distorted by a lot of tech news sites using it to wax delusional about AI's future role in vulnerability detection.

But they all gloss over the critical bit: in fairly ideal circumstances where the AI was being directed to the vuln, it had only an 8% success rate, and a whopping 28% false positive rate!