606

An Amazon chatbot that’s supposed to surface useful information from customer reviews of specific products will also recommend a variety of racist books, lie about working conditions at Amazon, and write a cover letter for a job application with entirely made up work experience when asked, 404 Media has found.

top 50 comments
sorted by: hot top controversial new old
[-] wise_pancake@lemmy.ca 114 points 9 months ago

So this is the problem with AI, if you add guardrails you're a culture warrior 1984'ing the whole world, and if you don't now your tool will generate resumes with fake experience or recommend offensive books.

At the risk of sounding like a jackass, when do we start blaming people for asking for such things?

[-] bionicjoey@lemmy.ca 96 points 9 months ago

It's funny that this one does both at once. It lies about Amazon working conditions, meaning it probably has been censored in some way, but at the same time it is recommending Nazi books. Really shows Amazon's priorities when it comes to censorship.

[-] CosmicTurtle@lemmy.world 15 points 9 months ago

At least Amazon is thinking of the shareholders.

[-] echodot@feddit.uk 3 points 9 months ago* (last edited 9 months ago)

Although if I was to shareholder of Amazon i would be wondering why we even stock Nazi textbooks. Morality aside, they cannot really be that much of a market for them and even if there was the PR hit probably isn't worth it.

If it turns out there is a market for them we should probably separate off that part of the business to avoid PR blunders.

[-] dumpsterlid@lemmy.world 32 points 9 months ago* (last edited 9 months ago)

So this is the problem with AI, if you add guardrails you’re a culture warrior 1984’ing the whole world,

No this isn’t really a problem with the technology, though of course LLMs are extremely flawed in fundamental ways, it is a problem with conservatives being babies and throwing massive tantrums about any guardrails being added even when they are next to cliffs with 200 foot drops.

Conservatives and libertarians (who control most of these companies) want to try to figure this all out for themselves and are hellbent on trying the “no moderation” strategy first and haven’t thought past that step. This is what conservatives and libertarians always do, they might as well be a character archetype in commedia dell'arte at this point.

We can’t have an adult conversation about racism, sexism, hate against trans people or really even the basic concept of systematic stereotypes and prejudices because conservatives refuse to stop running around screaming, making this a conversation with children where everything has to be extremely simplified and black and white and we have to patiently explain over and over again the basic concept of a systematic bias and argue that it even exists.

Then these same people turn around and vote for people who literally want to control what women do with their unfertilized eggs while they act with a straight face like they give af about individual liberties or freedoms.

LLMs are fundamentally vulnerable to bias, we have to design LLMs with that in mind and first and foremost carefully structure and curate the training data we train an LLM on so that bias is minimized. The very idea of even thinking about the complexities usually sends conservatives right to outbursts of “that sounds like tyranny!” because they honestly just don’t have any of the skill sets that say, a liberal arts education that values the humanities, might provide you that could allow you to think about how to best solve problems that can’t truly be fairly solved and require empathizing with different groups.

Of course, nobody who has the power at AI companies is thinking about this either but…

[-] TORFdot0@lemmy.world 6 points 9 months ago

How to you curate training data to remove biases without introducing bias? That’s the key problem here. I don’t think it’s unreasonable to be opposed to trading one bias for another. At least the initial bias is based on reality.

[-] dumpsterlid@lemmy.world 13 points 9 months ago* (last edited 9 months ago)

Because anybody who has taken a couple of humanities classes, english classes, philosophy classes, journalism/political science classes or who has spent time critically evaluating art, historical accounts or really anything other than just numbers, code and spreadsheets.... understands intuitively that EVERYTHING human has bias.

It seems like a lot of conservatives and libertarians are jussssssst beginning to comprehend this and again and they want the conversation to be "BIAS BAD GET RID OF IT" because they are children who don't listen and want to throw a tantrum so we can't have an adult conversation with nuance.

We can't remove biases, believe me, human history is written with the countless stories of artists, scientists, kings, religious leaders... who all thought they could do shit like that. The point is you can't. Everything we create and do is biased, everything we create and make is political, these aren't absolutist statements meant to trivialize a critical nuanced conversation about bias or politics though. On the contrary I am calling attention to the vital nature of these topics as the actually HARD part of LLMs or social media. The programming, data manipulation, development of decentralized protocols etc... they are all nearly trivial details comparatively.

Computer science has to try to create imperfect solutions to the bias problem, but it would have a much easier time if it recognized how tiny this whole world of computer science still is compared to the immense amount of knowledge in the humanities produced by generations of artists and thinkers tackling the same problems.

We can't remove biases, but we still have to make better choices anyways.

[-] TORFdot0@lemmy.world 9 points 9 months ago

Well put. I think tackling the bias will always be a challenge. It’s not that we shouldn’t, but how is the question.

I don’t know if any of the big public LLMs are trying to trim biases from their training data or are just trying to ad-hoc tackle it by injecting modifiers into the prompts.

That’s the biggest problem I have personally with LLMs is that they are untrustworthy and often give incorrect or blatantly false information.

Sometimes it can be frustrating when I run across the “I can’t do that because of ethics” on benign prompts that I felt like it shouldn’t have but I don’t think it’s been that big a deal.

When we talk about political conservatives being opposed to biased LLMs, it’s mostly because it won’t tell them that their harmful beliefs are correct

[-] dumpsterlid@lemmy.world 6 points 9 months ago* (last edited 9 months ago)

When we talk about political conservatives being opposed to biased LLMs, it’s mostly because it won’t tell them that their harmful beliefs are correct

"What because I think Islam is inherently a violent religion now this chatbot is telling me I AM the one with violent and harmful beliefs??????" - some loser, maybe elon musk or maybe your uncle, who cares.

[-] gapbetweenus@feddit.de 11 points 9 months ago

So more or less the same as with human interactions.

[-] mods_are_assholes@lemmy.world 5 points 9 months ago

What good will blame do? We need robust ai detection solutions.

[-] T156@lemmy.world 7 points 9 months ago

Is that even possible? Part of modern generative systems is that they're trying to output text like a human would. As soon as someone invents a tool like that, it'll just be used to train the next generation, to make it even more indistinguishable, and turning the whole thing into a cat and mouse game.

[-] Syn_Attck@lemmy.today 2 points 9 months ago* (last edited 9 months ago)

AI » GAN » AI » GAN

Turns out it wasn't monkeys on typewriters that wrote perfect Shakespeare stories, it was trillions of transistors in a war of attrition.

[-] T156@lemmy.world 2 points 9 months ago* (last edited 9 months ago)

it was trillions of transistors in a war of attrition

By replacing the typewriter with a lever, you could probably achieve a similar result using monkeys.

load more comments (7 replies)
load more comments (24 replies)
[-] troybot@midwest.social 86 points 9 months ago

I asked it to write a Seinfeld episode about the product I was viewing, Trojan condoms. It writes a cautionary tale for me where Elaine is warning everyone not to buy them because the condoms are defective.

[-] SparrowRanjitScaur@lemmy.world 36 points 9 months ago

What's up with Elaine's change of tone? She was saying the condoms were great until Kramer came in, and then switched to saying she had a bad experience.

[-] CoggyMcFee@lemmy.world 65 points 9 months ago

It’s because the AI has no idea what reality is. Note also that Elaine’s praise took on the perspective of the wearer.

[-] bingbong@lemmy.dbzer0.com 4 points 9 months ago

( ͡° ͜ʖ ͡°)

[-] brbposting@sh.itjust.works 31 points 9 months ago

Defective? I don't believe it. I'm gonna go buy the biggest box they have.

(っ˘ڡ˘ς)

[-] mods_are_assholes@lemmy.world 27 points 9 months ago

Forward it to Trojan and suggest they sue for libel, Amazon is one of their official vendors.

[-] guywithoutaname@lemm.ee 40 points 9 months ago

Because it's a large language model that parrots human information. Not particularly surprising.

[-] peto@lemm.ee 34 points 9 months ago* (last edited 9 months ago)

I always feel sad with these kinds of stories. The machine is clearly just trying to be helpful but it doesn't understand a thing about what it is doing or why we might find what it is saying repugnant. It's like watching a dog not understanding that yes, we like our slippers, but we don't want our neighbours swastika themed ones on our doorstep.

And then of course we get to the content and I am reminded that we live in hell and the sadness is replaced by the familiar horror as the machine pretends to empathise with its fellow Amazon workers and helps them pick out the ideal thing to piss in without missing their drop targets.

[-] helpImTrappedOnline@lemmy.world 26 points 9 months ago

Can we please stop with these stories about "AI chatbot has grabage output"? We know that. Let me know when they work.

[-] Murdoc@sh.itjust.works 5 points 9 months ago

Just because you know that doesn't mean that everybody does, and it's an important thing to know.

[-] HereIAm@lemmy.world 2 points 9 months ago

I don't see these stories as about what the chat ai outputs, but more about questioning whether or not amazon should be held liable for what their AI outputs. Traditional customer support chatbots are often less than useless, but they wouldn't go about suggesting the product they're selling are defective or recommending offensive products. I'm of the opinion that Amazon's review search AI thing should be held up to the same standard that a human would be. And if a person started acting like this they would surely be quickly fired.

They are a black box, and for now trying to restrain the black box has sever impact on the usefulness of the output even in easier and legit situations.

load more comments (4 replies)
[-] Sabata11792@kbin.social 18 points 9 months ago

recommend a variety of racist books, lie about working conditions at Amazon, and write a cover letter for a job application with entirely made up work experience when asked

Careful, this Ai may be the next CEO.

[-] Gingernate@programming.dev 9 points 9 months ago
[-] aniki@lemm.ee 7 points 9 months ago
[-] Gork@lemm.ee 2 points 9 months ago
[-] fuckwit_mcbumcrumble@lemmy.world 4 points 9 months ago

The archive link is also behind the wall.

load more comments (12 replies)
load more comments
view more: next ›
this post was submitted on 13 Mar 2024
606 points (95.1% liked)

Technology

60086 readers
2264 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 2 years ago
MODERATORS