543
submitted 8 months ago by Squire1039@lemm.ee to c/technology@lemmy.world

A trial program conducted by Pornhub in collaboration with UK-based child protection organizations aimed to deter users from searching for child abuse material (CSAM) on its website. Whenever CSAM-related terms were searched, a warning message and a chatbot appeared, directing users to support services. The trial reported a significant reduction in CSAM searches and an increase in users seeking help. Despite some limitations in data and complexity, the chatbot showed promise in deterring illegal behavior online. While the trial has ended, the chatbot and warnings remain active on Pornhub's UK site, with hopes for similar measures across other platforms to create a safer internet environment.

you are viewing a single comment's thread
view the rest of the comments
[-] ocassionallyaduck@lemmy.world 25 points 8 months ago

This is one of the more horrifying features of the future of generative AI.

There is literally no stopping it at this stage: AI generated CSAM will be possible soon thanks to systems like SORA.

This is disgusting and awful. But one part of me hopes it can end the black market of real CSAM content forever. By flooding it with infinite fakes, users with that sickness can look at something that didn't come from a real child's suffering. It's the darkest of silver linings I think, but I spoke with many sexual abuse survivors who feel the same about the loli hentai in Japan, in that it could be an outlet for these individuals instead of them finding their own.

Dark topics. But I hope to see more actions like this in the future. If pedos can self isolate from IRL interactions and curb their ways with content that harms no one, then everyone wins.

[-] gapbetweenus@feddit.de 35 points 8 months ago* (last edited 8 months ago)

The question is if consuming AI cp is helping to regulate the pedophiles behavior or if it's enabling a progression of the condition. As far as I know that is an unanswered question.

[-] Tolstoshev@lemmy.world 28 points 8 months ago
[-] gapbetweenus@feddit.de 11 points 8 months ago

For porn in general, yes - I think the data is rather clear. But for cp or related substitute content it's not that definitive (to my knowledge), be it just for the reason that it's really difficult to collect data on that sensitive topic.

[-] Asafum@feddit.nl 10 points 8 months ago

Why would it be any different? If it's about sexual gratification by their chosen media then I'd imagine it wouldn't matter what the subject was, but obviously it's always necessary to get actual data to know for sure.

[-] Baahb@lemmy.world 4 points 8 months ago

I think you're making assumptions that aren't fair but maybe aren't obvious either. Honestly I'm only thinking about this because I just watched the contrapoints video on twilight, and so I'll use her example, though she's talking about a slightly different topic. Gonna paraphrase like a mofo:

Weird Power dynamics between partners in a fantasy, like twilight, or say porn since we are being obvious here, is normal because self image often requires women to present one way while hiding their desires for sex. It's absolution of a sort, and is ostensibly healthy to explore in this way. That said... Some examples such as race play in fantasies may dehumanize the "other" in super not cool ways and reinforce negative racial stereotypes.

If we take that and extend it to pedophiles, normalization of the thought process leading to that sort of disfunction/disorder seems like a not great thing, but yeah, we'd need to study to learn more and that seems both difficult and likely undesirable for the researchers.

[-] gapbetweenus@feddit.de 1 points 8 months ago

Why would it be any different?

Because pedophiles display pathological deviation when it comes to sexual attraction.

[-] HonoraryMancunian@lemmy.world 4 points 8 months ago

Another question is, how will the authorities know the difference? An actual csam-haver can just claim it's AI

[-] phdepressed@sh.itjust.works -3 points 8 months ago

It'd still be CSAM whether AI or not.

[-] FilthyHookerSpit@lemmy.world 6 points 8 months ago

Don't know why you're down voted. You're right, it's CSAM, which encompasses any depiction of CP.

[-] cumming_normi@yiffit.net 0 points 8 months ago

Because "CSAM" states abuse as the third word in the acronym. Machine learning could (in theory, I lack knowledge on the current implementations) be trained without any children being abused (in any traditional sense anyway) and used to produce the content without any real children being involved (ignoring training data).

The downvotes likely come from a difference in definition between abuse and CP, images of nonexistent people cannot realistically harm anyone.

[-] FilthyHookerSpit@lemmy.world 4 points 8 months ago

Personally, I don't think it's arbitrary. A child in a sexual scenario is a depiction of abuse. Normal, healthy children don't engage in such behaviors.

[-] cumming_normi@yiffit.net -1 points 8 months ago

depiction yes, but if no person or animal is harmed is it abuse?

[-] yamanii@lemmy.world 6 points 8 months ago

What do you mean soon, local models from civitai can generate CSAM for at least 2 years. I don't think it's possible to stop it unless the model creator does something to prevent it from generate naked people in general like the neutered SDXL.

[-] ocassionallyaduck@lemmy.world 1 points 8 months ago

True. For obvious reasons I haven't looked too deeply down that rabbit hole because RIP my search history, but I kind of assumed it would be soon. I'm thinking more specifically about models like SORA though. Where you could feed it enough input, then type a sentence to get video content. That is going to be a different level of darkness.

[-] Zorque@kbin.social 1 points 8 months ago

Are... we looking at the same article? This isn't about AI generated CSAM, it's about redirecting those who are searching for CSAM to support services.

[-] ocassionallyaduck@lemmy.world 1 points 8 months ago

Yes, but this is more about mitigating the spread of CSAM. And my feeling was it's going to become somewhat impossible soon. AI generated porn is starting to flood the market and this chat it is also one of those "smart" attempts to mitigate this behavior. I'm saying that very soon, it will be something users don't have to go anywhere to get if the model can just fabricate it out of thin air, so the chat it mitigation is only temporary, and the dark web of actual CSAM material will become overwhelmed and swamped in artificially generating new tidal waves of artificial CP. So it's an alarming ethical dilemma we are on the horizon of that we need to think about.

this post was submitted on 05 Mar 2024
543 points (97.2% liked)

Technology

59210 readers
3984 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS