How come none of these sites being hit by AI to the point they can't function implement rate limiting?
Rate limiting grouped by IP or UA tends to work pretty well, and normal users have very slow request rates so generally never get impacted.
this post was submitted on 25 Jul 2026
37 points (93.0% liked)
Fuck AI
7712 readers
1133 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
TL;DR
AI scraped it to death on behalf of prediction markets. They couldn't handle the load.
Not only scraped, but hacked by people looking for information not-yet-public.
A year ago, Cloudflare launched pay-per-crawl, letting sites charge AI crawlers per page. Last week, it went further, announcing a pay-per-use model in which publishers get paid when their content actually appears in an AI answer, and declaring that, from 15 September, its customers’ ad-supported pages will block unpaid “mixed-use” crawlers by default.
I'd be OK paying a subscription fee to access websites.
Two BIG problems though:
a) my anonymity b) conglomeration of websites into increasingly enshittified monopolies.
It doesn’t read to me that you or I as normal people looking at a website would be paying these fees.
Correct, but if humans get in free and bots don't, bots will pretend to be human. The only way it works ultimately is if every access is charged.
A year ago, Cloudflare launched pay-per-crawl, letting sites charge AI crawlers per page. Last week, it went further, announcing a pay-per-use model in which publishers get paid when their content actually appears in an AI answer, and declaring that, from 15 September, its customers’ ad-supported pages will block unpaid “mixed-use” crawlers by default.
Whether any of this works depends on whether the AI companies play along rather than route around it. But as Bruce put it to me, somebody has to try.
I'm having a hard time blaming this entirely on Ai. We're talking about a 30 year old site that has probably never had a proper redesign (front and back end). Of course it's eventually going to be compromised.
This doesn't say anything.
Why did the server crash? Who knows!
Did it crash because of ai scrapers? Maybe! It was definitely stressed by them.
Was the server hacked? No idea!
If it was hacked, was it an attempt to rig prediction markets? Sounds plausible!
This nebulous theorizing really is worrying me! I'm worried for the critical thinking capacity of every commenter. Sure, ai bot traffic is a problem, the entire internet is dealing with it, but its tenuous link to TheNumbers.com mystery is merely clickbait journalism without any sourcing.