this post was submitted on 11 Nov 2025
10 points (91.7% liked)
important instance shit
165 readers
1 users here now
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I think it's just incompetence (on the part of the scrapers) - been keeping an eye on and having to deal with it for other infra for a while now, and you get a strong impression of very yolo value setting in code-side control: terrible pause/resume flow control logic, fuckall in the way of actual load spread, fuckall sensible backoffs, etc
I'm reasonably certain that a lot of it is written in the lowest-effort ways possible with extremely shit implementation quality, because their focus is purely on getting as much data as quickly as possible and they don't give a single fuck to be good netizens