Like I said, [edit: at one point] Facebook requested my robots.txt multiple times a second. You've not convinced me that bot writers care about efficiency.

[edit: they've since stopped, possibly because now I give a 404 to anything claiming to be from facebook]

[–] Quill7513@slrpnk.net 3 points 6 months ago (1 children)

You've not convinced me that bot writers care about efficiency.

and why should bot writers care about efficiency when what they really care about is time. they'll burn all your resources without regard simply because they're not who's paying

[–] grysbok 4 points 6 months ago (1 children)

Yep, they'll just burn taxpayer resources (me and my poor servers) because it's not like they pay taxes anyway (assuming they are either a corporation or not based in the same locality as I am).

There's only one of me and if I'm working on keeping the servers bare minimum functional today I'm not working on making something more awesome for tomorrow. "Linux sysadmin" is only supposed to be up to 30% of my job.

[–] grysbok 3 points 6 months ago* (last edited 6 months ago)

I mean, I enjoy linux sysadmining, but fighting bots takes time, experimentation, and research, and there's other stuff I should be doing. For example, accessibility updates to our websites. But, accessibility doesn't matter a lick if you can't access the website anyway due to timeouts.