Hi all. Fedia.io has for a long time been subject to ddos attacks, including many that are "accidental", caused by myriad scrapers constantly hammering the site. I gave up on trying to play whack-a-mole with blocking them based on IP address (they do not honor robots.txt and do not use a conspicuous user agent string) since I was inadvertently blocking some legitimate users. So, I've restricted access to the content of fedia.io to only those that are logged in. That will mean we don't show up in search engines and whatnot, which for some will considered a good thing and will likely cause others to leave.
There is a remaining problem related to the login form. Calls to the login page are breathtakingly expensive, computationally speaking, and so I also have a script that monitors unusual numbers of calls to that form and blocks at the firewall any offenders. I strongly suspect I'm catching some legitimate users with this too, and so I continue to try to tune it, but it's maddening, y'all.
These issues have been causing performance problems for everyone (despite the fedia.io app running on a dedicated 96 core, 256GB server with nvme disks), and became unavailable for certain people that accidentally tripped various thresholds. I'm hoping most of this is resolved now.
Thanks for the patience.
I’ve always appreciated your competence and diligence in setting a good example of responsible hosting without resorting to shitty technofeudal fiefdoms like Cloudflare. Nice to see you are standing your ground and not selling the users out (unlike lemmy world and many other boot licking hosts).
I must say there is a notable side-effect to this. Since mbin does not have a cross-posting feature, I have been cross-posting by creating a link post to my original post from other relevant magazines. Now all of those links are unreachable to outsiders. To outsiders, I polluted their magazine with access-restricted links.
I can think of only two workarounds:
Fix 1 is impossible for existing past threads. Fix 2 is tedious and it’s a maintenance burden esp. if a post needs edits or updates. Fix 2 is also problematic because if I withold the original link, users cannot find other discussions that are scattered; but if I supply the original link, then non-Fedia readers cannot reach the OP anyway.
Worth mentioning that paywall sites handle this by giving crawlers special treatment. I’m not necessarily suggesting that though.
The login form loads must be through the roof because whenever a non-fedia user follows a cross-post into fedia, they are redirected to a login form which did not happen before.
There would be some relief if Mbin would implement a cross-post feature that automatically copies the OP text into the body, which would cut down on the number of visits. At the same time, I’ve always considered that a sloppy approach because edits are not sychronised. So in principle the threadiverse probably needs a smarter API specifically for cross-posts.
The use of 3rd-party clients would obviously give relief on the login form loading. But I have not found any decent 3rd-party clients for Lemmy or [km]bin - (perhaps because I’m fussy… I could really use a text UI in linux that stores content locally).
I understand. I have tried hard to make fedia.io work - it’s been far and away the most challenging app I’ve managed (note: the problems are all legacy kbin issues, the mbin team has been nothing but amazing). I am stuck in a difficult position - the site isn’t useful if I keep it locked down like it is now, and the site is super slow/requires constant attention if I make it open. I’ll have to assess my options and decide what the future for fedia is
I’d say it’s crippled but not useless, just as old-fashioned non-federated forums are still useful despite limitations. And as it is now we still have some of the fedi benefits.
bug 1
One bug comes to mind, which should perhaps be reported against kbin. Is the current locked down state something that is facilitated by the software, or did you hack it to redirect outsiders to login screens? If it’s the former, then the software is disservicing users who unwittingly post a link back to the access-restricted resource. If I cross-post by posting a link to fedia.io/yadayada, I should ideally get a warning to say “are you sure you want to post a link that is inaccessible to outsiders?”
bug 2 (more of an enhancement)
One work around is for a Fedia user to create a post, wait for a non-fedia response, then dig up the cached version on the non-fedia host and publicise that link in other places. That’s already possible with a bit of navigation effort. It would be useful if users could obtain a link farm of cached versions of any post or comment. Not just for the situation at hand but with small hosts coming and going coupled with censorship as well, users of mastodon, lemmy, and [km]bin all suffer from dataloss. A sophisticated client could use caching info to locally build/recover a complete thread, as well as track points of data loss.
Anyway, just brainstorming here.