this post was submitted on 06 Mar 2026
21 points (95.7% liked)

Selfhosted

56953 readers
946 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

  7. No low-effort posts. This is subjective and will largely be determined by the community member reports.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz
21
Rate limiting (sh.itjust.works)
submitted 5 days ago by DrunkAnRoot@sh.itjust.works to c/selfhosted@lemmy.world
13 comments fedilink hide all child comments
 

I run a SearXNG instance and rate limiting has never been a issue until now. so i wanted to ask what the cheapest and most privacy respecting vpn/proxys are to use.

top 13 comments
sorted by: hot top controversial new old
[–] potatoguy@mbin.potato-guy.space 6 points 5 days ago (1 children)

On the vpn/proxy idk, I think the bots would pass through it.

There are some nginx blocklists like the ultimate bad bot blocker, I could share my Cloudflare rules if you want it, as these are rules, it can be abstracted to anywhere.

[–] SteveTech@aussie.zone 6 points 5 days ago

Nginx also has support for rate limiting built in.

On the topic of blocking, I block useragents starting with Mozilla/5.0 that are using HTTP/1.X, since all modern browsers default to HTTP/2.0 and anything else is usually always bad bots. You can also return 426 with the Upgrade: h2c header to let some older browsers know to use HTTP/2.0.

[–] Shimitar@downonthestreet.eu 4 points 5 days ago (1 children)

Keep it private only to yourself? I do this, and never been rate limited.

[–] DrunkAnRoot@sh.itjust.works 2 points 4 days ago

i want to keep it public is the thing

[–] shrek_is_love@lemmy.ml 3 points 5 days ago (1 children)

Take this with a grain of salt since I haven't done much scraping (yet; I have a project I just started planning)

I've heard you're more likely to get blocked for using a VPN since some sites will block requests originating from data centers, which is less likely to happen coming from a residential IP address. (Although if you're already using a VPS, the right proxy may help)

This might be useless advice, but it might just be best to increase (and randomize) the amount of time between requests.

And to answer your question, Mullvad is what I use, and it's what I see reccomended (from both Reddit, and The Wirecutter) the most often because they store so little of your data, and you can even pay in cash.

[–] DrunkAnRoot@sh.itjust.works 2 points 4 days ago

i use a vps so ill see if somthing like mullvad could work thanks.

[–] PeriodicallyPedantic@lemmy.ca 1 points 4 days ago (1 children)

Can you explain a bit more about the setup?

Like would just using a VPS with pangolin secure tunnel work? Or does it have to be a vpn for some other reason?

Asking because maybe the question really boils down to the VPS provider with the best data transfer rates

[–] DrunkAnRoot@sh.itjust.works 1 points 2 days ago

my issue is that search engine providers block my vps ip so im looking for the best option to get around this either vpn or proxy

[–] sem@piefed.blahaj.zone 1 points 4 days ago (1 children)

So like, if you passed on the rate limiting to your users, would you get less rate limited from the search providers that SearXNG uses?

[–] DrunkAnRoot@sh.itjust.works 1 points 2 days ago (1 children)

no searxng functions as a proxy between the user an the search engine so if the search engine is blocking/rate limiting the searxng servers ip the only option is to use a proxy/vpn but the issue i run into with the vpn is i cant just install it because it will make ssh and the domain configuration not work so the only options are install the vpn inside of the docker container or if searxng has a config setting for it wich i need to look into still

[–] sem@piefed.blahaj.zone 1 points 2 days ago (1 children)

Are you the only user of this searxng?

[–] Decronym@lemmy.decronym.xyz 1 points 4 days ago* (last edited 1 day ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
HTTP Hypertext Transfer Protocol, the Web
IP Internet Protocol
VPN Virtual Private Network
VPS Virtual Private Server (opposed to shared hosting)
nginx Popular HTTP server

5 acronyms in this thread; the most compressed thread commented on today has 8 acronyms.

[Thread #139 for this comm, first seen 6th Mar 2026, 20:30] [FAQ] [Full list] [Contact] [Source code]