91
submitted 10 months ago by maor@lemmy.org.il to c/lemmy_admin@lemmy.ml

My pictrs volume got quite huge and I wanna delete pics cached from other instances. The thing is I'm not sure which pics are from my instance and which aren't, because they all have cryptic filenames. Anyone knows of a way to differentiate?

top 8 comments
sorted by: hot top controversial new old
[-] Die4Ever@programming.dev 5 points 10 months ago* (last edited 10 months ago)

I just saw this and remembered your post

Lemmy Post Purger https://github.com/Fmstrat/lpp

No idea if it's good, never used it

I don't think it has an option to only delete images, you could file that as an issue on their GitHub

[-] maor@lemmy.org.il 3 points 10 months ago

That's beautiful, looks exactly like what I was looking for. Thank you, I'll let you know when I try it how it went

[-] maor@lemmy.org.il 1 points 10 months ago* (last edited 10 months ago)

It's working but I suspect it doesn't delete pics or smth. But it's nice to have nontheless

[-] Maiznieks@lemmy.world 5 points 10 months ago

Maybe you have to crossmatch db content to files (with a script) and remove anything that is not relevant? I don't manage instance, so i might be completely off.

[-] ticoombs@reddthat.com 4 points 10 months ago* (last edited 10 months ago)

I would recommend Backblaze B2 storage instead of removing post history.

The problem with post removal is that it will affect all other instances' as you don't (by default) cache original images from other fediverse instances (iirc)

So for example, my instance Reddthat will reference an image hosted on your instance, which will then 404. (But the thumbnail will be cached (I think))

Backblaze is dirt cheap and the pictrs object migration is now multi threaded making it very easy to migrate now. I was hosting 15GB and it amounted to bills of cents! $5/TB or $0.005 per GB. If you add CloudFlare Infront you then get free egress! Or any CDN that is part of the "Bandwidth alliance". I now have ~120GB of pictrs storage. After 2-3 months.

Also, there was a recent chat on the lemmy admin matrix channel about setting some variable in pictrs so it does not cache any images. But I don't remember what it was so maybe check the pictrs documentation.

[-] skankhunt42@lemmy.ca 2 points 10 months ago

I'm reading about this now. Thank you.

Do you put cloudflare between the VPS and Backblaze or is their website just referring to "CDN and compute partners" between the VPS and the end users? ... Or both?

[-] ticoombs@reddthat.com 3 points 10 months ago

To get it completely free it would need to be:

  • cloudflare > vps
  • cloudflare > backblaze

Because Lemmy does not have a way to have all media served from a different url/subdomain (like a CDN). Your setup will be:

  • cloudflare > vps
    • vps > backblaze

Basically just put cloudflare infront of your vps, and cache everything.

That means if a user uploaded a 10MB file, it would go though your VPS, get uploaded to backblaze. When a user looks at the uploaded image url, on the first request it gets requested from blackblaze, via your vps, and then gets cached on Cloudflare.

Then any extra requests from then on will be via Cloudflare. So you will technically be charged for some api requests and 1x 10MB of egress traffic.

[-] skankhunt42@lemmy.ca 2 points 10 months ago* (last edited 10 months ago)

Sounds like a normal setup then. Just have CF in front of the VPS. Thanks!

I'm reading about Canadian hosting laws around user generated content and it doesn't seem like we have the same protections the US or EU has. I'm still reading but my big concern is legal trouble because of end users. If I don't selfhost for the community I'll offer to help an instance I like.

Thanks again!

this post was submitted on 15 Aug 2023
91 points (96.9% liked)

Lemmy Administration

686 readers
1 users here now

Anything about running your own Lemmy instance. Including how to install it, maintain and customise it.

Be sure to check out the docs: https://join-lemmy.org/docs/en/administration/administration.html

If you have any problems, describe them here and we will try to help you fixing them.

founded 4 years ago
MODERATORS