ProleWiki

901 readers

1 users here now

ProleWiki

A community related to the ProleWiki project.

Post in this community to request articles, provide suggestions and discuss ways to develop our project

founded 4 years ago

MODERATORS

felipeforte@lemmygrad.ml

ProleWiki needs to expand its storage! (lemmygrad.ml)

submitted 4 months ago* (last edited 4 months ago) by felipeforte@lemmygrad.ml to c/prolewiki@lemmygrad.ml

10 comments fedilink hide all child comments

To expand our operations, we will have to eventually scale our storage. To simply pay for a better VPS server is not scalable and very costly for us

The way forward is to pay for external storage I believe. Do you have any experiences with cloud storage you could share with us? We're probably opting for Amazon S3 based on what they are able to deliver to us

you are viewing a single comment's thread
view the rest of the comments

[–] comrade_nomad@lemmygrad.ml 3 points 4 months ago (2 children)

I think it is safe to say we might need a bit more information to make a good suggestion here. Like how much storage do you currently need?

As said by others AWS(also GCP and Azure) can get really nasty with lock in(both egress fees and their specific offerings). All 3 are also US based so there is a chance they shut you down because they don't like the content.

Happy to discuss it a bit more and see where I can help.

[–] felipeforte@lemmygrad.ml 4 points 4 months ago (1 children)

Thanks for the reply!

We need to store 30 GB (safe estimate for 2-3 years) for publicly accessible images and documents. These would be constantly accessed by users. As far as I've measured, this could peak up to 2GiB/hour of network transfer

We also need to store up to hundreds of GBs (but less than a TiB in total) for files related to our server and backups, but these would have slow transfer rates, as they are not meant to be constantly touched. We plan on expanding towards other operations besides merely the wiki, so that would take extra space eventually

[–] comrade_nomad@lemmygrad.ml 3 points 4 months ago

Thanks for that additional info. I'll poke around a bit and see what I can find since you also mentioned hosting Matrix(I run an instance personally so will check usage info on mine). I'll fully admit most of my experience is with the big 3 cloud companies, at least professionally.

I'll also echo what another comrade mentioned in that a CDN would be helpful here.