Ryzen 5900X, 64 gig DDR4-3200, 2tb ssd,10tb hdd and an RTX2070. Hosting Stable Diffusion, various llama.cpp instances with python bindings, jellyfin, sonarr, multiple modded minecraft servers, and a network file share.
Certainly blows my collection of old junk out of the water.
Has running Stable Diffusion had a big impact on the other stuff running? I'm thinking the Minecraft servers in particular.
SD mostly uses the GPU, so it's pretty light on everything else. The largest process is probably web-chat-ui with Wizard-30b model running.
Nice, you must be into deep learning with such a setup, any particular reason the deep deep learning models and GPU run in your server rather than in a powerful desktop system? Maybe you're actively offering AI services to the outside world?
It is a powerful desktop system as in it's housed in a desktop case and uses a "non-server" CPU.
Which like any computer can be a server. Being a server is more of a role than a form factor even if there are form factors specifically aimed at servers (rack mounted).
Wow, a reply I made in another community ended up under this one. Yeah I'm doing a lot of work on local models and text embedding models for vector search.
Looks tidy buddy! Have fun 😊
Yup, mostly running pretrained models for text embedding and some generative stuff. No real fine tuning.
I'm very ignorant about it. I wonder if I can buy a cheap computer and turn it into a server that I can only use by keeping it on all the time?
@Alpagu @behohippy Sure, that's how we all started. An old computer, throw proxmox in there, a few VMs and/or a docker host and you have yourself a pretty sweet homelab.
Thank you for answer
Yup, typically we get into it after upgrading an older PC or something and instead of selling the parts, just turn it into a server. You can also find all sorts of cheap/good stuff on ebay from office off-lease.
Thanks for answer
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!