this post was submitted on 01 Mar 2026
74 points (100.0% liked)
United States | News & Politics
8987 readers
241 users here now
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
No problem. My desktop has an nvidia RTX 350 card that has 8GB of ram on it. It's a basic modernish video card. Ollama is an open source framework for running large language models. The model I'm using is qwen 2.5. It has 3 billion (3b) parameters(basically the size of the LLM) . Docker is a program that allows you to basically run smaller dedicated computers on your computer.
I am not in China. I'm an American living in Albania. I recommended DeepSeek because it's free, works well, and if a company is going to have the information on what you're chatting about, it might as well be one that isn't in the same country as you.
Thanks for all the info! I’d love to run a model locally, but I don’t have the money for a decent enough setup right now, but I know it’s getting close. How effective is the 3b model? Does it do the job for you or you feel like it’s lacking? Are requests pretty slow on that machine?