plasma8726

joined 1 year ago
[–] plasma8726@lemmy.today 1 points 1 day ago (1 children)

Thanks! I'll look into this. I'm a bit limited at 12GB of VRAM right now.

[–] plasma8726@lemmy.today 10 points 2 days ago (3 children)

Thanks for this link. Because of this article, I had claude stand up a llama.cpp container next to my already running ollama container. It ran side by side tests with the same model and parameters, and the results blew ollama out of the water. I'm in the process of moving hermes and openwebgui over to the llama.cpp instance to see how it goes day to day.

[–] plasma8726@lemmy.today 0 points 1 month ago (1 children)
[–] plasma8726@lemmy.today 2 points 10 months ago