Beginner questions thread
(sh.itjust.works)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 02 Oct 2023
23 points (96.0% liked)
LocalLLaMA
2168 readers
2 users here now
Community to discuss about LLaMA, the large language model created by Meta AI.
This is intended to be a replacement for r/LocalLLaMA on Reddit.
founded 1 year ago
MODERATORS
Late to the party, I never got FOSAI working until I found LMStudio, but I have 2 questions:
Is there any way I could utilize my GPU, a Radeon RX6800M (12GB VRAM)? I got Mistral-7B doing 5 tokens/s but it's all running on the CPU.
Is there any model specifically for programming questions? This could be of immense help to my projects without having to ask ChatGPT.
Have you tried the guide on AMD's site? It looks like it's for Windows, and I don't know what you're running. Plus, I use Ollama, so I probably can't be of much help.
For programing, my favorite is Dolphin-Mixtral, but I've had good results with Dolphin-Mistral and Llama2.