Beginner questions thread
(sh.itjust.works)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 02 Oct 2023
23 points (96.0% liked)
LocalLLaMA
2168 readers
2 users here now
Community to discuss about LLaMA, the large language model created by Meta AI.
This is intended to be a replacement for r/LocalLLaMA on Reddit.
founded 1 year ago
MODERATORS
Do you usually have some other front-end over the model? I can run llama.cpp directly in interactive mode but the results are a little underwhelming. However there seem to be various front ends that get better results? Is this down to better prompting and parameter control? I've seen temperature mentioned in relation to ChatGPT but I have no idea what rope and yarn factors are for?
I use text-generation-webui mostly. If you're only using GGUF files (llama.cpp), koboldcpp is a really good option
A lot of it is the automatic prompt formatting, there's probably like 5-10 specific formats that are used, and using the right one for your model is very important to achieve optimal output. TheBloke usually lists the prompt format in his model card which is handy
Rope and yarn refer to extending the default context of a model through hacky (but functional) methods and probably deserve their own write up