I don't know, but I'm willing to bet that economies of scale actually mean data centers are more efficient. This isn't to say their use is justified, just that they're able to take advantage of things a home computer can't.

However, having to run it locally means it needs to be much more limited. This is doubly true if you want to run the game and the LLM at the same time. The LLM is easily able to consume all resources your system has available if you allow it to, which means the game won't run well (if it runs at all). This limits the use so it can't just be shoved everywhere and constantly running, like it could if it's sent to a data center. It's not more efficient, just less consumption.

Local runs on device, so no need to connect to a big data center that chugs lots of water and all those other problems. Of course, because it’s a smaller far tinier model it’s nowhere near as accurate, but especially for things like this you don’t really need a big accurate LLM model.

I think I also though I should warrant a disclaimer that I am a Software Developer, not a AI Developer. So there’s far less backing then from my perspective than someone who works with this stuff for a living

I mean there might be a way, but it's not easy.

The laziest and worst method is to use ChatGPT and have it "pretend to be some character" with a system prompt.

If you want something really good, you would need to train the model from scratch based only on knowledge that one particular character would learn from their world up until that point. However this is going to be a ton of work just for one character.

For a middle ground you could probably cheat a little and start with a model that's close to the knowledge base you would want most characters to have. Then you would use something like a LoRA, or RAG on top of it for each individual character.

For instance, if you wanted to make a game in a Victorian Era setting, you could start with this model that's only trained on text from the 1800's: https://github.com/haykgrigo3/TimeCapsuleLLM

To make it better you would have multiple base models that are trained on various backgrounds that NPCs could have (Farmers vs Merchants vs Soldiers vs Nobility, etc).

Even then, this would not work well for certain games. For example, if you're trying to tell a specific story, you don't want a character that will go off script or give away some information that spoils an intended plot twist.

We need some kind of giant regex to filter out user input that would try to hack the NPC's instructions /s

Assuming this "AI NPC" is a functionally useless jelly blob that says jelly-blob things on occasion, "smoke and mirrors" may be good enough. I don't think it's supposed to be gameplay-driving or deep, just amusing.