When I was sick with Covid in April I built Turbopilot, a weekend hack to get code autocomplete models (like GitHub Autopilot) to run locally on low spec machines using the library behind llama.cpp. The models I used were codegen from salesforce and the idea is that if you're running these models locally it's free and you're not sending your source code back to the microsoft/github mothership.

Since then I've not really had time to work on it very much as my day job has been pretty busy but I really want to carry on development. I've got experimental nvidia acceleration building and I'm working on shipping a windows version at the moment.

BTW If anyone is interested, I'm looking for some help and I'm willing to offer some technical mentoring (I have a background in AI/ML and a dozen years exp doing software engineering professionally)