this post was submitted on 16 Dec 2025
20 points (83.3% liked)
Technology
1383 readers
69 users here now
A tech news sub for communists
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It would probably be wisest to use a pretrained model and feed it the prolewiki with particular focus on marxist leninist writers as to give it the correct bias.
mixture-of-experts but the experts are the five heads.
this is technically possible already, just that it takes so much resources that not many people could make this model and the cost would be prohibitive for what is a hobby project. This is why China is developing forces of production first.
It might be cheap enough to make a LoRA actually. It would still probably take a week or so, but doable. :)
we have to do this actually
https://github.com/percy-raskova/marxists.org-rag-db
It is in progress, if you'd like to help develop this let me know!
i love this idea but i must be honest: fine-tuning a mistral model to specifically embed the weights with the "voice" of Marx Engels Lenin Stalin and Mao would be an immense machine learning and engineering task. Now Abliterated DeepSeek 7B (abliterated means uncensored) = true power....