AI

6427 readers

13 users here now

Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.

founded 5 years ago

Run Qwen3.6 MTP GGUFs locally ~1.4–2.2× faster with no accuracy loss and with only 18gb VRAM (huggingface.co)

submitted 1 day ago by yogthos@lemmy.ml to c/artificial_intel@lemmy.ml

6 comments fedilink hide all child comments

The change is a result of MTP support landing in llama.cpp. The Qwen3.6 Unsloth GGUFs are now out of experimental mode, with llama.cpp has merged many PRs, and MTP is now properly supported in Unsloth.

https://unsloth.ai/docs/models/qwen3.6#mtp-guide

you are viewing a single comment's thread
view the rest of the comments

[–] yogthos@lemmy.ml 3 points 1 day ago (1 children)

I find it's the first model I can run locally that actually feels genuinely useful for coding. I'm really excited about what things might look like in another year. We might really get current frontier model performance on a laptop at this rate.

[–] kmartburrito@lemmy.world 2 points 15 hours ago

Unfortunately the new MTP model doesn't work yet with my LM Studio + Claudish + Claude Code setup. Will need to wait for support to be merged into the lm Studio ROCm llama.cpp runtime.