Free App Fix Lets Newest AI Models Run on Your Own Laptop

A free, volunteer-run software project called llama.cpp shipped a small update on September 17. The project is popular — about 129,000 people have starred it on GitHub, the site where programmers share code. This particular release fixes a technical bug: some of the newest AI models, including one family called Qwen 3.5 MoE, refused to load properly when they used a feature that lets the AI guess several words ahead at once. The fix sounds tiny, but without it, those models simply wouldn't start.

Why should you care? llama.cpp is the main tool that lets people run AI chatbots and assistants directly on their own computers — a Mac, a Windows PC, even an Android phone — instead of paying a company and sending every question to its servers. That means no $20 monthly subscription, no internet connection needed, and your notes, documents and messages never leave your machine. For anyone handling sensitive work, that privacy difference is the whole point.

The models in question are built with a 'mixture of experts' design. Think of a big company where only the relevant department answers your question, instead of the entire staff stopping work. That makes the AI faster and lighter, so it can fit on a laptop. The second feature, multi-token prediction, is like an assistant guessing the next few words of your sentence before you finish — the AI checks several possibilities at once rather than one at a time.

The honest catch: this is a pre-release test version, which means it's aimed at hobbyists and developers, not someone who just wants an app icon to tap. Running it still involves some setup. And the AI models you can run at home are noticeably less capable than the biggest cloud systems. Still, each release like this quietly closes the gap — and pushes free, private AI closer to something anyone can use.