Free AI That Runs on Your Laptop Just Got a Key Fix

Llama.cpp is a piece of free, open-source software with a slightly awkward name and a big following — roughly 128,000 people have starred it online. Its whole purpose is to let you run AI chatbots on hardware you already own, like a MacBook, a Windows PC, or an Android phone, without paying a subscription or sending your conversations to a company's servers. On September 11, the team released a small numbered update, b10906, that fixes one specific glitch.

The glitch involves a clever speed trick called "speculative decoding." Think of it like a fast typist guessing ahead and a slower editor checking the work. The typist races to predict the next words, and the editor approves or corrects them. That makes the AI feel much quicker. The bug appeared when you mixed in an image: the AI would lose track of its position and start guessing from the wrong spot, producing confused or slower output.

The fix is a one-line-ish change with a clear idea behind it. Software measures conversations in "tokens," small chunks of text, but an image doesn't count like words do. The old code passed along a token count when it should have passed along an actual position. Now it sends the right marker. The developer notes this affected every drafting system, not just the one called DFlash — so plenty of users will feel the difference.

The honest catch: this is a tiny patch, not a headline-grabbing launch. There's no polished app here. Most people use llama.cpp through third-party programs, and getting it running still takes a bit of patience. But it quietly keeps free, private AI on your own device working the way it should.