Free Offline AI on Your Mac Just Got More Accurate

There's a free, open-source project called llama.cpp that lets you run AI chatbots directly on your own computer — your Mac, your Windows PC, even your phone. No monthly fee. No internet required. Nothing you type gets sent to a faraway server. It's one of the most popular tools of its kind, with about 129,000 people having starred it on GitHub, the website where code projects live.

On 22 September, the team released a small update, build b11093. The fix targets a specific piece of machinery: something called "flash attention." Think of flash attention as a memory-saving trick that lets an AI read very long documents without running out of room. Inside that trick there's an invisible map telling the AI which words it's allowed to focus on. That map's edges were slightly wrong on Apple's chips, which are used in Macs and iPhones. The update corrects those edges.

Why does a tiny fix like this matter to you? Because when that map is off, the AI can quietly misread part of what you pasted in — a contract, a long email chain, a chapter of a book — and give you a confident but slightly wrong answer. Nothing crashes. Nothing warns you. You just get a worse answer. This update makes long-text handling on Apple hardware more dependable.

The honest catch: this is a maintenance release, not a leap forward. If you don't already run AI models on your own device, today changes nothing for you. And if you do, the improvement is subtle — it shows up in edge cases with long inputs, not in dramatic new abilities. But it's a good reminder of something quietly important: capable AI is increasingly something you can own outright, run offline, and keep private, rather than rent by the month.