Free AI You Can Run on Your Own Mac Just Got Faster

There's a quiet alternative to the AI you pay for every month: free, open-source software called llama.cpp that runs chatbots directly on your own laptop or phone. No account. No monthly bill. Nothing you type gets sent to a company's servers. On September 24, its volunteer developers shipped a small update (numbered b11172) that makes it noticeably faster on Apple hardware — Macs, iPads and iPhones.

The improvement is technical but easy to picture. AI models have a memory step where they decide which parts of your conversation actually matter, similar to skimming a long document instead of rereading every page. This update makes that skim faster on Apple's chips (called "Metal" in developer speak) by keeping the important bits in a small, super-fast scratchpad instead of fetching them from slower memory. Less fetching, faster replies.

Why should you care if you don't code? Because llama.cpp is the engine underneath many friendly apps — the offline chatbot apps, the private note-takers, the tools that summarize documents without uploading them anywhere. When the engine gets faster, those apps get faster too, usually within weeks. It also matters because faster local AI means less need for cloud subscriptions. If a free app on your own machine is quick enough, the $20-a-month habit starts looking optional.

The honest catch: this is a developer release, not a polished product. There's no installer, no setup wizard, and it's aimed mainly at Apple devices. If you want to try local AI today, look for a finished app built on llama.cpp rather than downloading this yourself. And "faster" here is a modest step, not a transformation — expect smoother, not dramatically different.