Free Tool That Runs AI on Your Mac Just Got Faster

Most AI chatbots live on giant company servers. You type, your words travel across the internet, and answers come back. llama.cpp is different: it's a free, open-source program that runs AI models directly on your own laptop or phone. Nothing you type leaves your device, there's no subscription, and you can use it on a plane with no Wi-Fi. That's why it has over 130,000 "stars" — the web's version of a five-star review — from fans around the world.

This update, labeled b11184, improves how the tool works on Apple hardware. Specifically, it teaches Apple's graphics chip to handle a particular kind of number-crunching at larger sizes than before. Think of it like a mixer that can now blend bigger batches at once instead of stopping at smaller ones. Newer, leaner AI models increasingly rely on this blending trick to run efficiently on modest machines, so supporting bigger batches means your Mac can handle models it couldn't before. The team also added a safety check so weaker devices politely decline the job instead of crashing.

Why should you care if you don't own Apple stock or write code? Two reasons. First, privacy: local AI means your notes, emails, and questions never touch someone else's server. Second, money: no per-month fee and no cloud bill. As this software matures, the dream of a genuinely capable, private AI assistant living on your laptop gets closer — and improvements like this are the small bricks that build it.

The catch is that this is a pre-release, meaning a testing build not meant for everyone. It only applies to Apple Silicon Macs and iPhones, only to a narrow slice of the AI math pipeline, and the developers report that tests pass — not that anything is dramatically faster. If you're a casual user, you probably won't feel a difference today. If you're an enthusiast running local models, it quietly unlocks a bit more capability.