Free AI on Your Mac Just Got Faster — No Cloud Required

There's a free piece of software called llama.cpp that does something quietly revolutionary: it lets you run AI chatbots on your own laptop or desktop, with no monthly fee and no company watching your conversations. It's the plumbing behind a small boom in "local AI" — AI that lives on your machine instead of a distant data center. This week it shipped an update aimed squarely at Apple Macs.

Here's the problem it fixed. Macs have special chips (called Metal) built for heavy graphics and AI math. But one specific piece of code, used to prepare AI calculations, had the number "4" baked in permanently. If an AI model needed any other number, the Mac threw up its hands and sent the work to the regular processor instead — much slower, like asking a sports car to take the bus lane.

That mattered because a new AI model, Kimi-K3, needs that number to change as it works through its layers — sometimes small, sometimes large. The developers rewrote the code so the Mac chip can handle any number, then tested it with the values 1, 2, 3, 5, 8 and even 65.

The result is unglamorous but real: fewer stalls, smoother performance, and one less reason for Mac owners to give up on running serious AI locally. The catch is that this is a pre-release version, and the benefit only shows up if you're running that particular model with that particular setup. For most people, this is a repair to the road, not a new destination — but it's the kind of repair that keeps free, private AI moving forward.