Free Local AI Just Got Faster on Your Own Laptop

There's a free piece of software called llama.cpp that lets you run AI chatbots on your own computer instead of paying a company like OpenAI or Google every month. It's the engine behind many private AI apps on Macs, Windows PCs and even phones. This week its developers pushed out a new version, build b10984, and the main change is about making things run faster.

Specifically, they improved how the software handles two common math routines — think of them as the repetitive arithmetic the AI does billions of times to answer a question. The new code lets those routines work on data laid out in a more natural way, which means less shuffling and copying behind the scenes. The biggest winner is Nvidia CUDA (the software that lets AI use Nvidia gaming graphics cards), so if you have a GeForce card in your desktop, this update is aimed partly at you.

The release also ships ready-made versions for a huge range of machines: Macs with Apple Silicon, plain Intel and AMD laptops, Android phones, and even specialized Chinese server chips. Basically, the project keeps widening who can run AI locally without buying anything extra.

The catch is that this is a small, technical improvement — not a dramatic leap. You likely won't notice the difference unless you're already running local AI models, and everyday users still need some patience to set this up compared with clicking a ChatGPT link. But it's one more step toward capable AI that lives on your device, works offline, and keeps your conversations to yourself.