Free Tool That Runs AI on Your Own Laptop Just Got Faster

A popular piece of free software called llama.cpp released a small update on September 16, labelled version b11002. llama.cpp is an open-source program that lets you download AI models and run them directly on your own laptop or desktop — no monthly bill, no sending your questions to a company's servers. This particular update changes how the program rearranges image and number data (a step called im2col) so that NVIDIA and AMD graphics cards can chew through it more efficiently.

Why should you care? Because this is the quiet engine behind a big shift: AI that lives on your device instead of in someone's data centre. When a chatbot runs locally, your private notes, medical questions, or work documents never leave your machine. There's no $20-a-month subscription, no internet required, and no company reading over your shoulder. Every time tools like this get faster, that private, free option becomes more realistic for ordinary people — not just hobbyists.

The honest catch: this is a technical maintenance release, not a shiny product launch. You can't just download an app and start chatting — setting it up takes some comfort with command-line instructions, and you need a reasonably powerful computer with a decent graphics card. The AI models you can run at home are also generally smaller and less clever than the giant cloud versions. And it's flagged as a 'pre-release,' meaning it's a test build that may still have rough edges.

Still, the momentum is telling. The project has roughly 129,000 'stars' (GitHub's popularity marker) and 23,000 copies made by other developers, and each release adds support for more chips and operating systems. Meanwhile, Apple, Qualcomm and others are all pushing AI that runs on your phone or PC. The direction of travel is clear: the AI you use every day is slowly moving out of the cloud and onto the device in your pocket.