Free Tool Lets Your Laptop Run AI Without the Cloud

llama.cpp is a free, open-source program that lets ordinary devices — laptops, phones, even web browsers — run AI models on their own, instead of paying a company to do it in the cloud. It's wildly popular with hobbyists and tinkerers, with roughly 130,000 people bookmarking it on GitHub, and it quietly powers apps like LM Studio that many non-engineers already use.

This update adds something called bfloat16 support to WebGPU. In plain English: AI does enormous amounts of math, and bfloat16 is a leaner way to store numbers so the math stays accurate while using less memory and power. WebGPU is simply a way for browsers and apps to borrow your computer's graphics chip — the part normally used for games — to do that AI math. Together, the change means AI running inside a browser or app can work on more devices and produce slightly better results.

Why should you care? Because this is the plumbing behind running AI privately and cheaply. When AI runs on your own machine, your documents and messages never leave your laptop, and there's no subscription. The release also ships updated versions for Mac, Windows, Linux, Android, and iOS, so the same free software keeps improving across the gadgets people actually own.

The honest catch: this is a pre-release, and a small one. It's one incremental commit in a fast-moving project, not a headline-grabbing launch, and many users won't feel any difference. Local AI is also still generally less capable than the biggest cloud models, and setting it up can be fiddly. Think of it as another quiet brick in the wall — not a new front door.