Free AI Tool for Macs Just Got Better at Saving Memory

Most people talk to AI through a website, which means their words travel to a company's servers and back. llama.cpp is the best-known alternative: a free, open-source program that downloads an AI model and runs it entirely on your own machine. No subscription, no internet connection needed, and nothing you type leaves your laptop. It's one of the most popular projects of its kind, with about 130,000 "stars" — the equivalent of likes on GitHub, where code lives.

This week's update is small and specific. On Apple devices, llama.cpp uses Metal, Apple's system for making graphics chips do heavy computing work. When the program moved data around during that work, it briefly set aside chunks of memory and didn't always hand them back right away. The fix releases those temporary buffers promptly. In practice, that means less memory sitting idle, which helps larger AI models fit and run without stuttering or crashing.

Why Apple specifically? Newer Macs, iPhones and iPads use chips where the main processor and graphics processor share one pool of memory. That design is great for AI, because there's no copying data back and forth — so every bit of memory you free up goes straight toward running a smarter model. The release includes builds for macOS, iOS, Android, Windows and Linux, so the same improvement reaches phones and tablets too.

The catch: this is a plumbing fix, not a new ability. If you don't already run llama.cpp, you won't notice anything — and setting it up still takes some technical patience. Think of it as the engine getting slightly more efficient. Most casual users will experience the benefit indirectly, through friendlier apps built on top of it, like private chat programs that work offline.