The Free Tool That Runs AI on Your Own Computer Just Got More Reliable

llama.cpp is one of the most popular free programs for running AI language models on hardware you own — your laptop, a gaming PC, or a company server — instead of sending your data to OpenAI or Google. It has 130,000 stars on GitHub, a rough popularity score, and nearly 24,000 copies of the project made by other developers. This week a contributor from IBM pushed a small maintenance update to it.

The update targets a specific kind of machine: IBM Z, the massive mainframes that banks, insurers, and airlines have used for decades to process millions of transactions a day. These companies increasingly want to run AI on that same hardware, because the data is already there and never has to leave the building. IBM built a helper library called zDNN to make that possible, and this commit connects it more cleanly to llama.cpp.

So what actually changed? Two things, both invisible to users. First, the code can now properly "reset" a chunk of memory between AI tasks, so the machine starts fresh each time rather than dragging along leftovers. Second, it plugs memory leaks — the digital equivalent of a tap that never fully closes. Over hours or days of continuous use, leaks like that quietly eat up a system's memory until things get slow or crash entirely.

For a bank running AI around the clock, that difference is the gap between a system that works and one that needs a reboot every few days. For everyone else, it's a reminder that the tools making private, local AI possible are being hardened piece by piece — mostly by volunteers and corporate engineers fixing unglamorous plumbing. No headline feature arrived here, but the foundation got a little less leaky.