Update Makes Home-Run AI 100x Faster at Restoring Chat History

Llama.cpp is a free, open-source tool that lets people run AI chat assistants on their own computers or phones, instead of sending their data to companies like OpenAI or Google. You get privacy, no monthly fees, and offline access. But there was a problem: if you closed the app and came back to a long conversation, the AI had to reload its "memory" of that chat. For a big conversation, that could take a full minute or more, which made the whole experience clunky.

The latest update speeds up that memory-restoring process enormously. The old system made over a million tiny copies of small pieces of data, one at a time, which took anywhere from 25 to 63 seconds. The new approach groups those pieces into fewer, larger blocks. For a 40,000-token conversation, it now takes just 221 to 424 milliseconds — roughly a tenth of a second. That's about a hundred times faster, and it feels instant to the user.

The developers also fixed a subtle bug that could cause the software to crash when restoring a conversation in certain situations. The fix makes the update more reliable, and they added tests to ensure the restored memory is byte-for-byte identical to what was saved. So the AI remembers exactly what was said, no details lost.

Why should you care? This makes running AI locally a much smoother experience. Whether you're using an AI journal, a coding assistant, or a chat tool that respects your privacy, you no longer have to wait or put up with awkward pauses. It's one more reason to use AI on your own terms — faster, safer, and completely private.