A Mini PC Can Now Hold a Million Words in an AI's Memory

WHAT HAPPENED: A developer named peonist-ai fixed a problem where an AI setup got slower the more text you fed it. The fix is a free piece of software called halogen 0.12.0. It runs an open AI model on one small desktop computer instead of renting power from a cloud company.

The big number is one million "tokens" of context. A token is a small chunk of a word, so a million tokens is roughly 750,000 words — about ten full-length novels, held in the AI's memory at the same time. Speed improved from 27 to 38 tokens per second (roughly 20 to 30 words a second), and loading that giant pile of text dropped from about 21 minutes to 18. Once loaded, follow-up questions get their first word out in about half a second.

The computer doing this is a Ryzen AI Max+ 395 with 128 gigabytes of memory — a compact machine you can buy for around $2,000, not a data center. That matters because nothing leaves your desk. Lawyers, doctors, researchers, or anyone with confidential files could ask questions about hundreds of documents without sending them to a tech giant, and without a monthly bill.

THE CATCH: You wait about 18 minutes the first time you load a million words, and the speed is still far slower than what huge cloud systems deliver. It also needs a pricey, memory-heavy machine, and the results come from single test runs rather than wide testing. Today this is for hobbyists and tinkerers. But it is a clear sign that private, on-your-desk AI is getting real.