Ollama's New Fix Stops AI Models From Crashing Your Mac

Ollama is a free tool that lets you run AI chatbots on your own laptop or desktop instead of paying a monthly subscription to a cloud company. Your words never leave your machine. But it had an annoying flaw: the longer the AI talked, the hungrier it got for memory (the short-term workspace your computer uses for whatever it's doing right now).

The culprit was a speed trick called "speculative decoding," where the AI guesses several words ahead and checks them all at once, like a student skipping to the end of a math problem. Fast, yes — but every time the AI's memory needed to grow, it left old chunks behind. The cleanup code only triggered on an exact number, and the fast mode usually skipped right past it. So the leftovers piled up, several gigabytes at a time, with nothing able to reuse them.

In practice, this meant that if you gave the AI a long document to work with — say 98,000 words of context — and asked for a lengthy answer, the memory use would climb and climb. On a 128GB Mac it blew past 90GB and crashed the entire operating system, forcing a hard restart and losing whatever you were doing. That's the kind of thing that makes people give up on a tool.

The fix is small but meaningful: the cleanup now runs whenever the AI crosses that memory threshold, even if it skips over it. In testing with a large model on that same 128GB machine, memory use stayed flat at 30GB instead of spiraling past 90GB. For anyone running AI locally, that means longer chats, bigger documents, and far fewer crashes.