Tiny AI Model Fits in Your Pocket and Runs on a Laptop

A developer just built a tiny AI model that works on a regular laptop without needing a powerful graphics card or cloud computing. The model is so small it fits in 60MB — about the size of a short MP3 file — and can search through a massive digital archive to find answers.

It works by compressing information to just 1 bit per token when storing old chat history on your hard drive, but keeps the most recent 2,048 tokens in memory for quick access. This means you can ask it questions about things it "remembered" from earlier in the conversation, even though the whole model runs locally on your machine. For example, it correctly recalled a device serial number buried 50 million tokens deep in its archive.

The creator trained this 250-million parameter model on 30 billion tokens of text (about the size of 30,000 books) using a unique compressed vocabulary system where every word is represented by a fixed 512-bit code. This approach scored surprisingly well on human word similarity tests, beating random codes by a huge margin. The whole project, including code and demo, is open-source and available for anyone to use or improve.

While it’s not as powerful as big AI models like ChatGPT, it’s a breakthrough in making AI usable without relying on expensive servers or the internet. That means faster responses, no subscription fees, and more privacy — your data stays on your device.