Hobbyist's Tiny AI Matches Bigger Models by Storing Facts on an SSD

A hobbyist researcher has created a small AI model that performs as well as a larger one by using an enormous lookup table stored on a standard SSD. The model itself has 21 million parameters, but it consults a table with 6.4 billion parameters—essentially a huge dictionary of learned facts. For each word it generates, it reads only a few hundred entries from the table, keeping the actual computation small. This approach matches the quality of a 114 million parameter dense model trained on the same data, but runs on a gaming PC with just 0.4 GB of video memory.

The key innovation is that the massive table doesn't need to sit in expensive video memory. Instead, it's stored on an NVMe SSD and memory-mapped, meaning the model can access it directly from disk. This allows the model to run at about 140 tokens per second on an AMD RX 9070 GPU. However, reading long prompts from the SSD is slow because each missed row costs a whole 4 KB page. The researcher also wrote custom software kernels that work on different GPUs, including AMD and Nvidia models.

Not everything worked. Trying to add this table to an existing model (Qwen3.5-0.8B) didn't improve performance. The model's outputs are fluent but factually incorrect—it writes made-up Wikipedia-style text. The project cost about $70 in cloud computing, and the researcher used AI assistance for coding. The code and a demo are publicly available, and the researcher hopes to test at a larger scale with better hardware.

This experiment shows that separating memory from computation could make AI more efficient and accessible. Instead of needing huge, expensive GPUs, future AI might rely on fast storage and smaller chips. While this is early-stage hobbyist work, it points toward cheaper, more energy-efficient AI that could run on everyday devices.