Gemma 4 running on Raspberry Pi5

A significant technical demonstration shows Google's Gemma 2, a state-of-the-art 9 billion parameter language model, running natively on a Raspberry Pi 5. The setup uses the 8GB RAM variant of the popular $80 single-board computer, with performance noted as consistent whether using an SSD or standard storage. The model deployed is the 'Gemma 2 e2b' variant, specifically optimized by Unsloth for efficiency, and is executed using the latest development branch of the llama.cpp inference engine on the lightweight Potato OS.

This achievement is a milestone for the edge AI and maker communities. It proves that compact, cost-effective hardware can now support the same class of AI models that recently required data center GPUs. The use of llama.cpp, a project known for quantizing models to run on limited resources, was key to making the 9B parameter model fit and run within the Pi's memory constraints. This opens the door for truly local, private, and low-latency AI applications—from smart home hubs and educational tools to portable coding assistants—all running on a device the size of a credit card.