Gemma-4-26B-A4B runs 7 T/s on budget CPUs
Old desktops now run advanced LLMs without a GPU.
Deep Dive
An old i5-8500 desktop with 32GB RAM and no GPU runs the Gemma-4-26B-A4B model at approximately 7 T/s using Koboldcpp on Linux—a setup that cost $150 on eBay. The user says this smaller model flies on CPU alone, making a GPU purchase seem unnecessary.
Key Points
- Gemma-4-26B-A4B runs at 7 T/s on an i5-8500 CPU.
- No GPU required, enabling use on budget desktops.
- Accessible AI technology for experimentation and innovation.
Why It Matters
Democratizes access to advanced AI, reducing hardware costs for professionals.