Two old P102 GPUs for $100 run a 35B AI model with 3 users
20GB VRAM and 448GB/s bandwidth for just $100 per card pair.
Deep Dive
A maker used two NVIDIA P102-100 GPUs (mining cards) costing ~$100 total to run the Qwen3.6-35B-A3B quantized model via llama.cpp. The setup supports 3 concurrent users with 32K context each. Performance rivals cards costing 4x more, making ultra-cheap AI inference accessible to hobbyists.
Key Points
- Two NVIDIA P102-100 GPUs with 10GB VRAM each cost ~$100 total, providing 20GB VRAM and 448GB/s bandwidth.
- Runs Qwen3.6-35B-A3B (quantized) via llama.cpp, supporting 3 simultaneous users with 32K context each.
- Performance rivals GPUs costing 4x more, though context length is limited to 32K vs. model's max 262K.
Why It Matters
Makes capable local AI inference affordable for hobbyists and small teams using cheap used hardware.