DeepSeek's New AI Fits on a Home PC — Here's What That Means
Run a top AI at home with no monthly cloud fees and full privacy.
Deep Dive
A hobbyist built a local AI inference machine—an Epyc 7663, 256GB of RAM, and an RTX 5090—and is running a ~151GB model (UD-Q8_K_XL) at 23.8–24.6 tokens per second. They were surprised it could run this well in their basement without spending $10k or adding heavy electrical work, and they shared the setup because few data points exist for DDR4 Epyc + Blackwell doing CPU-MoE. They're new to this and open to tips.
Key Points
- DeepSeek's latest model ran on a home PC at 24 words per second — usable for real conversation.
- It uses a 151GB AI model with a single RTX 5090 graphics card and a server CPU with 256GB RAM.
- Running AI locally means no monthly cloud fees and total privacy, but the upfront hardware cost is around $5,000.
Why It Matters
Cheaper local AI gives everyday users privacy, no subscriptions, and offline access to advanced assistants.