AI enthusiast's dual Inspur AGX-2 rig hits 512GB VRAM
One Reddit user's 16x V100 setup now has room for the largest LLMs
Deep Dive
A Reddit user, u/UltraFOV, shared that they now have a full 512GB of VRAM — but already think they’ll need a third one soon, since LLMs keep getting absurdly huge.
Key Points
- 16x NVIDIA V100 (32GB) GPUs across two Inspur AGX-2 servers
- 512GB total VRAM enough for 405B-class LLMs in quantized form
- User plans to add a third AGX-2 as open-source models continue to grow
Why It Matters
Shows a niche but growing demand for massive on-prem VRAM to run open-source LLMs at home.