5090 vs dual 5060 16g - why isnt everyone going dual?

A viral discussion on the r/LocalLLaMA subreddit, sparked by user jzatopa, is challenging conventional wisdom for building AI inference rigs. The core argument is financial: two new NVIDIA RTX 5060 16GB graphics cards can be purchased for roughly $1100, providing a combined 32GB of VRAM. This is presented as a stark contrast to the anticipated high price of a single flagship RTX 5090, which is expected to carry a premium cost for its top-tier performance. For enthusiasts and developers running large language models (LLMs) locally, VRAM is the critical bottleneck, making this dual-card setup a theoretically compelling path to running models like Meta's Llama 3 70B parameter version.

However, the community response highlights significant technical hurdles that explain why this isn't a default strategy. The primary issue is software complexity and performance scaling. While the combined VRAM pool is attractive, effectively utilizing two GPUs for a single LLM inference task requires sophisticated model parallelism, which isn't always seamless. Frameworks like llama.cpp and Ollama support multi-GPU setups, but users often report less-than-linear performance scaling, increased latency, and higher power consumption compared to a single, more powerful card. Furthermore, the RTX 5060's memory bandwidth and core performance are lower tier, meaning that even with ample VRAM, inference speed (tokens per second) on a dual 5060 system may not match a single 5090, which will have a vastly more powerful GPU die and memory subsystem.