Local LLM server for $6.4k pays for itself in under 2 years vs API

A developer shared the total cost of ownership (TCO) for a local LLM server built with four used AMD MI100 32GB GPUs, an ASRock EPYCD8-2T motherboard, a 1600W power supply, 64GB DDR4 ECC RAM, and an Epyc 7K62 48-core CPU. Total hardware cost: $6,406.45. The server runs four instances of llama.cpp serving Qwen 3.6 27B on Ubuntu with ROCm, processing 20.4M input tokens and 1.32M output tokens per day—all used for a real business process.

At OpenRouter rates ($0.29/M input, $3.2/M output), the daily token value is $10.14, or $3,701 per year. The hardware essentially pays for itself in under two years, especially since the builder properly accounted for depreciation (hardware actually holds value). A comparison with Z.AI's best coding plan ($144/month) shows it delivers only 4.5M input and 200k output tokens per day of a comparable model, costing $652.80 per month for the same capacity—more than double the API cost. The takeaway: local LLM servers can be cost-effective for high-volume inference, but coding subscriptions aren't always a bargain.