Small Business Builds Private AI Server for Sensitive Data

A small business owner has built a custom AI server using four high-end graphics cards for about $16,000 (Canadian dollars). The goal: run a powerful AI model entirely in-house, so sensitive customer data never touches the internet. For businesses that handle confidential information, this is a game-changer—it means they can finally use AI without the risk of data leaks.

The server, built with second-hand parts and a massive case, can handle up to eight users at the same time, each working with huge documents (up to 128,000 words). Response times are impressive: when eight people are using it, each still gets answers in under 2.5 seconds. Even in the worst case—eight people all submitting massive documents at once—the first response takes about 2.5 minutes, which is acceptable for such heavy use.

What's remarkable is the cost. Just a few years ago, this kind of computing power would have cost hundreds of thousands of dollars. The owner used four AMD Radeon AI PRO R9700 cards, each limited to 210 watts, and the system is quieter than a household air purifier. The whole setup fits under a desk and uses standard components.

This isn't just a hobbyist's toy—it's a practical solution for businesses that deal with sensitive data like medical records, legal documents, or financial information. By running AI locally, they avoid cloud subscription fees and data privacy concerns. The owner plans to use it for their business, replacing a smaller setup that was already impressive. This trend of affordable private AI could democratize access for small companies that previously couldn't afford such capabilities.