Amazon's New AI Trick Trains Models Without Moving Massive Data

Training giant AI models takes two things that don't always live together: massive computing power and massive datasets. Usually, companies have to pick between copying petabytes of data to where the computers are (expensive and slow) or letting the computers reach across long distances to grab data (also slow). Amazon Web Services and Qumulo say they've solved that trade-off. Their system lets a training cluster in one location pull data from another location without any noticeable slowdown, and without making an extra copy.

The magic is something called NeuralCache. Think of it like a really smart librarian who watches what books you keep asking for and starts pre-stacking them on your desk before you even ask. It learns the patterns of what data the AI model needs next and pre-loads it nearby. After a short warmup period — during which the system figures out your patterns — the remote cluster performed just as well as one sitting right next to the data, hitting the same 115 to 117 samples per second.

In real terms, that means GPU utilization — how hard the expensive AI chips are working — stayed between 98 and 100 percent after warmup. The initial catch-up phase is tiny: under 1 percent of total training time for a typical large job. For companies training frontier AI models, this removes a major logistical headache and expense. They no longer have to physically duplicate enormous datasets across the globe just to keep their models current.

The bigger picture: as AI models keep growing, the data they learn from is becoming too large to casually copy around. Solutions like this make it practical to keep data in one place — whether for cost, security, or legal reasons — while still using compute wherever it's cheapest or most available. That flexibility could lower costs for AI companies, which in turn could mean cheaper and faster AI tools for the rest of us.