New Tool Predicts AI Training Traffic Jams Before They Waste Millions

Training a modern AI model is a team sport. Thousands of specialised chips called GPUs work side by side, constantly passing data back and forth through network cables. When several training jobs run at the same time, they compete for the same cables — a digital traffic jam researchers call "network contention." Which job runs first, and next to which other job, changes how badly everyone slows down. Until now, planning these jobs was mostly guesswork.

The usual way to test scheduling choices is simulation rather than real hardware, because renting thousands of GPUs is expensive. But existing simulators either ignore network contention completely or slap on a fixed penalty. The authors show this leads to big errors — up to 73.64% off when predicting average job completion time. That's like a traffic app that assumes motorways are always empty, then tells you a cross-country drive will take four hours.

Their new tool, MoSim, models the jams properly. It measures each job's computing time, networking time and data volume without needing any real GPUs, then uses the current worker assignment to estimate how shared network connections slow each job down. Compared with older simulators, MoSim cuts errors by up to 3.28 times on average job time, 7.79 times on the slowest jobs, and 8.48 times overall. Setting it up is also 44.6 times faster.

Why should you care? AI companies spend fortunes renting chip time and electricity. If they can predict bottlenecks before booking hardware, they can run more work on the same machines, waste less energy, and ship products faster — and those savings eventually reach users. The catch: it's a simulation, not reality, and its accuracy was measured in the authors' own experiments. Still, it's a step toward less wasteful, cheaper AI.