New Software Makes AI Training Up to 88% Faster

Training a modern AI model isn't done on one computer. It's spread across thousands of specialised chips called GPUs (the number-crunchers behind AI), all working together like a huge team building one house. The problem is coordination. Those chips spend an enormous amount of time simply passing information back and forth, and while they wait, nothing gets built. Researchers estimate this waiting — not the actual thinking — is now one of the biggest bottlenecks in AI.

A team including researchers from Berkeley's Sky Computing Lab has released mKernel, a piece of software that attacks this waiting problem directly. The trick is simple in concept: instead of a chip finishing all its work and then sending results, mKernel sends each small piece of finished data the moment it's ready, while the chip keeps computing. It also splits each chip's internal workers into two teams — some doing math, some handling messages — and automatically adjusts that split depending on the job, because the ideal balance changes from task to task.

On two clusters of 16 top-of-the-line H200 chips, the results were striking: up to 1.72x faster on a common AI operation and 1.88x faster on Ring Attention, a technique used to process very long documents. In practical terms, a training run that took ten days could finish in roughly five and a half. The team also found something unexpected — a fancier, more expensive networking approach barely helped, which means the same speedups may be achievable on cheaper cloud setups.

Why should you care? AI companies pay for every hour of chip time, and those costs get passed to you as subscription fees, per-query charges, and usage limits. Anything that nearly halves training time pushes in the opposite direction. It also means models that handle long documents, videos, or complex reasoning — the expensive stuff — become more practical to build, and less electricity gets burned to get there.