AMD's New AI Speed Boost Makes Your Chatbot Replies Faster

AMD has switched on a new feature for its top AI chips, the MI300X and MI355X, that speeds up how they run large language models like those behind chatbots. The trick, called AsyncTP, lets the chip gather data and do math at the same time instead of one after the other. Think of it like a chef who preps ingredients while the oven preheats—dinner gets to the table faster. This means AI apps can reply more quickly and cost less to run, which could show up as lower prices or better free tiers for you.

The upgrade works by splitting the AI's work into chunks and using a special scheduler to process each chunk as soon as it's ready. AMD had to write custom code to make this work on their hardware, since the original version was built for Nvidia chips. They also fixed a rounding issue so results stay accurate—important if you're using AI for math or coding. However, the speed boost only applies to AMD's latest chips, so older hardware won't benefit.

One catch: the new method requires careful tuning to avoid deadlocks, where the chip gets stuck waiting for data. AMD engineers had to reserve a small part of the chip to prevent this, which slightly reduces the number of blocks available for computation. Also, the speedup varies: on the MI300X, turning off a certain setting made the math 14-32% slower, but on the MI355X it was 1.5% faster, showing that performance gains aren't uniform across chips.

For everyday users, this means AI services running on AMD hardware could become more responsive and cheaper to operate. Companies might pass those savings on to you through lower subscription fees or faster free tiers. But since most AI today runs on Nvidia, the impact will be gradual as more providers adopt AMD's chips. Still, it's a step toward more competition in the AI chip market, which is good news for your wallet.