AMD GPUs Get Faster AI with New PyTorch Backend

AMD has introduced a new backend called FlyDSL that makes PyTorch—the popular AI software—run much faster on its MI350-series GPUs. This means if you're using AMD hardware for AI, your models can train and respond quicker, saving you time and cloud computing costs. The speedups are significant: up to 1.68x for certain matrix operations, 4.82x for a common data-sorting task, and even 3x faster overall for some AI models.

The best part? You don't need to rewrite your code. FlyDSL plugs into existing PyTorch functions, so your current AI programs automatically benefit when running on supported AMD GPUs. It works with common operations like matrix multiplications, attention mechanisms (used in chatbots), and normalization layers. If a particular operation isn't supported, PyTorch falls back to its standard implementation, so nothing breaks.

These improvements matter most for AI models where these operations dominate the runtime—like large language models or image recognition systems. For example, in tests with vLLM (a popular tool for serving AI models), whole-request speedups reached up to 3x for certain data formats. This could translate to lower costs for AI companies and faster responses for users.

The catch: FlyDSL currently only works on AMD's latest MI350 GPUs with the gfx950 architecture, so older hardware won't see these gains. Also, some operations like RMSNorm backward still use the old implementation. But for those with the right hardware, it's a free performance boost that requires no code changes.