viable/strict/1774006048: [xpu] Update torch-xpu-ops commit pin (#177721)
The commit enables fused RMS norm forward/backward on Intel GPUs and adds key sparse tensor operations.
Intel's PyTorch team has pushed a significant update to the torch-xpu-ops library, marked by commit 9ea9b4e. The technical pull request (#177721) primarily enables fused RMS norm forward and backward operations on Intel XPU GPUs, a critical optimization for training modern LLMs like Meta's Llama family that use RMS normalization layers. This fused operation combines multiple computational steps into a single kernel, dramatically reducing memory bandwidth and improving training speed on Intel's discrete and integrated GPUs.
The update also adds support for several sparse tensor operations—specifically addmm, mm, bmm, and baddbmm on SparseCsrXPU—expanding the types of models and optimizations that can run efficiently on Intel hardware. A key technical challenge addressed was avoiding symbol collisions in the Ahead-of-Time (AOT) compilation pipeline; the engineers had to register the fused RMS norm operations upstream in PyTorch core rather than in the torch-xpu-ops extension to prevent duplicate function definitions that would break compilation.
This integration represents a deeper merging of Intel's hardware-specific optimizations into the main PyTorch ecosystem, moving beyond simple backend support. The inclusion of a dedicated unit test confirms the operations are registered correctly and produce accurate results, ensuring reliability for developers building and training AI models on Intel's GPU platforms.
- Enables fused RMS norm forward/backward on Intel XPU GPUs, optimizing LLM training for layers used in models like Llama
- Adds sparse tensor ops (addmm, mm, bmm, baddbmm) on SparseCsrXPU, expanding supported model architectures
- Solves symbol collision issue by registering ops in PyTorch core instead of extension library to ensure clean compilation
Why It Matters
This directly accelerates LLM training on Intel GPUs, making them more competitive with NVIDIA's CUDA ecosystem for AI workloads.