PyTorch Foundation projects: vLLM, DeepSpeed, and PyTorch 2.13 deliver major AI performance gains

The PyTorch Foundation's evolution into a multi-project organization in April 2025 is already bearing fruit, with four major projects posting significant updates. PyTorch 2.13 highlights Apple Silicon optimization: FlexAttention runs up to 12x faster than SDPA, and a new CuTeDSL Inductor backend plus nn.LinearCrossEntropyLoss reduces peak GPU memory by 4x. The release also adds Python 3.15 wheel support with free-threaded builds, torchcomms backend, FSDP2 communication overlap for large-cluster training, and broader ROCm/Arm/Intel XPU support. On-device LLMs get a boost via ExecuTorch, now with weekly examples and a Hugging Face partnership.

vLLM achieved a stable bi-weekly release cadence and redesigned Model Runner V2 for substantial GPTQ improvements. It supports cutting-edge models like Kimi K3, Minimax M3, and Qwen 3.8 day-zero. The Q3 2026 roadmap focuses on production agentic workloads through SIGs, with a redesigned scheduler and KV cache, better prefix caching for multi-turn agents, and faster speculative decoding. DeepSpeed delivered six releases, integrated Torch.xpu (replacing IPEX), and added Ulysses sequence parallelism to Hugging Face's Trainer, Accelerate, and TRL. It also won a best-paper honorable mention at ASPLOS 2026 for SuperOffload work. Ray's updates (though truncated) mention GB200/GB300 hardware support, rounding out a quarter of rapid, collaborative open-source AI progress.