PyTorch Just Made AI Training Nearly Twice as Fast on Gaming GPUs

PyTorch, the free software toolkit behind most modern AI, quietly fixed one of the most repetitive chores in AI computing. Whenever an AI model runs, it constantly rearranges numbers stored in its graphics card's memory — a bit like flipping a spreadsheet from rows to columns. That flipping, called a transpose, used to crawl. A new piece of code now does it up to 1.8 times faster.

The change matters most to people with ordinary gaming computers rather than giant data centers. On Nvidia's RTX 4060, a card many hobbyists own, the speed went from 127 to 225 gigabytes per second — essentially hitting the card's physical ceiling. That means local AI tools, chatbots running on your own laptop, and image generators all get faster for free, without buying anything new.

The trick was clever but simple in spirit. Instead of moving numbers one at a time, the new code moves them in big chunks, uses a small scratchpad on the chip to flip them all at once, and skips the fancy shortcuts when a card or data shape can't use them. The developers also noticed an unnecessary safety check was costing 10-48 percent on Nvidia's newest chips, so they made it switch on only when actually needed.

The catch: this is plumbing, not a new feature. It won't make AI smarter, and the gains shrink on high-end data center chips that were already near their memory limits. Most people also won't notice immediately — you'll see it only once PyTorch ships the update and the apps you use rebuild on top of it, which usually takes weeks or months.