New AI Speed Trick Makes Image Recognition Faster and Cheaper

Convolution is a major bottleneck in deep neural networks, and existing GPU methods often suffer from large memory overhead, poor cache utilization, limited effectiveness across kernel sizes, or numerical instability. This work extends the im2win approach—a memory-efficient convolution method with contiguous memory access for all kernel sizes—to run in full precision on CUDA cores and half precision on tensor cores. With optimizations like zig-zag memory access and asynchronous data movement, it leverages hardware-accelerated half-precision matrix operations. Across twelve CNN benchmarks, im2win achieves up to 2.8x higher TFLOPS than its CUDA core implementation, 1.4x higher than cuDNN, and 6.4x higher than GEMM-based convolution with cuBLAS, while using as little as 53% and 35% of their memory, respectively. This positions im2win as a unified, high-performance convolution framework for modern GPU architectures.