DPIFrame speeds up CTR inference 5.8x over PyTorch on GPU

Deep learning has boosted CTR prediction accuracy, but deploying these models on GPUs for efficient inference remains difficult due to the gap between serial computational patterns and parallel model structures. A team of researchers from academia and industry introduces DPIFrame, the first dual-parallelizable framework specifically designed for CTR model inference on GPU. The framework tackles the bottleneck through three key innovations: a dual parallelizable architecture that enables both intra-module and inter-module parallelism, an efficient multi-table lookup algorithm that anticipates the full workload to optimize embedding operations, and a breadth-first stream scheduling strategy for fine-grained parallel computation management on GPU.

Extensive experiments on two real-world datasets show DPIFrame dramatically reduces embedding latency by 23x compared to PyTorch. In end-to-end inference, it achieves state-of-the-art performance with speedups of 5.83x over PyTorch, 4.29x over TorchRec, 2.15x over HugeCTR, and 2.0x over OneFlow. The results highlight that DPIFrame's dual-level parallelism and smart scheduling can unlock significant performance gains for one of the most critical tasks in advertising and recommendation systems, making it a practical solution for production deployment.