SPDP Pruning Framework Boosts LLM Inference Up to 2.5x on GPUs

The computational and memory bottlenecks of autoregressive decoding in large language models (LLMs) have long been a challenge. Low compute intensity and bandwidth-bound kernels dominate inference cost. Weight pruning is a promising solution, but existing methods are confined to either static pruning (SP) — which permanently removes redundant weights but lacks adaptivity — or dynamic pruning (DP) — which adapts to input sparsity but introduces runtime irregularity. To address this, researchers Jinhyeok Kim, Yejoon Lee, and Jaeyoung Do propose SPDP, a unified sparse-inference framework that integrates unstructured SP with input-adaptive DP. The framework co-designs a new Tiled-Column-wise Bitmap Compressed (Tiled-CBC) format and two complementary GPU kernels: a CUDA-core spMspV kernel featuring Hybrid Activation-aware Dynamic Shared-Memory Bitmap Decoding (HAD-SMBD) for fine-grained runtime activation skipping, and a Tensor-Core SpMM kernel optimized for prefill computation. This joint design harmonizes static and dynamic sparsity, maintaining bandwidth-efficient memory access and high compute intensity across both phases of LLM inference.

Comprehensive evaluations on inference-optimized GPUs demonstrate that SPDP achieves a 1.24x-1.37x average speedup (up to 2.51x) over state-of-the-art sparse frameworks such as SpInfer, while matching perplexity with up to 25% higher sparsity. This represents a significant advancement in the inference efficiency-quality Pareto frontier, offering substantial throughput and performance-per-watt improvements for large-scale LLM serving. The paper has been accepted for publication in the Proceedings of the VLDB Endowment (PVLDB), Volume 19, Issue 11, 2026. The authors have also released code for reproducibility. SPDP's unified approach could enable more efficient deployment of LLMs in production environments, reducing costs and energy consumption without sacrificing model quality.