GriNNder enables single-GPU full-graph GNN training at 9.78x speedup

Full-graph training of graph neural networks (GNNs) preserves complete neighborhood information but typically requires multiple GPUs or servers, incurring high hardware and communication costs. Existing single-server methods remain bottlenecked by GPU and host memory as graph sizes grow. GriNNder, developed by researchers at (presumably Korean institutions), is the first work to break this memory wall by offloading data to NVMe SSDs, which offer multi-terabyte capacities and bandwidths exceeding 10 GB/s. The key innovation is structured storage offloading (SSO), a framework that manages the GPU-host-storage hierarchy through coordinated cache, regather, and bypass mechanisms.

GriNNder implements three core techniques: (i) a partition-wise caching strategy for host memory that exploits cross-partition dependencies, (ii) a regathering strategy for gradient computation that eliminates redundant storage operations, and (iii) a lightweight partitioning scheme that reduces the memory footprint of existing graph partitioners. In experiments across various models and datasets, GriNNder achieves up to 9.78x speedup over state-of-the-art baselines and delivers throughput comparable to distributed systems. This makes previously infeasible large-scale full-graph training possible on a single GPU. The paper has been accepted to MLSys 2026.