Rudder: Steering Prefetching in Distributed GNN Training using LLM Agents

A research team including Aishwarya Sarkar, Sayan Ghosh, and others has introduced Rudder, a novel software module integrated into the AWS DistDGL framework that tackles a major bottleneck in large-scale Graph Neural Network (GNN) training. Training GNNs on massive, distributed graphs requires frequent and irregular communication to fetch neighboring node data, which often stalls computational progress. Traditional static prefetching methods fail because the required data changes dynamically with graph structure, distribution, and sampling parameters. Rudder's breakthrough is using Large Language Model (LLM) agents to autonomously predict and prefetch this remote data, leveraging the models' emergent in-context learning and logical reasoning capabilities for adaptive control, even without extensive task-specific training.

Evaluated on the NERSC Perlmutter supercomputer with standard datasets, Rudder demonstrated remarkable performance gains. It achieved up to a 91% improvement in end-to-end training time compared to the baseline DistDGL with no prefetching, and an 82% improvement over static prefetching methods. Crucially, it reduced overall communication volume by over 50%, directly addressing the core inefficiency. This work, accepted to the ACM International Conference on Supercomputing (ICS 2026), represents a significant convergence of generative AI and high-performance computing. It shows that LLMs' reasoning can be effectively repurposed for real-time system optimization, paving the way for more intelligent and adaptive distributed computing frameworks that can self-optimize based on changing workloads.