Aergia: Leveraging Heterogeneity in Federated Learning Systems

Researchers Bart Cox, Lydia Y. Chen, and Jérémie Decouchant have developed Aergia, a novel federated learning system that tackles the persistent problem of client heterogeneity in distributed AI training. Traditional federated learning approaches like FedAvg suffer from performance bottlenecks when clients have varying computational power and network capabilities, often resorting to deadlines that ignore slow clients' updates. Aergia introduces a smarter solution where slower clients can freeze the most computationally intensive parts of their model and offload that training to faster clients in the network.

The system operates through a three-step process: slow clients identify and freeze resource-intensive model components, train the remaining unfrozen parts locally, and delegate the frozen components to faster peers who train them using their own datasets. A central federator orchestrates these offloading decisions based on clients' reported training speeds and privately evaluated dataset similarities, using trusted execution environments to maintain privacy. This approach fundamentally changes how federated systems handle heterogeneity, turning slower clients from bottlenecks into participants that can contribute meaningfully while receiving computational assistance.

Extensive experiments demonstrate Aergia's effectiveness, showing training time reductions of 27% compared to FedAvg and 53% compared to TiFL (a deadline-based approach) while maintaining comparable model accuracy. The system was presented at the 23rd ACM/IFIP International Middleware Conference and represents a significant advancement in making federated learning more practical for real-world deployments where client devices inevitably vary in capability. By enabling computational resource sharing among clients, Aergia moves federated learning closer to its promise of efficient, privacy-preserving distributed AI without sacrificing performance.