Stream Learning beats token-based gossip by 5.5% under 30% crashes

Gossip learning allows distributed nodes to train a shared model without a central coordinator by exchanging model partitions. The state-of-the-art protocol, Partitioned Token Gossip Learning (PTGL), uses fixed partitions, token-based fairness, and per-neighbor metadata to ensure balanced dissemination. In a new arXiv paper (arXiv:2608.06946) presented at SSS 2026, Fabien Mathieu, Alexandre Pham, Maria Gradinariu Potop-Butucaru, and Sébastien Tixeuil propose Stream Learning, which reimagines partition scheduling as a peer-to-peer live streaming problem: model partitions act as video chunks, and partition age acts as chunk scarcity. This analogy produces a two-stage selection design space (partition-first or neighbor-first), yielding ten concrete protocols.

Their key finding is that the simplest protocol, Ri, which transmits the locally least-trained partition to a uniformly random neighbor, matches PTGL on fault-free workloads while requiring no token counters or metadata exchange. Under an adversarial 30% permanent crash of the best-performing nodes, Ri matches or outperforms PTGL across all tested complete-graph configurations, with the gap reaching 5.53% on HAR and 5.41% on MNIST in the most heterogeneous regime (Dirichlet β=0.1). The experiments show that partition fairness—captured by a single local rule on partition age—accounts for the performance gap; token-based rate control and utility maximization do not improve over this rule and, under heterogeneity, sit below it. This suggests that complex coordination mechanisms may be unnecessary for robust distributed learning.