Social Learning study reveals paradoxes in rejection rate metric
Standard performance measure found unsuitable; new analysis uncovers irreducible gaps in decentralized decision-making
Social learning is a decentralized paradigm where multiple agents collect streaming observations and exchange beliefs to converge on the true hypothesis. A key metric has been the rejection rate – the speed at which erroneous beliefs vanish. However, in a new paper submitted to IEEE, Felice Scala and colleagues demonstrate that the rejection rate leads to several paradoxes, making it invalid as a performance measure. They argue for focusing on error probability instead.
For a binary Gaussian hypothesis testing problem, the team derives an analytical formula for the ratio of each agent's error probability to the optimal centralized Bayesian probability. This ratio factors into two components: one quantifying network connectivity (how well agents are linked) and another capturing prior information. Crucially, they show an irreducible gap between decentralized and centralized error probabilities that is agent-specific and does not vanish even with infinite observations. This finding has significant implications for the design and evaluation of distributed decision systems in applications like sensor networks, multi-robot systems, and federated learning.
- Rejection rate metric for social learning is shown to produce paradoxes, making it unsuitable for performance evaluation.
- For a binary Gaussian problem, researchers derive an analytical formula where the error probability ratio equals product of network connectivity and prior information terms.
- An irreducible gap exists between decentralized and centralized error probabilities that is agent-dependent and persists asymptotically.
Why It Matters
Forces re-evaluation of performance metrics in decentralized AI, revealing fundamental limits of network-based decision-making.