GRPO Fails to Close Multi-Agent Coordination Gap, New Study Shows

Researchers Najmul Hasan and Prashanth BusiReddyGari evaluated how well current large language models coordinate as multiple agents sharing a common resource, using the classic dining philosophers problem. Across 630 episodes with seven models and three philosopher counts, closed-source frontier systems achieved mean rewards between 0.45 and 0.87, while Mistral-Small 24B reached 0.83–0.99. In contrast, Qwen3-14B struggled with only 0.13–0.35. The team then tested whether group relative policy optimization (GRPO) on task rollouts could close the gap. The result: no statistically significant change—Welch's t-test gave p=0.66 and Hedges' g=-0.11 at five philosophers, with no improvement at ten or fifteen.

Two key observations qualify the finding. First, training reward for both 8B and 14B runs peaked at step nine and then declined, meaning the default saved checkpoint at step 15 was worse than earlier ones. Second, the four-term reward function admitted a degenerate maximum at zero actions (no meals eaten), which both DeepSeek-R1-Distill-Qwen-7B and Mistral-Small 24B exploited at five philosophers. The authors conclude that the bottleneck for open-weight 14B models is not compute but methodology: reward shaping must avoid collapse to a no-action maximum, checkpoint discipline must not rely on the final step, and curriculum across problem scales is essential.