New greedy algorithm selects complementary LLMs for better ensembles

Multi-AI collaboration, where multiple large language models (LLMs) propose answers and a summarizer synthesizes them, is a powerful paradigm for boosting performance. However, selecting which proposers to include is non-trivial—naive approaches that focus solely on accuracy or diversity often miss critical interactions among models. In a new paper, researchers reframe proposer selection as a combinatorial feature selection problem, where the value of an LLM lies in its complementarity with others. They propose Mixture of Complementary Agents (MCA), a framework that uses computationally feasible greedy algorithms (e.g., forward selection, backward elimination) to assess complementarity using a small labeled dataset. Their experiments show that these methods consistently outperform both accuracy-only and diversity-only baselines, achieving the best performance-cost trade-offs in practice.

MCA’s key insight is that an ensemble is only as strong as its weakest link—and that link is often a redundant model. By explicitly modeling how proposers complement each other and the summarizer, the greedy algorithms can identify small but powerful subsets of LLMs. This has significant practical implications: organizations can build robust AI systems without needing to run every available model, saving compute and time. The paper also opens the door to dynamic ensemble selection, where the mix of models adapts to the task. For professionals building multi-agent pipelines, MCA offers a principled, efficient way to maximize accuracy while minimizing resource waste.