Heterogeneous multi-agent AI boosts reasoning accuracy by 19%

A new paper from researchers J. de Curtò and I. de Zarzà introduces a multi-agent framework for collective intelligence with foundation models. The system comprises three agent types: solver models that independently generate drafts, a critic agent that provides structured critique and revision, and an aggregator that synthesizes a final consensus. A scoring module evaluates semantic, numerical, and procedural quality across all agents. The authors benchmarked this framework on a diverse STEM dataset including calculus, physics, chemistry, biology, economics, optimization, statistics, and mathematics.

They compared four configurations: individual baseline, homogeneous framework using one model, redundant homogeneous solvers (multiple instances of same model), and heterogeneous framework with diverse specialized models. Results show that while framework structure and redundant sampling offer modest gains, model heterogeneity is the critical factor. The heterogeneous configuration achieved superior step-wise accuracy (0.64 vs 0.54 for individuals) and a 2.3x improvement over homogeneous setups, with reduced variance across categories and difficulty levels. Step-wise reasoning quality—correctness of intermediate steps—improved dramatically only with model diversity, highlighting the importance of complementary error detection and reasoning refinement for explainability and auditability.