Power and Limitations of Aggregation in Compound AI Systems

Stanford researchers Nivasini Ananthakrishnan and Meena Jagadeesan have published a foundational paper, 'Power and Limitations of Aggregation in Compound AI Systems,' providing a formal analysis of a common practice in AI engineering. The work tackles a core question: when does querying multiple instances of the same model (like running 10 calls to GPT-4o) and aggregating their outputs actually produce better, more reliable results than a single query? The authors use a principal-agent framework to model how system designers can steer AI agents through reward functions, but are still constrained by prompt engineering and inherent model capabilities.

The analysis proves that aggregation expands the set of possible outputs through three distinct mechanisms: feasibility expansion (making new outputs possible), support expansion (increasing the probability of rare outputs), and binding set contraction (reducing the set of outputs a model is incentivized to produce). The paper establishes that any effective aggregation operation must implement one of these mechanisms. The researchers also provided an empirical illustration using LLMs on a reference-generation task, validating their theoretical framework. This work moves the field from heuristic practices to a principled understanding, giving engineers a roadmap for designing compound AI systems that can overcome limitations in single-model performance and imperfect prompting.