New framework lets humans steer multi-agent AI systems with fine-grained control

A new paper from researchers Zeyu He, Hannah Kim, Dan Zhang, and Estevam Hruschka, accepted at ACM CAIS 2026, tackles a core challenge in multi-agent AI systems: how humans can effectively steer complex agent plans. The key insight is that current approaches rely on outcome-level supervision—users only verify final outputs, missing intermediate reasoning. The team formalized a design space for human-LLM co-planning interactions along three axes: mode (semantic tweaks vs. structural changes), scope (global versus targeted edits), and level (low-level details vs. high-level goals).

To explore this space, they built AMBIPOM, a prototype that supports process-level supervision through both semantic and structural interactions. In a user study, they characterized how people navigate these dimensions, uncovering hybrid workflows—e.g., mixing global structural changes with targeted semantic edits—and revealing explicit trade-offs between effort, control, and risk. A controlled benchmark then analyzed how LLMs revise plans under varying scope and revision strategies. The findings produce actionable design insights for building multi-agent systems that are more transparent, controllable, and ultimately more effective for real-world human-AI collaboration. The code and data are open-sourced.