MOASEI 2026 competition sees low participation but advances multi-agent evaluation
Only one team submitted a final entry across four tracks in 2026's open-system benchmark.
The 2026 MOASEI (Methods for Open Agent Systems Evaluation Initiative) Competition, held at AAMAS 2026, built on its inaugural 2025 edition by retaining three core domains: wildfire fighting, cybersecurity, and ride-sharing. A new bonus wildfire track introduced frame openness, where agent equipment states (e.g., suppressant capacities and firefighting range) vary unpredictably over time, testing adaptability. The competition also expanded reporting metrics to focus on total task completions, mean task-completion time, and mean value of completed tasks—emphasizing practical mission success over raw efficiency.
Participation was notably low: eight teams registered, but only one final entry was submitted, targeting the ride-sharing track. The winning approach, DLC (planning and replanning), dynamically solved routing problems as passengers appeared, outperforming baseline policies. The organizers published this technical report detailing the competition design, year-over-year changes, and evaluation results. While low engagement limits statistical significance, the work highlights the challenge of creating open-system benchmarks robust enough to attract broader participation.
- Only 1 of 8 registered teams submitted a final entry: DLC's planning and replanning approach for ride-sharing.
- New metrics introduced: total task completions, mean task-completion time, and mean value of completed tasks.
- Added a bonus wildfire track with frame openness (varying agent equipment states like suppressant capacity and range).
Why It Matters
Benchmarking open multi-agent systems is crucial for real-world deployment, but low participation signals a need for more accessible evaluation frameworks.