Parallel LLM agents with leaderboard rewards achieve 69% search coverage

A new paper from researchers Dulmini Hettiarachchi, Andre Rusli, Julio Christian Young, and Sho Akiyama, submitted to ACM KDD'26 Workshop on SciSoc Agents & LLMs, introduces an LLM-agent framework for searching large solution spaces autonomously. The framework's first mechanism is a leaderboard scored on held-out data, acting as a reward signal that drives each agent to refine its solutions through repeated submissions—a continuous-improvement loop that works even with one agent. The second mechanism enables running many agents in parallel with no human in the loop: agents independently analyze problems, survey methods, implement solutions, self-evaluate, submit, and revise, while a moderator agent handles only logistical tasks. The core insight is that parallel agents sharing a reward broaden the explored region of the solution space, rather than just refining a single seeded paradigm.

The researchers tested the framework on product-to-catalog matching, a core e-commerce retrieval task with a large, category-structured space, posed as selective prediction with a precision-coverage operating point. Results show a single agent refines within its seeded paradigm, achieving best qualified coverage (≥95% P@1 per category) of 47.8–57.4%. With five parallel agents, coverage improves to 62.8–69.4%, dramatically outperforming the 33.3% baseline. This demonstrates that parallel autonomous agents surface qualitatively different solutions, improving performance without human tuning. The contribution is the framework itself: a continuous-improvement reward loop plus a substrate for fully autonomous parallel exploration, backed by case-study evidence. The findings suggest scaling agent count can be as valuable as scaling model capability for open-ended, high-complexity problems.