New Algorithm Optimizes Task Assignment to AI and Humans Under Capacity Constraints

The paper addresses a practical bottleneck in hybrid AI-human systems: how to route prediction tasks when every agent—whether a human expert or a large language model—has a limited number of tasks they can handle. The authors provide a theoretical characterization of the problem in terms of agent capacities, expertise differences, and task context. They then introduce a family of sequential explore-exploit policy-learning algorithms that dynamically learn which agent to assign to each task, balancing exploration of unknown expertise with exploitation of known strengths.

Experimental results span tabular, image, and text prediction tasks using both LLMs and human participants. The proposed policies consistently outperform non-contextual baselines (e.g., round-robin or random assignment), demonstrating that learning capacity-aware assignment improves overall accuracy. The framework is general enough to be applied to any setting where multiple predictive agents with finite capacity are available, from crowdsourcing to automated AI pipelines.