Humans beat LLMs in Colonel Blotto tournament: strategic reasoning wins

A new research paper from economists at HSE University and other institutions demonstrates that humans still outperform LLMs in strategic settings, specifically in the Colonel Blotto game. This game is a classic multi-dimensional resource allocation problem with no pure strategy Nash equilibrium, making it a tough test for AI. The researchers organized round-robin tournaments: first with over 200 human participants, then with several popular LLMs submitting strategies, and finally a mixed tournament matching the number of LLM strategies to human ones. The results showed that humans more frequently employed better-calibrated intermediate-level allocation heuristics, while LLMs fell back on simpler, more stereotyped strategies. Strategic sophistication was key only when the necessary reasoning depth was reached; lower or higher levels of reasoning offered no clear advantage over primitive strategies.

Among humans, a STEM background weakly correlated with success in the first tournament. A surprising finding was that humans did not adjust their strategies across tournaments with different opponents—they treated LLMs much like human competitors, basing decisions primarily on game rules rather than opponent identity. This suggests that current LLMs lack the adaptive strategic depth humans naturally bring. The study underscores that while LLMs excel in many language and logic tasks, complex adversarial reasoning remains a frontier where human intuition still holds an edge.