New benchmark tests if LLMs vote strategically like humans

A new paper from researchers Seyed Pouyan Mousavi Davoudi, Alireza Amiri-Margavi, and colleagues introduces a novel oracle-based benchmark to determine whether large language models (LLMs) can strategically manipulate voting outcomes. The benchmark, detailed in arXiv preprint 2606.21001, presents each LLM with its true preference ranking, other voters' ballots, a deterministic voting rule, and a prompt condition. An exact oracle then enumerates every feasible ballot the LLM could submit, computes the sincere outcome, identifies all profitable manipulations, and records the best achievable result. This provides ground truth without human labeling, covering five voting rules: plurality, Borda count, approval voting, instant-runoff voting (IRV), and Copeland-style pairwise majority voting.

The core experiment uses 600 balanced election instances with a single electorate size, producing 9,600 model-prompt responses across four model configurations and four prompt conditions (sincere, strategic, civic, and expert framings). By reducing strategic-voting behavior to exact counterfactual evaluation, the benchmark turns the question "Do LLM voters vote sincerely or strategically?" into a reproducible social-choice experiment. The authors report exact oracle-calibration baselines to bound and contextualize subsequent model results, as no existing peer-reviewed work reports manipulation discovery, optimal manipulation, false manipulation, near-miss, or invalid-ballot rates for this specific task. This framework could have significant implications for AI governance, automated decision-making, and the design of resilient voting systems.