Foreign Policy AI Evaluation Gap: A Critical Governance Blindspot

A new paper by Charles Pozniak and Jeba Sania identifies a critical gap in how we evaluate AI systems used for foreign policy—what they term 'statecraft.' These systems are already being deployed in high-stakes domains like war and peace, yet current evaluation practices are dangerously inadequate. The authors argue that foreign policy tasks have structural properties that standard evaluations handle poorly: partial observability (policy makers can't see all outcomes), unbounded action spaces (AI can take any action), contested ground truth (no clear right answer), and multidimensional objectives (economic, military, diplomatic goals simultaneously). These features combine with catastrophic tail risks—one bad AI decision could spiral into global conflict—making this the 'perfect storm' for technical governance.

The paper's ecosystem review finds that the AI governance community focuses asymmetrically on assessment features (benchmarks, red-teaming) while neglecting access controls, verification protocols, security measures, and operational constraints. To close this gap, the authors propose a demand-side evaluation framework that decomposes complex foreign policy workflows into bounded, evaluable sub-tasks. Each sub-task can be tested with human-in-the-loop recombination, ensuring AI outputs are checked before real-world deployment. As AI agents increasingly aid diplomats, intelligence analysts, and military planners, this research agenda is not merely academic—it is an urgent priority to prevent catastrophic failures in the conduct of war and peace.