GPT-4o and Llama-3.3 fail at conflict zone escalation forecasting

New research reveals that large language models cannot forecast armed conflict escalation but instead merely categorize countries based on media coverage bias. The study, led by Poli Nemkova and published on arXiv, analyzed 22 conflict zones from 2020-2026, finding a staggering 224x asymmetry in English-language media attention between the most and least covered countries. When tested on a held-out set of 660 cases, both Llama-3.3-70B and GPT-4o failed to distinguish stable from escalating periods. Llama predicted escalation on every under-covered case, matching the trivial Always-YES baseline to three decimal places, while GPT-4o predicted NO on every over-covered case, completely missing all five actual escalation events.

The core failure stems from the models' reliance on a country-categorical prior rather than temporal signals. A logistic regression using only eleven observation-window features with no country information achieved an F1 score of 0.402, outperforming both LLMs in every measurable tier. Crucially, the failure cannot be fixed at inference time: adding structured ACLED conflict event evidence actually degraded GPT-4o's F1 score from 0.323 to 0.168 on under-covered zones, falling 2.4x below the logistic regression baseline. The researchers conclude that under-covered populations receive not just less accurate AI, but qualitatively different AI that cannot distinguish stable from escalating periods. They call for coverage-stratified benchmarking, conflict NLP datasets for under-covered zones, and training data documentation standards for geographic representation.