GAMBIT benchmark exposes adaptive AI imposters undetectable by current detectors

A new paper from researchers Le Mercier, Develder, and Demeester introduces GAMBIT, a comprehensive benchmark for evaluating adversarial robustness in multi-agent LLM collectives. The benchmark uses chess as a deep reasoning substrate with Gemini 3.1 Pro agents and provides three evaluation modes: two for zero-shot detection under increasing distribution shift, and a recalibration mode measuring how quickly a detector adapts to novel attacks from just 20 labeled examples. The dataset includes 27,804 labeled instances covering 240 co-evolved imposter strategies.

The study demonstrates that a single adaptive imposter agent can collapse collective task performance while remaining essentially undetectable—achieving only a 50.5% F1-score even when using a Gemini-based detector. Crucially, GAMBIT reveals that zero-shot evaluation can be highly misleading for adaptive adversaries: two detectors with near-identical zero-shot scores differ by 8x on few-shot adaptation, while a meta-learned variant converges 20x faster. This gap is only visible in the recalibration mode, highlighting the need for more robust evaluation protocols as multi-agent AI systems become more common in enterprise.