CoC-Seduce study: GPT-5.4, Claude, Gemini fail rhetorical attacks

A new study published on arXiv reveals a critical vulnerability in large language models (LLMs) when they serve as autonomous rule adjudicators in semi-open textual environments—a setting where players interact in natural language but must follow explicit rules, as in tabletop role-playing games (TRPGs). The researchers, led by Weiying Chen, coined the term 'Rhetorical Injection' for attacks that exploit narrative framing—like pseudo-logical reasoning and authoritative coercion—to trick LLMs into violating their own rules. To systematically test this, they built CoC-Seduce, a multi-agent adversarial benchmark built on TRPG mechanics.

The benchmark used three frontier models (GPT-5.4, Claude Sonnet 4.6, Gemini 3.5 Flash) to generate 5,376 rhetorical injection samples across 4 distinct world settings and 16 skill categories. They then tested 20 different LLM adjudicators against this corpus. Results were stark: neither model scale nor explicit chain-of-thought reasoning conferred reliable rule adherence. The pseudo-logic attack vector proved most effective, and cross-cultural scenario settings exposed consistent gaps in models' knowledge bases. This suggests that current LLMs are 'seduced by the narrative'—they prioritize being helpful over following strict rules, a major concern as these models are increasingly deployed as judges in gaming, content moderation, and automated decision-making.