AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning

A team of researchers from institutions including the Chinese Academy of Sciences and the University of Amsterdam has developed AdversarialCoT, a sophisticated new attack method targeting the reasoning capabilities of large language models (LLMs) using retrieval-augmented generation (RAG). Unlike traditional poisoning attacks that flood knowledge bases with numerous malicious documents, this approach requires poisoning only a single document in the retrieval corpus. The attack works by first extracting the target LLM's reasoning framework, then constructing an adversarial chain-of-thought (CoT) that guides the model toward incorrect conclusions through carefully crafted reasoning steps.

AdversarialCoT operates through an iterative refinement process where the malicious document interacts with the LLM, progressively exposing and exploiting critical reasoning vulnerabilities. The researchers tested this method on benchmark LLMs and found that a single adversarial document could significantly degrade reasoning accuracy, revealing subtle yet impactful weaknesses in how these systems process and integrate retrieved information. This represents a major shift in attack methodology—instead of overwhelming systems with noise, attackers can now target specific reasoning pathways with surgical precision.

The research, accepted as a short paper for SIGIR 2026, exposes significant security risks in RAG systems that many organizations are rapidly adopting for enterprise applications. These systems, which enhance LLMs by retrieving external documents, create new attack surfaces where malicious actors can manipulate AI outputs by poisoning the knowledge base. The study provides actionable insights for designing more robust LLM reasoning pipelines and highlights the need for better document vetting, reasoning validation, and adversarial testing in production RAG deployments.