Led to Mislead: Adversarial Content Injection for Attacks on Neural Ranking Models

A team of researchers from multiple institutions has unveiled CRAFT, a novel adversarial attack framework that leverages large language models (LLMs) to manipulate neural ranking models (NRMs) — the backbone of modern search engines. Published on arXiv (2605.01591), the method operates in three stages: adversarial dataset generation via retrieval-augmented generation and self-refinement, supervised fine-tuning on curated examples, and preference-guided optimization to align outputs with rank-promotion goals. On the MS MARCO passage dataset and TREC Deep Learning 2019/2020 benchmarks, CRAFT significantly outperforms existing baselines, achieving higher promotion rates and rank boosts while maintaining fluency and semantic fidelity.

CRAFT's strength lies in its black-box nature — it does not require access to the target model's internals — and its transferability across diverse ranking architectures, including cross-encoders, embedding-based models, and LLM-based rankers. This suggests that even state-of-the-art retrieval systems remain vulnerable to adversarial content injection. The authors publicly release their source code, trained models, and prompt templates to facilitate reproducibility and further research. The work underscores the dual-use risk of generative AI: the same LLMs that power helpful search can be weaponized to systematically game ranking algorithms, raising urgent concerns for search integrity and content moderation.