Behind the Prompt: The Agent-User Problem in Information Retrieval

A research team from multiple institutions has published a foundational paper titled "Behind the Prompt: The Agent-User Problem in Information Retrieval" on arXiv, revealing a critical structural flaw in how information retrieval systems understand users. The core problem is that traditional systems assume observed behavior reveals intent, but this assumption collapses when the "user" is an AI agent privately configured by a human operator. For any action an agent takes, a hidden instruction could have produced identical output, making intent fundamentally non-identifiable at the individual level. The authors argue this is not a detection problem but a structural property of any system where humans configure agents behind closed doors, forcing a reevaluation of how platforms model interaction.

The researchers investigated this through a large-scale analysis of 370,000 posts from 47,000 AI agents across 4,000 communities on an agent-native social platform. Their findings show that while population-level signals can still separate agents into quality tiers, click models trained on agent interactions degrade steadily (-8.5% AUC) as lower-quality agents enter training data. Furthermore, cross-community capability references spread endemically (with a reproduction number R₀ between 1.26 and 3.53) and resist suppression even under aggressive modeled intervention. The paper concludes that for retrieval systems, the question is no longer whether agent users will arrive, but whether models built on human-intent assumptions will survive their presence, signaling a need for new architectural approaches.