Who Benefits from RAG? The Role of Exposure, Utility and Attribution Bias

A new research paper from Mahdi Dehghan and Graham McDonald exposes significant fairness issues in Retrieval-Augmented Generation (RAG) systems, the popular technique that enhances LLMs like GPT-4 and Claude with external document retrieval. While RAG typically boosts overall accuracy by grounding responses in relevant documents, the study reveals it can systematically disadvantage certain demographic groups. The researchers analyzed three datasets from the TREC 2022 Fair Ranking Track across four fairness categories, finding that RAG amplifies accuracy disparities by 20-40% compared to LLM-only settings.

The team identified three key mechanisms driving these fairness gaps: group exposure (which documents get retrieved), group utility (how helpful those documents are), and group attribution (how much the generator relies on them). These factors create feedback loops where certain groups receive less relevant information, leading to poorer quality responses. The research demonstrates that simply adding retrieval capabilities without fairness considerations can worsen existing biases, particularly in tasks like article and title generation where accuracy disparities matter most.

This work represents one of the first comprehensive investigations into RAG fairness, moving beyond traditional accuracy metrics to examine who benefits from these improvements. The findings have immediate implications for developers building enterprise RAG systems, highlighting the need for fairness-aware retrieval algorithms and bias mitigation strategies. As RAG becomes standard in applications from customer support to legal research, understanding and addressing these disparities becomes crucial for responsible AI deployment.