Research reveals LLMs encode relevance in middle layers, beating generated judgments

A new paper by Bernardelle et al. (arXiv:2607.15555) investigates how large language models represent relevance internally, rather than just evaluating their output. The researchers used instruction-tuned LLMs in the 4-9B parameter range, prompted with UMBRELA-style relevance judgment prompts, and extracted last-token activations from every transformer layer. They trained linear probes to predict relevance labels from these activations for query-document pairs from TREC DL20 (English) and MIRACL (multilingual) collections.

Key findings reveal a depth-dependent encoding: probe performance is weak in early layers and peaks in middle-to-late layers, suggesting relevance becomes linearly accessible after contextual integration. In several models, the probe-derived pseudo-labels matched or even outperformed the LLM's own generated relevance judgments when evaluating system ranking consistency against human judgments. This indicates a separation between internal relevance representation and external expression. Multilingual experiments showed partial cross-language portability, but within-language decoding remains stronger. The work provides a new representation-level perspective for diagnosing when generated judgments fail to reflect internally available evidence.