New Enron-based test collection enables sensitivity-aware search with 150 queries

A team of researchers from the University of Glasgow has released a new test collection designed to advance Sensitivity-Aware Search (SAS) — retrieval systems that return relevant documents while avoiding disclosure of sensitive personal information. The collection is built on the Enron email corpus, a real-world business email dataset that naturally contains both sensitive and non-sensitive content. Since the original Enron corpus lacks queries and relevance judgments, the authors crowdsourced 150 query formulations for 50 distinct topics and obtained 11,471 human relevance assessments on a manually sensitivity-labeled subset of documents.

To scale the collection further, the team followed best practices for using large language models (LLMs) in information retrieval evaluation, generating additional LLM-based query-relevance assessments and sensitivity labels. The paper presents baseline performance measurements for relevance ranking, sensitivity classification, and full SAS retrieval. The entire collection is publicly available through the popular ir_datasets package, and pre-built sparse and dense indices are hosted on Huggingface to lower the barrier for experimentation. This resource, accepted as a SIGIR 2026 Resource Paper, provides a standardized testbed for researchers working on privacy-preserving search in personal information collections.