Research agents get smarter with RubricRanker from Tsinghua team
New RubricRanker model lifts deep research benchmarks by 2.6 points using LLM-designed search rubrics
A team led by Wenhan Liu from Tsinghua University and involving ten co-authors from multiple institutions has proposed RubricRanker, a novel approach to improve document selection for deep research agents. Traditional retrievers often return individually relevant documents that fail to collectively satisfy complex information needs. The researchers introduced search-oriented rubrics—hierarchically structured criteria that explicitly define what constitutes a high-quality document set for a given query. These rubrics are synthesized using a powerful LLM and then used to train a document reranker called RubricRanker.
The team implemented a two-stage training framework: rubrics-guided supervised fine-tuning followed by rubric-based reinforcement learning. In extensive experiments, RubricRanker achieved a 2.6-point improvement over the strongest baseline on four deep research benchmarks and demonstrated strong generalization across five RAG benchmarks. This method shifts the focus from individual relevance to collective quality, offering a more robust foundation for agents tackling complex research tasks.
- Researchers from Tsinghua and 10 collaborators built RubricRanker, a document reranker for deep research agents
- Uses LLM-synthesized search rubrics to define high-quality document set criteria and outperforms baselines by 2.6 points on deep research benchmarks
- Two-stage training: rubrics-guided SFT + reinforcement learning; generalizes to five RAG benchmarks
Why It Matters
Sets a new standard for document selection in research agents, improving answer quality for complex queries