OThink-SRR1: Search, Refine and Reasoning with Reinforced Learning for Large Language Models

A research team led by Haijian Liang has introduced OThink-SRR1, a novel framework designed to solve persistent problems in Retrieval-Augmented Generation (RAG) systems. Current RAG methods, which help LLMs access external knowledge, often struggle with complex, multi-hop questions. They either retrieve irrelevant information that misdirects reasoning or process entire documents at prohibitive computational cost. OThink-SRR1 addresses this with a structured, iterative Search-Refine-Reason process, where a key 'Refine' stage distills retrieved documents into only the most relevant facts before the model begins its reasoning.

The framework is trained end-to-end using a new reinforcement learning algorithm called GRPO-IR. This algorithm uniquely rewards the model for accurately identifying correct evidence while penalizing excessive or unnecessary retrieval steps. This dual objective trains the AI to be both more focused and more efficient. In experiments across four established multi-hop question-answering benchmarks, OThink-SRR1 demonstrated superior accuracy compared to strong baseline models. Crucially, it achieved these results while using fewer retrieval steps and processing fewer tokens, indicating significant gains in both performance and cost-effectiveness. The authors position OThink-SRR1 as a potent foundational architecture for building the next generation of efficient, reliable information-seeking AI agents.