SE-Search: Self-Evolving Search Agent via Memory and Dense Reward

A research team led by Jian Li has introduced SE-Search, a novel self-evolving search agent that significantly advances retrieval-augmented generation (RAG) systems. Unlike traditional RAG approaches that often accumulate irrelevant documents and rely on sparse reinforcement learning signals, SE-Search implements a sophisticated Think-Search-Memorize strategy with three key innovations: memory purification to filter noise, atomic query training to generate more diverse search terms, and dense rewards for fine-grained feedback. The system addresses critical limitations in current AI search agents, particularly their tendency to gather low-quality evidence during multi-turn information-seeking processes.

The technical implementation demonstrates impressive results, with the SE-Search-3B model achieving a 10.8-point absolute improvement and 33.8% relative gain over the Search-R1 baseline on question-answering benchmarks. This performance boost comes from the agent's ability to continuously refine its search behavior through memory management and better reward signals. The researchers plan to release code and model weights upon acceptance, potentially enabling more accurate AI research assistants, enterprise search tools, and factual QA systems that can handle complex, multi-step queries with reduced hallucinations.