SimpleWikiSearch offers a reproducible offline Wikipedia environment for agentic search

Large language model (LLM)-based agentic search systems are often evaluated as if the underlying LLM were the only component that matters, yet measured performance also depends on the surrounding search environment: the Wikipedia snapshot, preprocessing pipeline, chunking policy, retrieval backend, tool schema, observation format, and answer submission rule. These details are frequently under-specified, making it difficult to compare results or reproduce reported baselines. To address this, researchers Guanming Xiong and Penghui Zhang introduce SimpleWikiSearch, whose corpus construction, retrieval stack, tool contract, and evaluation protocol are explicit and runnable. The environment starts from a full English Wikipedia dump, cleans and chunks the corpus, builds keyword and dense retrieval indexes, and exposes a minimal tool interface consisting of search, open_url, and submit_answer.

SimpleWikiSearch reports baseline results on six QA datasets using open-source LLMs and provides a random-300 subset for comparisons with closed-source commercial models. The environment offers a domain-specific agent harness and a controlled offline environment for reproducible agentic-search evaluation. Its contribution is this specified reference setup, rather than a new agent algorithm. Code and data will be made available. By standardizing the search environment, SimpleWikiSearch enables fairer comparisons and more reliable progress in agentic search research.