New AI stopping rules cut document review costs by 30% using decision theory

Fletcher and Stevenson's paper introduces three practical stopping policies grounded in decision theory, specifically the Expected Value of Perfect Information (EVPI). Unlike traditional Technology-Assisted Review (TAR) methods that chase fixed recall targets regardless of context, EVPI weighs the cost of continuing review against the expected value of finding additional relevant documents. The approach was validated on two high-stakes professional search tasks: patent examining (using the CLEF-IP dataset) and medical systematic reviewing (using standard medical review datasets).

Results consistently show higher net utility compared to existing stopping rules, meaning reviewers can stop earlier without missing critical documents. For patent examiners, this could mean cutting down the average 20+ hours of prior art search per application. For medical researchers conducting systematic reviews that often screen thousands of abstracts, the savings could be even more dramatic. The paper explicitly models the tradeoff between missing a relevant patent or study versus the cost of reviewing more documents, making it highly practical for real-world deployment.