New CODS method slashes AI training costs by 90%

A team of researchers led by Ibne Farabi Shihab has introduced CODS (Critic-Guided Data Selection), a novel method for optimizing offline reinforcement learning (RL) by intelligently curating training data. Traditional offline RL trains policies from fixed datasets, where redundant or irrelevant transitions waste compute resources. CODS addresses this by iteratively fitting a critic model to identify high-residual transitions—data points with the greatest learning potential—before freezing a reusable subset for subsequent training rounds. This approach contrasts with prioritized replay methods, which don’t produce static, reusable artifacts.

The method’s effectiveness is striking: at a mere 10% data budget, CODS retains 96.6% of the performance achievable with full datasets across 20 diverse D4RL task-algorithm combinations. It outperforms state-of-the-art baselines like ReDOR and OPER in 19 out of 20 cases, with significant advantages persisting even under rigorous statistical corrections. The team also demonstrated that five acquisition rounds improve performance by 11.23 points over single-round selection, though further rounds yield diminishing returns. CODS isn’t just about speed; it’s about reusable efficiency, as equal-pass and equal-hour evaluations confirm that reuse—not just faster runs—drives the compute advantage.