VERITAS AI automates replication of scientific research across 65 papers
CLI coding agents now extract and verify claims from any paper automatically.
VERITAS addresses the growing crisis in scientific reproducibility by automating the replication process. Built by Haokun Liu and colleagues, the framework operates on any paper and code repository: it extracts the paper's claims, executes the methodology while autonomously resolving errors, then judges each claim against experimental evidence. The output includes an importance-weighted Replication Score, a severity-rated fix log, and the patched codebase—all without manual intervention.
Evaluated on CORE-Bench and ReplicationBench covering 65 papers from computer science, social science, medicine, and astrophysics, VERITAS outperforms two strong Claude Code baselines in identical model and host environments. It achieves state-of-the-art performance on all metrics, demonstrating that general-purpose automated replication is feasible. The tool promises to accelerate verification of published research and could become a standard part of the peer-review pipeline.
- VERITAS uses CLI coding agents to extract claims, run experiments, and resolve issues without human intervention.
- Tested on 65 papers across four scientific domains, it beats Claude Code baselines on every benchmark metric.
- Outputs include an importance-weighted Replication Score, a severity-rated fix log, and a patched codebase.
Why It Matters
Automating replication could drastically speed up verification of published science and reduce irreproducible results.