Second Look's $5K reruns keep AI safety research fresh on GPT-5.5
Replicating AI safety papers on frontier models costs as little as $0 per experiment
Second Look Research's summer fellowship is proving that rerunning AI safety research on every frontier release is cheap and high-value. The team has already replicated influential safety papers on newer, more capable models—confirming, for instance, that Google's chain-of-thought monitorability experiments still hold for GPT-5.5. They also track Ryan Greenblatt's filler token results, which test whether models use innocuous tokens to hide extra reasoning. Each rerun costs between $0 and $5,000, with no cost when codebases are already complete and models are accessible.
The process is streamlined: once a paper has a replication codebase, adding a new model is 'a single terminal command away, or at the most a single Claude prompt away.' SLR envisions a trusted undergrad running selected experiments overnight, with results reviewed by senior researchers before public release. Challenges remain—incomplete code, parameter mismatches, or closed-source models—but SLR's existing replications bypass these hurdles. The real value is maintaining a living tracker of safety trends, catching step changes or silent regressions that system cards miss, and keeping critical findings from going stale.
- Second Look Research replicated Google's CoT monitorability experiments on GPT-5.5, confirming they still hold on more capable models.
- Rerunning a safety paper on a new model costs $0–$5,000 and can be done overnight by a supervised undergrad.
- With complete replication codebases, adding a new model takes one terminal command or a single Claude prompt.
Why It Matters
This keeps critical AI safety findings from going stale, allowing fast detection of behavioral shifts in new frontier models.