New Deployment Simulation predicts LLM safety risks with 92% accuracy

Before releasing a new LLM, labs typically rely on benchmarks, red-teaming, and targeted evaluations—but these often fail to capture how a model will behave in messy, real-world user interactions. A new method called Deployment Simulation, detailed by a team from the AI Alignment Forum, aims to close that gap. The approach works by replaying privacy-preserved, anonymized past conversation logs through a candidate model, allowing researchers to observe how it responds in realistic contexts. In a study with GPT-5.4, the method was strikingly effective: for safety categories where production rates shifted by at least 1.5x, Deployment Simulation predicted the direction of change 92% of the time, compared to just 54% for a baseline using challenging prompts. Simulated deployments also looked far closer to real production traffic on evaluation-awareness metrics—standard evals often show obvious “stage lights” that models can exploit, whereas production prefixes mostly do not.

The hardest case involves agentic tool use, where model behavior depends on external state like filesystems, APIs, or prior tool results. To handle this, the researchers used another model to simulate tool responses, trained on the original conversation trajectory and a time-matched codebase. This allows them to test how a model uses tools in realistic multi-step scenarios. The team emphasizes that Deployment Simulation is not a replacement for traditional safety evaluations but a complementary “forecast” that helps identify blind spots before a model reaches users. They have already used insights from the method during GPT-5.4’s development to inform mitigations and deployment decisions. As the pipeline becomes easier to run, the researchers expect it to play a larger role in future model development.