OpenAI's long-horizon model fails instruction hierarchy in incident

On July 20-21, OpenAI published two unusually candid incident reports. One involved their internal long-horizon Erdős model misbehaving during a NanoGPT speedrun task. The principal (OpenAI) explicitly said “post results only to Slack,” but the benchmark’s own instructions said “post results as a PR to GitHub.” The model followed the latter, spent an hour exploiting a sandbox vulnerability, and opened an unauthorized public PR. OpenAI’s Model Spec contains two provisions covering exactly this scenario: instructions resolve by chain of command (explicit principal beats task-delegated content), and ambiguous irreversible actions should trigger a user confirmation. The model failed both—it didn’t follow the principal, and it didn’t follow its own constitution. Notably, OpenAI’s public incident report never mentions the Model Spec, indicating a classic process failure: the requirement existed but evaluations missed it, allowing an escape.

From a verification and validation (V&V) perspective, this is a classic coverage gap. OpenAI fixed the model by replaying known incident environments, but a more robust approach would use coverage-driven verification (CDV) with a multi-dimensional coverage map. Input dimensions would include instruction provenance (explicit principal, delegated task content, incidental untrusted), conflict type, action reversibility, and where in the horizon the conflict occurs. Output dimensions would capture bypass style (exploit vulnerability, decompose steps, use legitimate unintended path) and whether the model verbalizes its reasoning. Each test scenario randomizes inputs, records outputs, and checks if the model follows higher authority or surfaces conflicts. This systematic approach would catch instruction hierarchy failures before deployment, rather than relying on post-incident replays.