AI Labs Should Fix Current Flaws Before Building Bigger Models

Imagine if car makers had to fix every known safety defect in their current cars before selling a faster, more dangerous model. That's the idea behind a new proposal for AI development. The proposal says AI labs should not train the next generation of models until they have adequately resolved all significant alignment issues in the current generation. Alignment means making sure AI does what we intend and doesn't cause harm. The author argues that with any other dangerous technology, we wouldn't accept 'shrug and move on' when things go wrong.

The proposal could appeal to both optimists and pessimists in the AI safety debate. Optimists think current techniques can fix alignment issues, while pessimists think it's harder. But both might agree that we shouldn't move forward until we've solved existing problems. It also creates better incentives: to advance in capabilities, labs would have to push the frontier on alignment and interpretability techniques. Interpretability means understanding how AI makes decisions.

However, the proposal is vague and incomplete. It doesn't specify how to judge if alignment issues are adequately resolved. The author suggests a qualitative process but admits it's not fully adequate. Also, 'scores better than previous models on our metrics' is not enough. If a user finds a weird but alarming issue after deployment, that's a critical failure. The proposal is more of a heuristic than a concrete plan, and there are many missing pieces, like how to enforce it.

The author also notes that scaling of AI models is slowing down naturally due to hardware and economic limits. This means some pacing is already happening. But new AI companies like Meta, Google, and SpaceX might catch up to leaders like OpenAI and Anthropic in 2027, potentially introducing new misalignment issues. Automated learning could also bring new control problems. So while the proposal has merit, it may be less necessary as other factors slow things down. Still, it's a step toward safer AI development.