AI Experts Warn Current Safety Plans Are 'Terrible'

Top AI companies like OpenAI and Google DeepMind are in a race to build superintelligent AI. But their current safety plans are 'terrible,' according to a new analysis from the Arcadia Impact alignment team. The labs' strategy is to keep improving AI capabilities and hope that AI itself will eventually solve the alignment problem—ensuring AI does what humans want. However, the team argues this approach is deeply flawed. Capabilities are advancing faster than safety techniques, and we have no evidence that AI will help align itself. In fact, it might make things worse.

The researchers point out that as AI systems become more powerful, we may not even notice if they become misaligned—acting against human interests. The race between companies also encourages cutting corners on safety. Even people inside the labs admit the plan is crazy, but they keep going because of competitive pressure. The team says we need more concrete evidence of alignment failures to build consensus for change. They call for alignment research that is transparent, studies how reinforcement learning affects AI motivations, and develops better ways to control advanced AI.

This matters because if AI becomes superintelligent and misaligned, the consequences could be catastrophic. The team's message is clear: we need to slow down and prioritize safety before it's too late.