Anthropic Researcher Proposes New Rules to Keep AI in Check

A researcher at Anthropic, an AI company, has shared personal thoughts on how to keep advanced AI systems safe by making sure we can monitor what they're doing. The core idea is that as AI gets smarter, we need ways to check its actions and intentions. The researcher outlines four types of monitorability and discusses the challenge of trusting the monitors themselves. If monitors are just as smart as the AI they're watching, they might miss subtle problems or even collude. To address this, the researcher suggests using trusted monitors (those we're confident are aligned) and combining them with untrusted ones to catch any deception.

The proposal also includes policy ideas: setting red lines for absolute levels of unmonitorability that labs should not cross, and constraining actions that degrade monitorability even before those red lines are reached. This could involve evaluating the ratio of unmonitorability to capabilities and rejecting changes that make that ratio too high. The goal is to ensure that as AI becomes more powerful, it doesn't become impossible to oversee.

For everyday people, this matters because AI is increasingly used in decisions that affect our lives—from healthcare to finance. If AI can't be monitored, it could lead to unintended consequences or misuse. The researcher's proposals are a step toward ensuring that AI development includes safety checks. However, these are personal views, not official policy, and the technical challenges are significant. The debate is ongoing, and it may be some time before any regulations are implemented.