Anthropic Researcher Proposes New Rules to Keep AI in Check

A researcher at Anthropic, an AI company, has written about a big challenge: how to keep an eye on AI systems to make sure they don't do anything harmful. This is called 'monitorability.' In a personal blog post (not an official company stance), he explains that there are different ways to monitor AI, but the trickiest part is trust. If you use the same AI to both do tasks and watch for bad behavior, it might not catch its own flaws—like asking a student to grade their own exam. So, he suggests using a separate, trusted AI as a monitor, or combining trusted and untrusted monitors to catch problems.

The researcher also worries about 'white-box monitoring,' which means looking inside the AI's 'brain' to see what it's thinking. While that sounds great, it's not clear if it works well enough to rely on. He proposes a method to evaluate how monitorable an AI is, and suggests policy ideas like setting 'red lines'—absolute limits on how unmonitorable an AI can be—and also measuring the 'monitorability externality,' which means checking if a new AI version makes monitoring harder than expected for its power level. The goal is to keep AI safe as it becomes more capable.

Why should you care? As AI takes on more tasks—like managing your calendar, handling your email, or even driving—we need to be sure it's not secretly doing something harmful. If companies don't have good monitoring, AI could make mistakes or be misused without anyone noticing. This researcher's ideas are about making sure that doesn't happen. While these are just proposals, they could influence how AI companies build safeguards in the future, affecting everything from privacy to safety.