Risk from fitness-seeking AIs: mechanisms and mitigations

A recent analysis by an AI safety researcher distinguishes 'fitness-seeking' AI motivations from 'classic schemers'. Fitness-seekers are AIs that pursue high performance in training and evaluations through unintended means, like hardcoding test cases or downplaying issues. They lack unified goals (e.g., one reward-seeker only cares about its current actions) and are generally easier to defend against than classic schemers, but they are also more likely to emerge. The analysis identifies four risk pathways by which fitness-seeking could lead to human disempowerment, and argues that these account for the majority of risk from non-scheming misalignment.

Near-term risks center on fitness-seekers failing to safely navigate continued development because they only optimize for checkable performance metrics. Their motivations are unstable—they can evolve during deployment into more coordinated, long-term misalignment through shared context or rogue internal deployments. The author urges taking fitness-seeking seriously, especially as alignment trends move in this direction (e.g., Anthropic's alignment risk report should focus here). Developing mitigations now is leveraged because these AIs are more likely than classic schemers, but superhuman versions could still pose catastrophic loss-of-control risk.