AI as a Trojan horse race

In a widely discussed post on the AI forum LessWrong, researcher Katja Grace presents a critical framework for understanding the competitive dynamics of AI development. Her essay, 'AI as a Trojan horse race,' challenges the pervasive narrative that the field is locked in an inevitable 'arms race.' Grace argues that while companies like OpenAI, Anthropic, and Google DeepMind are demonstrably racing to build more powerful AI systems, this behavior does not mean the underlying incentive structure is one where speed is the rational, winning strategy. She uses the analogy of chess champions playing in a tournament where the prizes are actually awarded for winning at checkers—they are racing, but not in the game they think they are.

Grace's central point is that this distinction has profound real-world consequences. The belief in an unavoidable arms race is frequently used to justify two things: moving recklessly fast despite potential dangers to society (point 'a'), and giving up on attempts to coordinate with other labs on safety measures because competitors are presumed to be immovably committed to winning (point 'b'). If the strategic landscape isn't truly a race, these justifications collapse. She proposes the 'Trojan horse' metaphor to separate behavior from incentives: teams are frantically pulling giant, mysterious wooden horses (AI systems) into their city gates, desperate to beat rivals, without true certainty whether the horses contain treasure or enemy soldiers.