Shots Fired in the Third War of Priors
A philosophical battle over how to predict AI's path is heating up, with prediction markets misfiring.
A viral essay by Zvi Mowshowitz has framed the intense debate over artificial intelligence's future as the latest chapter in a centuries-old philosophical conflict: the 'Third War of Priors.' This modern clash pits the Bay Area Rationalists—who gained prominence by correctly predicting AI's importance—against a still-coalescing opposition. At its core, the war is about epistemology: how we should form our initial beliefs (priors) about unprecedented technological change. Rationalists advocate for using pure logical deduction from first principles, while their critics lean on empirical comparisons to historical technological shifts.
The first major skirmish in this war, according to the essay, was the Rationalists' championing of prediction markets as the ultimate tool for forecasting. They reasoned that markets with 'skin in the game' would aggregate collective wisdom and prove human minds could accurately predict complex futures like AI progress. However, this bet has largely backfired; prediction markets have been more associated with sports gambling and insider trading scandals than delivering actionable insights on AI, representing an ironic and public stumble for the Rationalist methodology.
Now, the central battlefield is the AI safety and development debate itself. The essay suggests each side has scored points: Rationalists correctly foresaw AI's transformative potential early on with little data, a win for logical priors. Yet, their specific catastrophic risk scenarios often rely on theoretical chains of reasoning that critics find unmoored from empirical reality. The opposition, drawing from empiricist traditions, argues for priors based on historical tech adoption curves and the consistent failure of past apocalyptic predictions, urging a more evidence-based, incremental outlook. This unresolved conflict dictates whether we approach AI with radical caution or measured optimism.
- The debate is framed as a 'Third War of Priors,' continuing historical clashes between rationalist deduction and empirical experience for forming beliefs.
- Bay Area Rationalists' early flagship project—prediction markets for forecasting—has largely failed to deliver on its promise, instead fueling gambling and insider trading.
- The AI safety debate is the current battleground, with Rationalists winning on predicting AI's importance but struggling to ground doomsday scenarios in empirical evidence.
Why It Matters
This philosophical clash directly shapes trillion-dollar investment decisions, regulatory approaches, and the very trajectory of AI development.