LessWrong post challenges rationalist dogma on AI doom probabilities
A critique of absurdity bias and Pascalian wagers in AI risk arguments
Manueldelrio's 'Map and Territory, Predictably Wrong' distinguishes epistemic from instrumental rationality and notes that strong emotions can be rational. The author is skeptical about fully overcoming cognitive biases, specifically highlighting absurdity bias—the tendency to dismiss events that have never happened as impossible. He connects this bias to discussions of an AGI apocalypse and expresses deep dislike for Pascalian wagers, arguing that such reasoning should be tightly bounded and that highly unlikely speculative events should not be assigned enough expected value to become urgent.
- Manueldelrio argues that truth-seeking for moral reasons is superior to curiosity or practical goals
- He casts doubt on the efficacy of de-biasing techniques, citing low observed alpha from rationalist practitioners
- The author explicitly rejects assigning non-zero probabilities to unprecedented events like AGI catastrophe, calling out Pascalian wagers
Why It Matters
This critique of rationalist core assumptions may influence how AI safety advocates frame existential risk arguments.