LessWrong essay: 'Big-World Intuitions' says stop overthinking AI risks

Sarah Constantin's LessWrong post 'Big-World Intuitions' argues that many situations call for non-consequentialist heuristics rather than explicit game-theoretic optimization. When a startup faces a huge market, a trader operates in a liquid market, or a species is r-selected (with resources far exceeding needs), the optimal move is to ignore the environment and focus on maximising your own value—whether that's customer growth, truthful bidding, or offspring count. Even in repeated games, being trustworthy beats betrayal because long-term reputation costs dominate short-term gains.

Constantin extends this to AI development: when working on a hard technical problem far from solved, worrying about 'what if we succeed and it's harmful?' or 'what if this falls into the wrong hands?' is akin to a chess player in the opening fretting about the endgame. The heuristics 'do good work' and 'share knowledge freely' are more robust than complex utilitarian justifications. She contrasts this with truly strategic situations—like endgames or oligopolies—where you must model other agents' reactions. The essay has sparked discussion on LessWrong with 73 upvotes, challenging the AI safety community to question whether 'big-world' thinking applies to transformative AI or whether we're already in an endgame.