AI Safety

Longtermist AI work: why trying less hard may improve your impact

New LessWrong essay argues hyper-focus distorts reasoning in high-uncertainty domains.

Deep Dive

Elias Schmied's LessWrong essay presents a novel argument against extreme effort in high-stakes domains: when the sign of your impact is highly uncertain (as it often is in AI safety and longtermism), pushing yourself too hard distorts your reasoning and makes negative outcomes more likely. He enumerates numerous ways AI safety interventions could backfire — from bad regulation and increased authoritarianism to adversarial dynamics with future AIs and safety-washing — and argues that these risks are often underconsidered because activists engage in motivated reasoning to justify their work.

The core claim is that "trying hard" reduces epistemic vigilance: it leaves less mental slack for examining counterarguments, biases evidence in favor of continued effort, and makes course-correction harder. Schmied specifically targets longtermist AI work, where uncertainty about the sign of impact is highest, but notes the argument applies more broadly. He suggests that instead of avoiding these uncomfortable uncertainties, practitioners should build robustness into their approaches — for example by maintaining capacity to change direction and keeping epistemics in good shape. The essay does not dismiss AI safety work but calls for a more reflective, less driven approach to maximize positive expected impact.

Key Points
  • Trying hard distorts epistemics through motivated reasoning and reduced mental slack for reflection.
  • Many AI safety interventions carry hidden downside risks: bad regulation, adversarial AI relations, safety-washing, and capability externalities.
  • The argument applies most strongly to longtermist work, where uncertainty about the sign of impact is highest.

Why It Matters

Challenges the 'move fast and break things' ethos in AI safety, urging deliberate, epistemically humble work.

📬 Get the top 10 AI stories daily