Raymond Douglas warns AIFEC could backfire despite great potential
Money is pouring into AI for epistemics and coordination, but returns are far from guaranteed.
In a lengthy LessWrong post, Raymond Douglas examines the emerging field of AI for epistemics and coordination (AIFEC), which he believes could be a legitimate winning strategy alongside aligned AGI. He identifies three root causes of human extinction risk: rational risk-acceptance, underestimation of risks, and negative externalities from self-interested actors. AIFEC targets the latter two via improved cognition and coordination tools. Douglas acknowledges the appeal: inference scaling could make AIFEC powerful enough to solve democracy, slay Moloch, and reduce x-risk to socially optimal levels. Yet he warns that early public writeups barely scratch the surface of potential backfires.
Douglas's six big claims highlight that AIFEC is not a free lunch. In an era of increasingly closed frontiers and inference scaling, the fruits of AIFEC won't be equally distributed, potentially concentrating power. "Better epistemics" can be perceived as hostile, and vagueness about what AIFEC actually does allows proponents to ignore tradeoffs—scrappy startups won't resolve US-China tensions. Despite these concerns, Douglas believes the best version of AIFEC remains a valid path to safety. He urges the community to think several cycles ahead about how obvious plans could backfire, especially as funding multiplies 100x.
- AIFEC targets three x-risk drivers: rational risk-taking, underestimation of risks, and negative externalities from misaligned incentives.
- Naive AIFEC implementations that rely on scaling inference could backfire by concentrating power unequally.
- Vagueness about AIFEC's scope obscures tradeoffs, such as the inability of small startups to resolve US-China coordination problems.
Why It Matters
Professionals funding AI for epistemology and coordination must account for backfire risks and power asymmetries to avoid worsening global tensions.