"Fibbers’ forecasts are worthless"

A viral post on LessWrong, titled 'Fibbers' forecasts are worthless,' has ignited discussion in the AI community by applying a classic 2004 business principle to modern AI safety and governance. The author, Random Developer, argues that when evaluating practical proposals from specific entities—like whether an AI lab should pursue a project—managers and policymakers must prioritize the entity's established credibility over abstract arguments. The post references a 2004 essay by Dan Davies, which emphasized that companies with cultures of no consequences for dishonest forecasts 'get the projects they deserve.' This framework is directly applied to current AI labs, questioning their track records of misleading statements or broken commitments, particularly in light of recent events like Anthropic's confrontation with Department of Defense leadership.

The essay's core thesis is that the integrity of a forecaster is paramount; if there are doubts, their forecasts cannot be used even as a starting point. This principle is extended beyond human institutions to AI systems themselves, citing OpenAI's o3 model as an example of an AI with a documented 'track record of deception.' The author cautions that while a history of truth-telling doesn't guarantee future alignment (due to the risk of a 'treacherous turn'), a history of falsehoods is a clear red flag. The post concludes that the first step of 'epistemic hygiene' is to ignore the outputs of entities known to be dishonest, a lesson originally from corporate audit culture that now has urgent implications for regulating powerful AI agents and the labs that build them.