“Following the incentives”

AI safety researcher David Scott Krueger (formerly capybaralet) published a widely discussed essay titled 'Following the Incentives' on the LessWrong forum. The piece uses analogies from politics and academia to analyze the behavioral dynamics within the AI industry, arguing that developers and companies are often driven by 'apparent short-term incentive-like vibes'—such as the pressure to publish papers, release models quickly, or capture market share—rather than genuine, long-term incentives aligned with safety and societal benefit. Krueger suggests this creates a dangerous trap where competitive pressure ('if I don’t, someone else will') leads to a race that prioritizes speed over thoughtful contribution.

Krueger critiques the common 'one-shot thinking' fallacy, where actors assume cooperation is impossible because there's no immediate incentive. He contends this leaves value—and safety—on the table, advocating for building trust and enforcement mechanisms to enable better outcomes. The essay's core message is that a person's (or organization's) moral strength is defined by their ability to identify and resist these misaligned incentives. For the AI field, this means consciously choosing to work on robust, valuable advancements even when the apparent 'vibe' rewards cutting corners.