There should be $100M grants to automate AI safety

Richard Ngo, a researcher at Apollo Research, has published a viral proposal arguing that AI safety funders must radically shift their spending strategies. He contends that under potential short-timeline scenarios for advanced AI, current "normal" spending—on competitive salaries and organizational growth—is insufficient. The core of his argument is that the only way to spend the massive sums required (potentially $1-50B per year) quickly and effectively is to fund the automation of safety work itself. This means creating grants specifically designed to purchase vast amounts of compute and API access to run automated safety pipelines at scale.

Ngo outlines a concrete two-step grant structure. Step 1 is a smaller, initial grant (e.g., $5M) where a team demonstrates a scalable safety pipeline with empirical evidence. A key component is agreeing on a "scaling condition," such as a plot showing how increased spending on the pipeline correlates with a validated safety proxy, like the number of distinct, egregious misalignment features discovered through interpretability. If this condition is met, Step 2 triggers a much larger follow-on grant (e.g., $14M) to massively scale the pipeline's operation, with the majority of funds directed at compute costs rather than human labor.

The proposal is a direct challenge to the conservative, human-centric funding models that have dominated AI safety philanthropy. It pushes funders to think in terms of capitalizing automated AI agents that can perform safety research and evaluation, arguing this is the only plausible path to achieving safety at the scale and speed that may be required. The call for $100M+ grants specifically for compute marks a significant evolution in how the field conceptualizes resourcing its most critical challenge.