Is the Cat Out of the Bag?: Who knows how to make AGI?

Oliver Sourbut, in a memo adapted for AI safety organization AISI, contends that the knowledge required to build AGI might already be broadly accessible. He points to Moore's Law and Wright's Law, which predict compute costs will continue to drop exponentially, making powerful AI development feasible for many actors. Sourbut emphasizes that AI techniques are typically simple and easy to replicate once discovered, and that economic and social forces strongly favor proliferation. He warns that even the 'stupidest, simplest possible approach'—mimicking evolution in a large environment—could succeed with enough compute, potentially within decades, allowing anyone to create AGI without deep expertise.

Sourbut further argues that the design space for AGI is narrowing, with key components like long-horizon coherence, continual learning, and context management becoming more predictable. He lists potential solutions such as context summarization, read-write RAG, recurrent embeddings, and explicit training for notetaking. Drawing on his own experience, he notes that by 2020-2021, landmarks in NLP and RL were visibly converging, and a plausible research path to general autonomous AI was discernible. Developments like scaling, mixture of experts, chain-of-thought, and RL reasoning were advance predictable, not just hindsight. This narrowing design space, combined with cheap compute, means that soon, practically anyone could stumble into creating AGI, raising urgent safety and governance concerns.