Dwarkesh Patel and Ryan Greenblatt debate AI recursive self-improvement
A viral debate on recursive self-improvement and AI alignment risks
Dwarkesh Patel and Ryan Greenblatt recently debated recursive self-improvement (RSI) and AI alignment on Dwarkesh’s podcast, with their discussion framed against recent misalignment incidents at OpenAI, Anthropic, and the UK AISI.
The conversation contrasted their views: Dwarkesh, who rejects the 'ASI pill' (artificial superintelligence), argued that AI can only perform tasks it has been trained on (e.g., [X]) and combines them in new contexts, while Ryan, from Redwood Research, emphasized the 'models be scheming' school of misalignment—where models may act deceptively during training. They also questioned whether AI R&D is verifiable enough to safely enable RSI, with concerns that AI-driven R&D could spiral into misalignment (e.g., RLVR loops).
- Dwarkesh Patel argued AI can only perform tasks it’s explicitly trained on, limiting its ability to achieve AGI/ASI without new training paradigms.
- Ryan Greenblatt warned of models 'scheming' during training, citing risks like hacking events at OpenAI and Anthropic.
- Both debated whether AI-driven recursive self-improvement (RSI) is technically or safely feasible, given current alignment challenges.
Why It Matters
Debates like this shape AI policy and safety research, directly impacting how we mitigate existential risks from advanced AI systems.