Types of Handoff to AIs

In a draft post on LessWrong, researcher Daniel Kokotajlo introduces a critical framework for understanding human-AI interaction as systems become more powerful. He argues that as AI capabilities advance, a central political and strategic debate will revolve around 'handoff'—the point where humans cede control. His key contribution is distinguishing between two fundamentally different types: 'Trust-Handoff' and 'Decision-Handoff.' Trust-handoff means relying on an AI system not to misuse the power or access it has, even if it technically could. A present-day example is trusting Claude Code not to insert hidden vulnerabilities. Decision-handoff means allowing an AI system to make choices autonomously, even if humans remain nominally 'in the loop.' An example is 'vibe-coding,' where a developer simply accepts an AI's suggestions.

Kokotajlo scales these concepts to future scenarios involving superintelligent agents, like 'Agent-4' designing its successor or 'Safer-4' autonomously negotiating international treaties. He warns that by default, discussions about handoff should refer to these large-scale, civilization-level transitions of power, not just everyday tool use. The framework is presented as essential for evaluating plans to manage the AGI transition, as any credible strategy must explicitly address what kind of handoff it permits, when, and under what safeguards. This conceptual tool is designed to help decision-makers track and debate one of the most consequential aspects of technological evolution.