Krakauer's tool model shows AI use can irreversibly collapse human competence
Above a critical tool availability, user competence collapses to a low floor—and can't be recovered easily.
David Krakauer (Santa Fe Institute) introduces a formal dynamical model where an agent's competence and reliance on a tool co-evolve. The system is bistable: above a critical level of tool availability, the competent state collapses into a low-competence dependent regime. Crucially, reversing this collapse requires reducing availability far below the original threshold—a hysteresis effect that means history, not current access, determines the user's state. Two users with identical tool access can occupy opposite, lasting states depending on which they built first.
The collapse threshold depends jointly on the user's initial competence and the tool's transparency (the fraction of its internal workings the user can reconstruct). In goal-uncertain tasks, agency itself can transfer to the tool, turning the human into an agentic-instrument—irreversible when the tool's model is too large to internalize. Krakauer validates the model against multiple data sets: GPS and map navigation, arithmetic proficiency loss, and language model usage. The paper reframes how AI tools should be designed and what a tool-resistant education requires.
- Bistable dynamics: above critical tool availability, competence collapses to a low floor; lowering availability doesn't restore it until a far lower threshold.
- Collapse threshold depends on user's initial competence and tool transparency (fraction of workings the user can reconstruct).
- For uncertain goals, agency can transfer irreversibly from human to tool when the tool's model is too large to internalize (e.g., LLMs).
Why It Matters
Redefines AI deployment risks: competence collapse from tool overreliance may be irreversible, demanding new education and tool design.