ICML paper: AI must mirror human reasoning for high-stakes decisions

A team of researchers led by Vijay Keswani has published a position paper accepted at ICML 2026 arguing that current AI alignment methods are insufficient for high-stakes decision-making. The paper, titled "Position: We Need Practical AI Alignment Methods to Mirror Human Reasoning," contends that AI systems deployed as decision aids, delegates, or autonomous decision-makers must be 'cognitively aligned'—meaning they reason in ways similar to their users and transparently communicate that reasoning. The authors review existing evidence that cognitive alignment improves understandability and trustworthiness, then present new survey data showing many users find cognitive alignment 'essential' when an AI's rationale matters to them.

The authors outline significant gaps between today's alignment techniques—which focus on behavior, values, and intent—and what true cognitive alignment requires. They propose a concrete research agenda to bridge these gaps, including developing methods to verify that AI reasoning mirrors human reasoning and to make that reasoning faithfully explainable. The paper warns that cognitive misalignment could severely limit AI adoption in domains like healthcare, finance, and criminal justice, where users must be both willing and justified to rely on AI judgments. By reframing alignment around cognitive mirroring, the work challenges the AI community to move beyond outcome-focused safety and toward process-focused transparency, potentially shaping future alignment research priorities.