On the Formal Limits of Alignment Verification

A new research paper by Ayushi Agarwal, titled 'On the Formal Limits of Alignment Verification,' establishes a foundational impossibility result for AI safety. The work proves that no procedure can formally certify that an AI system is 'aligned'—meaning it reliably pursues its intended objectives—while simultaneously satisfying three desirable properties: soundness (never certifying a misaligned system), generality (holding over the system's full input domain), and tractability (running in polynomial time). This creates a formal trilemma: you can achieve any two of these properties, but never all three at once.

The impossibility stems from three independent mathematical barriers: the computational complexity of verifying neural networks over all possible inputs, the fundamental non-identifiability of an AI's internal goals from its external behavior alone, and the limits of finite testing for properties defined over infinite domains. The result does not mean alignment assurance is hopeless, but it precisely delineates the trade-offs. Practical safety engineering must therefore relax one constraint, opting for bounded verification (sacrificing generality), probabilistic guarantees (sacrificing soundness), or domain-specific checks (which may be intractable to scale). This paper provides a rigorous framework for understanding the inherent limitations of AI safety certification, guiding future research toward viable, if imperfect, assurance methods.