CODI and COCONUT: Latent CoT reasoning modeled as dynamical systems
Lyapunov exponents reveal two distinct stability classes in hidden AI reasoning.
A new paper on arXiv (2607.09698) tackles the interpretability problem of latent chain-of-thought (CoT) reasoning methods. Unlike explicit CoT which follows a single transparent reasoning trace, latent methods like CODI and COCONUT maintain multiple superimposed candidate traces in hidden space, making it difficult to understand how reasoning evolves. The researchers model latent token sequences as trajectories in representation space and apply dynamical systems analysis—using step-to-step change, direction consistency, and Lyapunov sensitivity—to characterize the evolution of reasoning.
Their quantitative and qualitative analysis (via UMAP, DMD/PHATE) reveals that latent CoT exhibits structured, non-random dynamics with two distinct stability classes: CODI behaves as a stable attractor, while COCONUT behaves as an unstable expanding system. They also find that SIM-CoT supervision tightens both behaviors without changing the underlying dynamics. This framework provides a new lens for interpreting latent reasoning and offers actionable insights for improving model reliability and transparency.
- CODI functioning as a stable attractor suggests robust convergence during reasoning.
- COCONUT's unstable expanding dynamics indicate potential for more diverse but less predictable inference paths.
- SIM-CoT supervision amplifies stability/instability patterns without altering fundamental dynamical behavior.
Why It Matters
Unlocking hidden reasoning structure helps build safer, more interpretable AI systems with predictable decision paths.