The Scaling Properties of Implicit Deductive Reasoning in Transformers

A new paper by Enrico Vompa and Tanel Tammet explores whether Transformers can perform deductive reasoning without explicit step-by-step prompts (Chain-of-Thought, or CoT). The authors focus on implicit reasoning over Horn clauses, a classic logical structure. They systematically decorrelate provability from spurious features and enforce algorithmic alignment to isolate true reasoning capability. The key finding: in sufficiently deep models using a bidirectional prefix mask (allowing the model to attend to both past and future tokens), implicit reasoning approaches the performance of explicit CoT across various graph topologies and problem widths. This suggests that deep transformers can internalize the reasoning steps, effectively doing 'thinking' in their hidden states.

However, the paper also reveals a critical limitation: when it comes to extrapolating to deeper reasoning chains than seen during training, explicit CoT remains necessary. The implicit approach hits a depth barrier, while CoT can generalize further. This implies that while deeper models can compress reasoning into implicit computations, they may still require overt step-by-step generation for novel, more complex logical problems. The results have implications for model architecture design, token efficiency, and the debate about whether explicit reasoning is always needed for AI systems. The work was submitted to arXiv in May 2026.