LLMs use dual mechanisms for in-context learning, study finds
Researchers show LLMs combine pattern-matching and latent structure inference in parallel.
A team of researchers from Tufts University (Kowalyshyn, Duggan, Little, Hughes) has published a study under review at the ICML Mechanistic Interpretability Workshop 2026 that unpacks how LLMs learn in-context. Using a carefully designed toy graph random-walk task—where the correct answer depends on either tracking global topology or copying local transitions—they found that neither mechanism alone suffices. Principal Component Analysis of internal representations revealed that at intermediate mixture ratios, both graph topologies are encoded in orthogonal subspaces concurrently. This suggests the model maintains two parallel representations rather than simply pattern-matching recent tokens.
To causally test this, the team applied residual-stream activation patching and a novel graph-difference steering method. Late-layer patching almost fully transferred the clean graph preference, while linear steering shifted predictions in the intended direction but failed under norm-matched and label-shuffled controls. These findings strongly support a dual-mechanism account: genuine structure inference (belief) and induction circuits (pattern-matching) operate in parallel. For AI practitioners, this means current LLMs are more sophisticated than simple next-token predictors—they simultaneously maintain multiple hypotheses about latent structure. This could inform more efficient training strategies and interpretability tools for debugging model behavior.
- PCA reveals that LLMs encode two competing graph topologies in orthogonal principal subspaces simultaneously during in-context learning.
- Late-layer activation patching transfers clean graph preference, while linear steering shifts predictions—but fails under norm-matched controls.
- Paper provides causal evidence for a dual-mechanism: genuine structure inference and induction circuits operate in parallel, not one or the other.
Why It Matters
This work refines our understanding of in-context learning, potentially leading to more interpretable and controllable LLM behavior.