LLMs reorganize internal geometry during in-context learning, paper finds

A new preprint from Hua-Dong Xiong and colleagues investigates how large language models (LLMs) achieve in-context learning (ICL)—the ability to adapt to new tasks from examples without updating parameters. The authors propose a geometric perspective: ICL depends on the online 'untangling' of neural representations, a concept borrowed from neuroscience where classification is viewed as making representations linearly separable. Using tasks where labels are defined by the model’s own internal representations with known structure, they find that ICL effectiveness correlates systematically with the representational geometry of the underlying classification. Successful ICL is accompanied by a geometric reorganization that increases online separability—the model dynamically reshapes its representation space as it processes in-context examples.

Further analysis reveals that LLM behavior during ICL is well described by a prototype-like algorithm that integrates evidence while simultaneously transforming the representation to aid classification. This offers a mechanistic account of ICL grounded in representational geometry, establishing it as a key constraint on what pretrained representations afford versus what ICL can exploit. The findings quantify the gap between static pretrained features and the dynamic reorganization that ICL performs, with implications for understanding and improving how LLMs generalize from few examples.