Disentangling Geometry, Performance, and Training in Language Models

A new research paper titled "Disentangling Geometry, Performance, and Training in Language Models" challenges a common assumption in AI interpretability. The study, led by Atharva Kulkarni and colleagues, systematically investigates whether the geometric properties of a model's weights—specifically the effective rank of its unembedding matrix—can reliably predict its performance on downstream tasks. By training and analyzing a suite of 108 OLMo-style language models under controlled conditions, the researchers found that while high-performing models often have high effective rank, this relationship is not universal and can be misleading.

The technical analysis reveals that geometric metrics like effective rank are heavily influenced by pre-training hyperparameters such as batch size and weight decay, which in turn affect final performance. Crucially, the team identified adversarial cases where models with low effective rank did not exhibit the expected performance saturation, disproving a direct causal link. The research concludes that existing geometric metrics are largely aligned with each other but are more reflective of the training process itself than a reliable proxy for how well the model will perform on real-world tasks, urging caution in their use for performance estimation.