Guide Labs' Steerling-8B proves interpretability scales with capability

Traditional interpretability is a post-hoc afterthought—models are trained as black boxes, then researchers attempt to reverse-engineer explanations with methods of questionable reliability. Guide Labs challenges this by making interpretability an explicit constraint of the training pipeline itself. The result is Steerling-8B, a diffusion language model with a causal attention mask optimized for both language modeling and human-understandable representations. Across three orders of magnitude of compute, the team found that representations become more disentangled and aligned with human concepts as models scale—contradicting the assumption that interpretability comes at the cost of capability.

Steerling-8B offers unprecedented transparency: for any group of generated tokens, it attributes output to relevant input tokens, human-understandable concepts, and the specific training data that influenced it. This enables a closed-loop workflow—diagnose an output via feature attribution, retrieve similar training examples, and correct behavior through concept steering without retraining. Despite the interpretability constraints, Steerling-8B remains competitive with open peer models trained on 2–16x more compute. This suggests a new scaling paradigm where interpretability is not a tax but a design principle that improves with scale.