Anthropic's J-space method reveals LLMs use a tiny verbal thinking subspace

Anthropic’s latest research, detailed in the paper *Verbalizable Representations Form a Global Workspace in Language Models*, introduces the concept of a “J-space”—a low-dimensional subspace within the residual stream of transformers that corresponds to verbal thinking. The approach starts with a simple question: can we identify directions in residual space that increase the likelihood of a specific token being output soon? To probe this, the authors compute a Jacobian matrix that measures how a small perturbation at a given layer and token position affects the unembedding of a future token. By averaging this Jacobian across many prompts and token positions, they obtain a single global Jacobian per layer. The key empirical finding is that this global Jacobian has a rank much smaller than the full model dimension (d_model), suggesting that the transformer only uses a small fraction of its residual space for verbal reasoning. This subspace is the J-space.

The paper makes several simplifying assumptions, such as using tokens as proxies for verbalizable concepts and treating the zero vector as a baseline for perturbations—a point the author of this LessWrong post finds potentially unjustified. Despite these assumptions, the resulting methodology provides a tractable way to identify the “thinking” subspace in LLMs. The J-space could enable new interpretability techniques: for example, projecting activations into this subspace to monitor or steer verbal reasoning. The author suggests the work is more valuable for the questions it opens than as a finished tool, noting low-hanging fruit for replication and extension, including building open-source J-space libraries and creating benchmarks for non-LLM networks. Overall, it offers a fresh lens on how LLMs internally represent and process language.