Anthropic's Jacobian Lens Reveals LLM 'Global Workspace' in J-Space
New interpretability technique finds ~25 conscious-like concepts that drive model reasoning.
Deep Dive
Anthropic's paper introduces the Jacobian Lens, a technique that identifies a 'J-space' of verbalizable representations in LLMs. This J-space exhibits properties of a global workspace: it holds at most a few dozen concepts at a time (tracked ~25, though effectively ~6 distinct due to overlap). Injections into J-space can be verbalized and alter outputs, while ablating J-space impairs internal reasoning
Key Points
- Jacobian Lens computes per-layer causal effects to identify verbalizable representations (J-space).
- J-space holds ~25 concepts at once (~6 distinct after overlap), acting as a 'global workspace' for reasoning.
- Ablating J-space impairs abstract tasks like translation but leaves basic output coherence intact.
Why It Matters
Gives unprecedented visibility into LLM reasoning chains, enabling safer steering and deeper alignment understanding.