Manifestation Units Protocol Makes AI Models Interpretable via Structured Retrieval
New typed tuple (E,S,R,D,G) plus attention primitives outperforms unstructured baselines on retrieval tests.
Mechanistic interpretability has produced a rich inventory of component-level analyses describing what neural-network components encode and how they interact. However, these outputs remain locked in per-study notebooks—non-composable and not directly actionable for downstream audit or intervention. To solve this, Hussein Chouman and colleagues introduce the Manifestation Unit Protocol, a structured representation layer that sits between these analyses and downstream use. The protocol defines a typed tuple (E, S, R, D, G) extended with attention-head primitives (T) for transformer architectures. It organizes per-component statistics into structured fields automatically populated and queried through hybrid retrieval (natural language + vector search).
Instantiated across generative vision (beta-VAE), discriminative vision (CNN), and language (GPT-2), the protocol shows that typed structure substantially outperforms unstructured baselines on retrieval tasks. CNN filters retrieved via the schema satisfy causal sufficiency and necessity criteria under matched-budget controls. The schema absorbs attention-head primitives without modification and set-recovers known IOI circuit members under retrieval-budget-matched controls. Notably, the analysis reveals an irreducible two-field core (S+R), with remaining fields either redundant or actively interfering. The paper positions this as schema infrastructure for mechanistic interpretability rather than frontier-scale validation, and provides interactive demos.
- Manifestation Units use a typed tuple (E,S,R,D,G) + attention primitives (T) for transformers.
- Tested on beta-VAE, CNN, and GPT-2; typed structure beats unstructured baselines on retrieval.
- Reveals an irreducible two-field core (S+R); remaining fields are redundant or interfering.
Why It Matters
Makes AI interpretability outputs queryable and reusable, enabling practical auditing and intervention for deployed models.