Vigouroux & Sharkey's tPD cuts interpretability compute by 93%

A new paper from Vigouroux and Sharkey, accepted at the ICML 2026 Mechanistic Interpretability Workshop, tackles a key bottleneck in neural network interpretability: the immense compute cost of parameter decomposition (PD). Full PD breaks networks into faithful, interpretable components but doesn't scale well. Their solution, targeted PD (tPD), introduces a high-rank catch-all component that handles all non-target data. This allows tPD to focus compute only on circuits processing specific inputs—from isolated prompts to broad subtasks—slashing resource requirements by an order of magnitude. They demonstrate this on transformer language models trained on The Pile, recovering a CSS-only submodel from a 4-block transformer using just 7% of the FLOPs needed for the full decomposition.

The team's validation extends to more complex scenarios. In a 12-block transformer, they surgically ablated and rewired memorized sequences with negligible side effects on other inputs, proving tPD's precision. This opens the door to practical, fine-grained mechanistic analysis of large models—enabling researchers to edit specific behaviors (like memorized facts) without retraining. By targeting only relevant circuits, tPD makes interpretability cost-effective for production-scale models, a critical step toward safe and transparent AI.