Graphical tensor notation compresses frontier LLM to 33 PyTorch lines

Designing interpretable machine learning architectures often stumbles on representation: symbolic equations lack a global view, while probabilistic graphical models and flowcharts obscure actual tensor manipulations, hurting reproducibility. Pietro Barbiero's new paper, "Graphical Design of Interpretable Architectures," addresses this with a graphical notation adapted from Penrose tensor notation. The system provides an at-a-glance architecture diagram that maps one-to-one onto PyTorch einsum code, bridging high-level design and low-level implementation without ambiguity.

Barbiero demonstrates the notation by describing five major families of interpretable models: concept bottlenecks, sparse probes, prototype networks, neural additive models, and mixtures of linear models. He then applies it to Steerling-8B, a frontier interpretable language model. The resulting diagram exposes global structural insights—such as Steerling being a residual model—offers geometric interpretations of each operation, and translates directly into just 33 lines of PyTorch code. This suggests a powerful workflow for researchers and engineers: sketch an architecture visually, and the implementation follows mechanically.