Maia 3 Chess Transformer's Knight Fork 'Snap' Located After Block 5 via Logit Lens

In a mechanistic interpretability study, researchers applied the logit lens technique to Maia 3, an 8-block bidirectional encoder transformer that outputs a move policy (over 4,352 possible moves) plus a win/draw/loss evaluation. Unlike typical autoregressive transformers, Maia 3 processes a static board as 64 square tokens, allowing straightforward reading of the residual stream at each layer. The team focused on the knight fork—a single knight move attacking two valuable pieces—as a discrete, unambiguous tactic. They found that the knight-fork policy logit becomes decodable immediately after block 5's attention layer, with strong correlational evidence that the network 'sees' this tactic at that depth.

This result is part of a larger project aiming to causally describe a chess feature. Controls confirmed the snap is specific to the knight-fork move, not just the best overall move. Future parts will demonstrate causality and compositionality, and explore unexpected findings that complicate the simple 'knight fork attention head' thesis. The study bridges interpretability and cognitive science, as Maia 3's skill knob allows experiments on how representation changes with expertise. The hope is that understanding compressed representations in transformers can inform both AI safety and theories of human chunking and insight.