Maia-3 chessformer single head ablation kills Morphy's queen sacrifice
Ablating one of 128 attention heads makes Maia-3 miss the Opera Game's brilliant queen sacrifice.
Deep Dive
Ablating one of Maia-3 23m chessformer's 128 attention heads destroys the policy toward the brilliant queen sacrifice. Uses chessformer_lens: https://github.com/chessformer-lens/chessformer_lens. Discuss.
Key Points
- chessformer_lens is a new open-source Python toolkit for attention head ablation on the 23M-parameter Maia-3 chessformer.
- Removing one of the 128 attention heads eliminates the queen sacrifice policy in Morphy's Opera Game, while other moves remain mostly unchanged.
- The demo includes reproducible notebooks that let researchers verify how a single head encodes a complex tactical pattern.
Why It Matters
Localizing a complex chess tactic to a single attention head advances mechanistic interpretability, a key step toward understanding and controlling transformer behaviors.