Functional Reconstruction boosts MLA draft acceptance in 37 of 64 tasks

Multi-head latent attention (MLA) is a key technique for long-context LLM inference because compact latent states replace the growing KV cache and slash memory traffic. However, most capable open checkpoints still use MHA or GQA, so teams rely on conversion methods like TransMLA or MHA2MLA to get MLA's cache efficiency without retraining. The problem: direct conversion introduces attention-function errors from low-rank factorization and RoPE handling that barely hurt standalone generation but sharply lower draft-token acceptance in speculative decoding — the very mechanism that accelerates inference.

Enter Functional Reconstruction. Instead of treating conversion as cache compression, the authors formulate it as functional reconstruction: each converted MLA module is optimized to reproduce the post-output-projection response of its original MHA/GQA counterpart on calibration hidden states. The method is converter-agnostic, requires no verifier logits, and preserves the converted cache and inference graph. Across 192 model-converter-backend-method-task configurations, it materially improves acceptance in 37 of 64 matched task cells (within a 0.5-percentage-point tolerance), leaves 26 unchanged, and degrades only one. That's a strong signal for anyone running speculative decoding on converted MLA models.