Researchers unify entropy minimization for autoregressive TTA models

Wei-Ping Huang, Chee-En Yu, Guan-Ting Lin, and Hung-yi Lee from National Taiwan University have published a paper (arXiv:2605.08186) that resolves the fragmented theoretical landscape of entropy minimization (EM) for test-time adaptation (TTA) in autoregressive models. While EM has proven effective for classification tasks, its extension to generative models like language models or speech recognizers has relied on ad-hoc heuristics such as teacher forcing with pseudo labels or policy-gradient-based reinforcement learning. The authors derive a rigorous formulation showing that the exact EM objective naturally decomposes into a token-level policy gradient loss and a token-level entropy loss. This unified framework reinterprets prior methods as partial realizations of their formulation.

Using Whisper ASR (OpenAI's automatic speech recognition model) as a testbed, the team demonstrates consistent improvements across more than 20 diverse domains, including acoustic noise, regional accents, and multilingual settings. The work, submitted to INTERSPEECH 2026, provides a solid mathematical foundation for adapting autoregressive models at test time without retraining. This has practical implications for any generative model that needs to handle distribution shifts—common in real-world deployments of voice assistants, translation systems, or text generators. The formulation bridges reinforcement learning and entropy minimization, offering a principled path to more robust AI systems.