MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesis

A research team from KAIST and Korea University has introduced MamTra, a novel hybrid architecture for text-to-speech (TTS) synthesis that tackles a major bottleneck in current AI voice models. While large language model (LLM)-based TTS systems produce remarkable quality, their reliance on autoregressive Transformers creates quadratic computational complexity, making them slow and memory-intensive for practical deployment. MamTra proposes an elegant solution by interleaving layers of Mamba—a state-space model known for its linear-time efficiency—with Transformer blocks that excel at capturing global context. This hybrid design aims to capture the best of both worlds: the speed of Mamba and the expressive power of Transformers.

Crucially, the team developed innovative knowledge transfer strategies to distill the capabilities of a large, pre-trained Transformer TTS model into the new MamTra architecture. This bypasses the prohibitive cost of training a hybrid model from scratch on massive datasets. Their systematic experiments identified the optimal configuration for this fusion and yielded striking results. MamTra was able to reduce inference VRAM (Video RAM) usage by up to 34% while maintaining speech fidelity comparable to the full Transformer baseline. Perhaps most impressively, this performance was achieved after training on only 2% of the original dataset, demonstrating the efficiency of their distillation approach.

The work, submitted to Interspeech 2026, represents a significant step toward making high-quality neural speech synthesis more accessible and deployable. By dramatically cutting memory requirements without sacrificing output quality, MamTra addresses a key barrier for real-world applications on consumer hardware or at scale in cloud services. The hybrid approach also opens a new design paradigm for efficient generative AI, potentially influencing architectures beyond speech synthesis.