MambAdapter: Mamba-based adapters beat Transformers on audio tasks

Fine-tuning large Transformer-based foundation models for audio and speech tasks is computationally expensive. To address this, parameter-efficient transfer learning (PETL) methods like adapters have emerged. Now, researchers from the University of Padua introduce MambAdapter, which replaces the usual Transformer bottleneck in adapters with a lightweight Mamba state-space model. Mamba, a recent alternative to Transformers, excels at sequence modeling with linear complexity. MambAdapter combines parameter sharing across adapter layers with a compact Mamba module, enabling efficient extraction of temporal audio features without the full parameter cost of fine-tuning.

Tested across four audio classification benchmarks (including speech commands and environmental sounds) and five speech recognition languages (English, French, German, Spanish, Italian), MambAdapter matches or outperforms established PETL methods like LoRA, AdapterHub, and BitFit. Remarkably, it does so with fewer trainable parameters—often 2-4x less than competing methods. This demonstrates that state-space models can effectively replace Transformers in low-rank adaptation for audio. The code is publicly available, and the paper has been accepted to Interspeech 2026. For practitioners, this means faster, cheaper fine-tuning of speech and audio models with minimal performance loss.