WaveFT lands in Hugging Face PEFT, beating LoRA on vision
Sparse wavelet updates beat LoRA on vision using fewer parameters than LoRA's minimum rank.
A team of researchers from Koç University (Ahmet Bilican, M. Akın Yılmaz, A. Murat Tekalp, and R. Gökberk Cinbiş) has introduced WaveFT (Wavelet Fine-Tuning), a new parameter-efficient fine-tuning method that learns sparse updates in the wavelet domain of weight matrices. Unlike LoRA, which is constrained to low-rank updates and discrete rank choices, WaveFT enables fine-grained control over trainable parameters, going well below LoRA's minimum rank. The wavelet basis provides semi-local receptive fields that aggregate spatially coherent gradients, offering better coverage than direct weight sparsity (SHiRA) and avoiding the destructive interference seen with global Fourier bases (FourierFT).
WaveFT includes theoretical analysis showing that sparse methods can achieve high-rank updates, overcoming LoRA's subspace bottleneck for higher representational capacity. A gradient coverage framework explains when WaveFT is preferable. In experiments across text-to-image generation, image classification, and language understanding, WaveFT achieves state-of-the-art results among PEFT methods for vision tasks and performs comparably on NLP. The method is now officially included in the Hugging Face PEFT library, making it immediately available to practitioners.
- WaveFT learns sparse updates in the wavelet domain, enabling parameter counts below LoRA's minimum rank.
- State-of-the-art PEFT results on vision tasks (text-to-image generation, classification) with comparable performance on NLP.
- Now integrated into Hugging Face PEFT library for immediate use in fine-tuning workflows.
Why It Matters
WaveFT gives vision teams a more efficient fine-tuning method than LoRA, with finer parameter control and higher representational capacity.