Parameter Efficiency Is Not Memory Efficiency: Rethinking Fine-Tuning for On-Device LLM Adaptation

A new paper from researchers at MIT and KAIST challenges the prevailing assumption that parameter-efficient fine-tuning (PEFT) methods like LoRA and IA3 are also memory-efficient. The authors demonstrate that these methods still require intermediate tensors that scale linearly with sequence length, often causing out-of-memory errors on edge devices. To address this, they introduce LARS (Low-memory Activation-Rank Subspace), a framework that constrains the activation subspace during training rather than model parameters, directly targeting the primary source of memory consumption. LARS reduces memory footprint by an average of 33.54% on GPUs and 51.95% on CPUs compared to LoRA across reasoning, understanding, and long-context datasets, while maintaining competitive accuracy and throughput.

Beyond GPU benchmarks, the team successfully deployed LARS on a Raspberry Pi and consumer-grade CPUs, proving its viability for sophisticated LLM personalization on resource-constrained hardware. This opens the door for on-device adaptation of large language models without requiring cloud connectivity or expensive hardware. The paper, titled "Parameter Efficiency Is Not Memory Efficiency: Rethinking Fine-Tuning for On-Device LLM Adaptation," is available on arXiv (2604.22783) and challenges the broader AI community to rethink the true costs of fine-tuning.