StreamSplit cuts audio AI latency 4.7x on edge devices with adaptive RL

StreamSplit addresses a fundamental conflict in edge AI: large-batch contrastive learning (CL) models like CLAP and COLA require big batches for quality, but edge devices have limited memory and variable resources. Existing approaches either degrade model fidelity with small local batches or offload to the cloud, incurring high latency and bandwidth costs. The authors from Deakin University propose a distribution-based streaming framework that decouples representation quality from batch size using a tractable Hybrid Loss function. On top of that, they introduce an Uncertainty-Guided Adaptive Splitter — a lightweight reinforcement learning (RL) policy that monitors real-time resource availability and embedding ambiguity to decide how much computation to perform locally versus offloading. This dual approach adapts on the fly to runtime volatility, enabling continuous audio processing on heterogeneous ARM devices.

Evaluated on Raspberry Pi 4 and Apple M2 hardware, StreamSplit delivers dramatic efficiency gains: per-sample latency drops up to 4.7x, bandwidth usage falls 77.1%, and energy consumption reduces 52.3% compared to server-centric baselines. Crucially, accuracy stays within 2.2% of full cloud-based models, proving that adaptive distributed learning is viable. Accepted at ACM MobiSys 2026, the work has implications for always-on audio assistants, voice-controlled IoT devices, and real-time acoustic monitoring — anywhere continuous audio representation learning needs to run on constrained hardware without sacrificing quality.