Meta's cloud-boosted speech enhancement doubles quality on low-compute wearables

Low-latency, low-compute speech enhancement is critical for wearable devices with real-time communication, but strict computational limits cap on-device performance. In a new paper accepted to Interspeech 2026, Meta researchers propose a collaborative framework that leverages a more powerful server-side model to guide edge inference without burdening the wearable. The framework introduces three key techniques: (1) delayed server output fed as additional input to the edge model, (2) layerwise feature boosting that transfers intermediate server representations to guide edge layers, and (3) collaborative multichannel Wiener filtering, which fuses weighted covariance matrices from both server and edge models to improve beamforming.

Experiments show the proposed system significantly outperforms an edge-only baseline while adding minimal computational overhead, making it practical for real-time use on low-power hardware. The approach builds on earlier "knowledge boosting" ideas but achieves stronger gains by enabling deeper server-edge collaboration rather than using server outputs only at the loss level. The authors report clear improvements across speech quality and intelligibility metrics, with the collaborative beamforming contributing the largest gains. This work could enhance voice calls, hearing aids, and AR/VR headsets where on-device compute is scarce but cloud connectivity is available.