Spatial-Magnifier: Spatial upsampling for multichannel speech enhancement

Spatial-Magnifier, introduced by Dongheon Lee and colleagues at Meta, addresses a fundamental trade-off in multichannel speech enhancement: larger microphone arrays yield better directivity but are impractical for compact edge devices. The proposed neural network takes signals from just a few real microphones (RMs) and generates multiple virtual microphone (VM) signals, effectively upsampling the spatial array. A companion framework, Spatial Audio Representation Learning (SARL), uses these estimated VM signals and features to condition a downstream speech enhancement system.

Experimental results on standard benchmarks show that Spatial-Magnifier outperforms existing spatial upsampling baselines across end-to-end multichannel speech enhancement and neural beamforming tasks. Most impressively, the approach nearly recovers the oracle performance achieved when all microphones are physically present. This means devices like smart speakers, hearing aids, or AR/VR headsets could achieve studio-quality speech capture without bulky arrays—just a few mics plus a lightweight neural network.