CNNs turn 4-microphone arrays into 32-channel virtual arrays for sound imaging

A team of researchers (Adamopoulou, Sudarsanam, Diaz-Guerra, et al.) has developed convolutional neural network models that can algorithmically upscale the spatial resolution of a compact 4-microphone tetrahedral array to match the performance of a 32-microphone spherical array. The paper, published at IEEE AI4IM 2026 and available on arXiv, focuses on covariance matrix upsampling for acoustic imaging. The models take the 4-channel time-frequency covariance matrix as input and predict the full 32-channel covariance matrix, effectively creating a virtual high-density array from sparse sensor data.

Five different 2D CNN architectures were tested, each incorporating frequency dynamic convolution to capture frequency-dependent spatial patterns. Using the real-world STARSS23 dataset for training and evaluation, the best-performing model achieved a root mean square error (RMSE) of 0.432, significantly outperforming a random-guess baseline (0.548). Qualitative analysis via delay-and-sum beamforming heatmaps showed that sound maps produced from the predicted 32-channel covariance matrices were visually nearly identical to those obtained from actual 32-microphone recordings. This approach could drastically reduce hardware costs and complexity for applications like drone-based acoustic surveillance, industrial noise monitoring, and augmented reality audio, where multiple microphones are often impractical.