Spatial-Aware Conditioned Fusion for Audio-Visual Navigation

Researchers Shaohang Wu and Yinfeng Yu have introduced Spatial-Aware Conditioned Fusion (SACF), a novel AI architecture designed to solve audio-visual navigation tasks where agents must locate and move toward continuously vocalizing targets using only visual observations and acoustic cues. Unlike previous methods that relied on simple feature concatenation or late fusion, SACF creates an explicit, discrete representation of the target's relative position. It first discretizes the target's relative direction and distance from audio-visual inputs, predicts their probability distributions, and encodes this information into a compact spatial descriptor. This descriptor is then used to condition the agent's policy and model its state.

SACF's core innovation is its fusion mechanism. The model uses audio embeddings and the generated spatial descriptors to produce channel-wise scaling and bias parameters. These parameters modulate the agent's visual features through a conditional linear transformation, creating target-oriented fused representations that guide navigation. This method has demonstrated improved navigation efficiency with lower computational overhead compared to existing approaches. Crucially, SACF exhibits strong generalization capabilities, performing well even when presented with target sounds it has never encountered during training. The paper detailing this work has been accepted for publication at the International Joint Conference on Neural Networks (IJCNN 2026).