WHTMix slashes stereo depth compute by 2.46x with Walsh-Hadamard token mixing

Stereo depth estimation for autonomous driving, robotics, and augmented reality demands high resolution under tight latency budgets. Transformer-based matchers rely on global self-attention to aggregate scene context, but this operation grows quadratically with pixel count and dominates runtime. A new paper from researchers at the University of Illinois at Chicago proposes WHTMix, which replaces the joint self-attention stage of a stereo transformer with a data-independent Walsh-Hadamard token mixer. This mixer operates in the transform domain at log-linear cost, while retaining data-dependent cross-attention for left-right correspondence. On synthetic driving data, WHTMix achieves the same endpoint error as the attention baseline but reduces model compute by 2.46x and single-image inference latency by 2.65x.

The paper's complexity analysis shows that the benefit scales with the ratio of sequence length to channel width, making high-resolution stereo matching an ideal use case (unlike classification transformers, where token-to-channel ratios are different). The authors confirm this scaling on non-stereo long-sequence benchmarks. Additionally, they introduce a hybrid log-disparity loss function that up-weights small-disparity pixels corresponding to distant objects. This reduces error on long-range objects without additional computational overhead, a critical improvement for autonomous driving safety. WHTMix is a practical step toward efficient, real-time depth perception at high resolution.