New 'Lift Spectrum' framework makes single-pixel sensing robust at 3% sampling

Single-pixel sensing captures a scene via a short sequence of coded measurements, and image-free methods infer tasks directly from that sequence without reconstructing the full image. This paper from a large team of researchers (Han et al., submitted to IEEE TCI) reframes the central challenge as the 'lift'—the transformation from 1D measurements to a 2D representation, which prior work treated as a trivial reshape. They introduce the 'lift spectrum,' a continuum from fixed-physics inverses through learned static projections to content-adaptive retrieval, showing that where a method sits on this spectrum predicts its robustness as acquisition quality degrades.

Their proposed spatiotemporal soft-fusion (STSF) network uses a probe-selected recurrent encoder and a cross-attention lift, paired with task-prioritized loss scheduling (TPLS). In simulation, STSF+TPLS surpasses all prior image-free baselines on three datasets at 3.13% sampling (+3.2 to +9.9 pp foreground mIoU) and maintains performance down to 0.39% sampling. Critically, under calibrated measurement noise, image-free inference overtakes the best reconstruct-then-segment baseline—because the reconstruction pipeline amplifies noise before segmentation. The method transfers without fine-tuning to a real single-pixel optical bench, delivering masks in ~14 ms. The work turns a scattered design space into a practical map for choosing the right lift at each operating point.