LACA: New model improves dual-hand action segmentation with 5% F1 gain

Dual-hand action segmentation typically fuses left- and right-hand features at the same time step, but real hand movements often occur with nonzero, variable delays. Fatemeh Ziaeetabar’s paper introduces Lag-Aware Cross-Hand Alignment (LACA), a lightweight module that learns to estimate directional temporal offsets between hand-specific feature streams. By retrieving cross-hand information from these offsets and suppressing transfer when no compatible transition exists, LACA aligns representations more accurately. The module is trained using only existing frame-level annotations, requiring no additional labels.

Integrated into the Polyphony architecture, LACA boosts two-hand mean F1@50 from 40.4 to 42.5 on HA-ViD and from 19.9 to 21.8 on ATTACH, while boundary F1 rises from 56.5 to 59.6 and 44.7 to 47.9 respectively—all with just 0.0086 million additional parameters. A future-free variant, LACA-C, achieves 83.6% transition-cue recall, 233ms median availability delay, and 224.9 predictions per second. These results demonstrate that explicit cross-hand temporal alignment significantly improves action segmentation and boundary localization for dual-hand tasks, making it practical for real-time applications in robotics, AR/VR, and human-computer interaction.