Reliability-Aware Weighted Multi-Scale Spatio-Temporal Maps for Heart Rate Monitoring
A novel self-supervised learning approach uses weighted maps to filter noise, improving remote heart rate estimation.
A team of researchers from the Indian Statistical Institute, led by Arpan Bairagi, has developed a novel AI method to significantly improve the accuracy of contactless heart rate monitoring. The system tackles the core challenge of remote photoplethysmography (rPPG), where subtle skin color changes in facial videos are used to estimate heart rate but are easily corrupted by real-world noise like motion, shadows, and lighting changes. Their solution is a Reliability-Aware Weighted Multi-Scale Spatio-Temporal (WMST) map, which intelligently models pixel reliability to suppress these environmental artifacts and focus computational attention on physiologically valid facial regions.
The technical core of their approach is a self-supervised learning (SSL) framework built on a Swin-Unet architecture. It uses a contrastive learning strategy where positive training pairs are created from conventional rPPG signals and the enhanced WMST maps. Crucially, the team also introduces a synthetic 'High-High-High' wavelet map as a negative example—this map retains motion and structural details but filters out the physiological pulse signal, teaching the model what *not* to learn. This dual strategy forces the AI to isolate the true heart rate signal from confounding noise.
Experiments on standard public rPPG benchmarks demonstrate the method's effectiveness. The proposed system outperforms existing self-supervised rPPG techniques, achieving a lower heart rate estimation error and a higher Pearson correlation coefficient. This represents a meaningful step toward reliable, camera-based health monitoring that can function outside controlled lab environments, potentially enabling applications in telehealth, fitness tracking, and driver drowsiness detection using everyday devices.
- Introduces a Weighted Multi-Scale Spatio-Temporal (WMST) map to model pixel reliability and suppress noise from motion and illumination.
- Uses a self-supervised contrastive learning approach with a Swin-Unet, employing a synthetic 'HHH' wavelet map as a negative example to improve learning.
- Outperforms existing SSL-based methods on public benchmarks, achieving lower heart rate error and higher correlation for more robust remote monitoring.
Why It Matters
Enables more accurate, contactless health vitals monitoring via standard cameras, advancing telehealth and passive wellness tracking.