PhysAgent's multi-agent framework makes remote heart-rate tracking reliable

A team of researchers led by Yehui Yang (including Bo Zhao, Junzhe Cao, Hui Ma, Yue Sun, Wenjin Wang, and Zitong Yu) has introduced PhysAgent, a novel inference-time framework for remote photoplethysmography (rPPG). rPPG estimates heart rate from facial video, but the signal is notoriously fragile—motion, lighting, occlusion, and skin variations corrupt it easily. Existing methods rely on a single model to directly predict heart rate or pulse waveforms, and when multiple strong estimators disagree, there's no principled way to reconcile them. PhysAgent reframes this problem as one of candidate verification rather than direct prediction.

PhysAgent works by treating the outputs of multiple base rPPG estimators as physiological hypotheses. A lightweight 4-billion-parameter multimodal LLM, Qwen3-VL-4B, drives multi-agent reasoning over video conditions, signal reliability, and disagreement among candidates. A deterministic physiological verifier then checks the proposed fusion, and a reproducible numerical fusion process produces the final heart rate. The key advantage: it avoids the irreproducibility and physiological inconsistencies that come from asking an LLM to output a number directly or from unconstrained ensemble fusion. Experiments on multiple public rPPG benchmarks show that PhysAgent significantly improves fusion stability and reliability, even across different source domains. The code is set to be released soon.