What Physics do Data-Driven MoCap-to-Radar Models Learn?
Two new metrics measure whether models truly understand Doppler physics, not just reconstruction.
A new paper from Kevin Chen, Kenneth W. Parker, and Anish Arora tackles a critical question in AI for signal processing: do data-driven models that convert motion capture (MoCap) data to radar micro-Doppler spectrograms actually learn the underlying physics, or are they just pattern-matching? The authors introduce a physics-based interpretability framework with two complementary metrics. The first measures alignment between model-predicted spectrograms and the theoretically derived Doppler frequency from the MoCap motions. The second tests whether the model preserves the fundamental velocity-frequency relationship when the input velocities are artificially altered—a kind of intervention test to see if the model’s internal representations respect physical laws.
Testing several model architectures, the team found a surprising result: low reconstruction error (e.g., mean squared error) does not correlate with physical consistency. Some models achieved excellent reconstruction but failed both physics metrics, meaning they generated plausible-looking spectrograms without encoding the correct physics. For transformer-based models, temporal attention mechanisms proved critical for learning the physics—models that lacked meaningful attention weights over time performed worse. This has implications for applications like radar-based activity recognition, autonomous driving, and defense, where models must generalize to unseen motions rather than just memorizing training data. The framework requires only MoCap input and model predictions, no measured radar data, making it widely applicable.
- Two novel metrics measure physical alignment of MoCap-to-radar models: Doppler frequency consistency and velocity-frequency relationship preservation under input perturbation.
- Low reconstruction error (e.g., MSE) does not guarantee that models have learned correct physics—some high-performing models failed both physics tests.
- For transformer models, temporal attention is crucial: models without proper attention mechanisms cannot learn the underlying velocity-Doppler physics.
Why It Matters
Ensures MoCap-to-radar models generalize beyond training data, crucial for safety-critical applications like autonomous driving and surveillance.