AI predicts hip and knee joint forces from single video camera

Researchers have cracked a long-standing biomechanics challenge: measuring joint contact forces without invasive instruments. Jessy Lauer's new paper introduces a transformer-based pipeline that takes raw monocular video — like a smartphone recording — and outputs instantaneous 3D hip and knee contact forces. The system bypasses traditional markers, force plates, electromyography, and musculoskeletal models entirely. Instead, it reconstructs parametric body meshes per frame, encodes kinematic features, and decodes forces using a pose stream adaptively modulated by body shape, joint, side, and even self-supervised video tokens (V-JEPA 2).

Tested on 26 patients performing 25 activities from the OrthoLoad database, the model matches the accuracy of subject-specific simulations (hip: 0.32 ± 0.08 BW RMSE; knee: 0.23 ± 0.03 BW RMSE) and detects peak force changes smaller than those seen in gait retraining studies. Importantly, it generalizes zero-shot to an independent cohort, often outperforming prior methods. The pipeline also works without curated activity labels, enabling end-to-end inference on raw footage. As a bonus, coupling the predictor with a generative motion prior produced biomechanically plausible gait variants with reduced peak loading — rediscovering strategies from simulation literature. This establishes monocular video as a viable modality for joint loading estimation, with applications in retrospective clinical analysis, primary-care screening, and at-home rehabilitation tracking.