Meta's V-JEPA powers video-based coastal wave monitoring from monocular cameras

A team led by Abubakar Hamisu Kamagata has introduced a video-based, high-performance computing (HPC) approach for estimating coastal wave parameters without expensive in-situ sensors. Leveraging Meta's V-JEPA (a self-supervised Vision Transformer) as a backbone for robust spatiotemporal feature extraction, the architecture also uses a dual-stream SlowFast encoder to capture both breaking and swell wave regimes, plus Farneback optical flow to highlight hydrodynamic activity. The multi-task regression layer predicts five parameters simultaneously: significant wave height (Hs), maximum wave height (Hmax), peak period (Tp), zero upcrossing period (Tz), and wave direction (θ).

Trained on an NVIDIA DGX A100 cluster with just six annotated monocular video scenes, the model achieved Pearson correlation coefficients ranging from 0.451 (Hs) to 0.832 (wave direction) on geographically diverse held-out test data. Though R² values (max 0.246) indicate limited variance capture due to the small dataset, the results confirm proof-of-concept feasibility for sensor-free coastal monitoring. The method promises to reduce deployment costs and overcome the poor spatial coverage and storm vulnerability of traditional wave buoys.