DINOv2 framework predicts cardiotoxicity from echocardiograms with 70% AUC
Google's DINOv2 handles three heart-health tasks without needing cardiac cycle annotations.
A new paper from Xiaotong Zhang and colleagues introduces a unified framework built on DINOv2 to tackle three complementary cardio-oncology tasks: estimating left ventricular ejection fraction (LVEF), classifying global longitudinal strain (GLS) dysfunction, and predicting early cardiotoxicity. DINOv2 is a self-supervised vision transformer that provides strong general image representations. The team froze the backbone and added parameter-efficient Low-Rank Adaptation (LoRA) alongside temporal aggregation to learn task-specialized features without fine-tuning the entire network.
The framework was evaluated on a patient-level split of 1,203 training videos from 237 patients and 300 validation videos from 59 independent patients. It achieved a mean absolute error of 5.03% for LVEF estimation, an AUC-ROC of 76.48% for GLS dysfunction classification, and an AUC-ROC of 70.26% for cardiotoxicity prediction. Notably, the inference pipeline operates without cardiac cycle segmentation or explicit end-diastolic/end-systolic annotations, simplifying real-world deployment. For Task 1, a specialized ED/ES-guided 2D/3D hybrid multi-view regression model further lowered the MAE to 4.64%.
- DINOv2 backbone with LoRA and temporal aggregation handles LVEF estimation, GLS classification, and cardiotoxicity prediction in one framework
- Achieves 5.03% MAE on LVEF (4.64% with specialized ED/ES-guided model), 76.48% AUC for GLS, 70.26% AUC for cardiotoxicity
- Runs without cardiac cycle detection or ED/ES annotations, enabling use on raw echocardiography videos
Why It Matters
Could enable earlier, automated detection of heart damage in cancer patients, improving preventive cardio-oncology care.