Scaling Ultrasound Volumetric Reconstruction via Mobile Augmented Reality
A new AI system turns standard 2D ultrasounds into accurate 3D models using just a smartphone and AR.
A research team from Singapore has unveiled MARVUS (Mobile Augmented Reality Volumetric Ultrasound), a breakthrough system that democratizes 3D medical imaging by leveraging AI and smartphone-based augmented reality. The system addresses a critical gap in oncology: while 2D ultrasound (2D-US) is the preferred first-line tool for breast and thyroid imaging due to its cost and safety, it suffers from high variability in volume estimates between clinicians, complicating diagnosis and treatment planning. Existing 3D ultrasound solutions require expensive specialized probes or external tracking hardware, limiting widespread adoption.
MARVUS works by being interoperable with conventional ultrasound systems. It uses a foundation model to process 2D ultrasound data, enhancing cross-specialty generalization, and then employs a smartphone's AR capabilities to spatially track the probe and reconstruct a volumetric 3D model. This approach drastically minimizes hardware requirements. In a user study with experienced clinicians measuring breast phantoms, MARVUS demonstrated a mean improvement in volume estimation accuracy of 0.469 cm³ and reduced inter-user variability by 0.417 cm³. The AR visualizations were also proven to enhance both objective performance metrics and clinician-reported usability.
The implications are significant for global healthcare accessibility. By eliminating the need for costly dedicated 3D-US hardware, MARVUS provides a scalable, resource-efficient path to accurate volumetric assessment. This could standardize and improve cancer screening, diagnostic workflows, and treatment planning in both well-resourced and low-resource clinical settings, potentially transforming point-of-care diagnostics.
- MARVUS uses a foundation AI model and smartphone AR to create 3D models from standard 2D ultrasound data, requiring no specialized hardware.
- In clinical tests, it improved volume estimation accuracy by 0.469 cm³ and cut inter-user variability by 0.417 cm³ compared to traditional 2D methods.
- The system is interoperable with existing ultrasound machines, aiming to make precise 3D cancer screening scalable and cost-conscious worldwide.
Why It Matters
This could democratize precise 3D cancer diagnostics, making them affordable and accessible in clinics without expensive imaging upgrades.