VIDS: A Verified Imaging Dataset Standard for Medical AI
New open standard reveals even top datasets like BraTS and CheXpert meet only 20-39% of quality and provenance requirements.
Researchers Joan S. Muthu and John Shalen have introduced VIDS (Verified Imaging Dataset Standard), a new open specification designed to address critical gaps in medical imaging AI development. While existing standards like DICOM handle image acquisition and BIDS organizes neuroimaging data, VIDS focuses on the curation layer—providing machine-enforceable validation for dataset structure, annotation provenance, quality documentation, and ML readiness. The framework defines 22 compliance dimensions and 21 validation rules across two profiles, using NIfTI as a working format while preserving full DICOM metadata for traceability.
In a revealing benchmark, the team assessed four major public datasets—LIDC-IDRI, BraTS, CheXpert, and the Medical Segmentation Decathlon—against VIDS dimensions. The results showed these widely used resources satisfy only 20-39% of requirements, with systematic deficiencies in provenance (who annotated what, when, and how) and quality documentation. To demonstrate the standard, the researchers released LIDC-Hybrid-100, a 100-subject CT reference dataset with consensus segmentation masks from four radiologists (mean pairwise Dice 0.7765) that achieves 21/21 validation on the Full compliance profile.
The VIDS ecosystem is fully open: the specification is CC BY 4.0, tools are Apache 2.0, a reference validator is available via PyPI (`pip install vids-validator`), and the LIDC-Hybrid-100 dataset is published on Zenodo. This creates a practical pathway for dataset creators to improve transparency and for AI developers to verify data quality before model training, potentially reducing failures in clinical validation.
- VIDS introduces 21 machine-enforceable validation rules and 22 compliance dimensions for medical imaging datasets, filling a critical gap in provenance and quality documentation.
- Benchmarking reveals major datasets like BraTS and CheXpert meet only 20-39% of VIDS requirements, highlighting widespread quality issues in current AI training data.
- The team released LIDC-Hybrid-100, a 100-subject CT reference dataset validating 21/21 rules, alongside open-source tools including a PyPI validator (`vids-validator`).
Why It Matters
Provides a standardized way to verify medical AI training data quality, potentially reducing model failures and improving trust in clinical applications.