New dataset framework cuts MRI cohort bias from 34% to 2.1%

A new paper from researchers at Oslo University Hospital (Qinghui Liu, Jon André Ottesen, Atle Bjørnerud, and Kyrre Eeg Emblem) tackles a hidden culprit in medical AI: sloppy dataset partitioning. Their Tripartite Dataset Analytics Framework is an auditable pipeline that characterizes spatial grid integrity, multi-parametric intensity fingerprints, and longitudinal temporal trajectories before splitting cohorts into training, validation, and test sets. By quantifying heavy-tailed feature dispersion and episodic sampling intervals common in clinical data, the framework exposes how naive random shuffling introduces covariate shifts and temporal imbalances, ultimately causing out-of-distribution evaluations.

The core contribution is an unsupervised standard operating procedure that combines elbow-optimized K-means clustering over a standardized six-dimensional joint intensity-temporal feature space with intra-cluster proportionate stratified sampling. Tested on 149 contrast-enhanced T1-weighted brain MRI scans, this protocol reduces maximum cross-subset intensity bias from 34.1% (random shuffling) to under 2.1%, while tightly aligning longitudinal follow-up intervals around the population mean. Monte Carlo stress testing across ten random seeds and three split configurations confirms the alignment remains stable, unlike the high variability of random partitioning. This reproducible, generalizable approach to cohort engineering promises more trustworthy deep learning for variable-length longitudinal clinical imaging workflows.