NIH researchers expose optimization clash in sparse-to-dense 3D MRI segmentation

In a new paper submitted to MELBA, Paul Hoareau and eight co-authors from the NIH, University of Montreal, and Polytechnique Montreal systematically dissect optimization strategies for weakly supervised 3D segmentation of high-resolution ex vivo MRI. Using 9.4T scans of multiple sclerosis spinal cords (over 104,000 slices with only 428 ground-truth 2D annotations), they trained a 2D Teacher network to generate dense pseudo-labels, which then guided a 3D Student. Their experiments reveal two fundamental conflicts.

First, the 2D model thrives on heavy spatial augmentation and soft-label regularization—improving White Matter Lesion Dice scores by more than 11 points. But those exact techniques backfire in the 3D Student, degrading its performance. Second, human-centric preprocessing like CLAHE (contrast-limited adaptive histogram equalization) disrupts the global statistical cues that 3D models rely on, causing Gray Matter Lesion Dice to fall by roughly 25 points. The study underscores that 2D and 3D architectures have fundamentally different optimization landscapes, and that practices designed for human visual interpretation can be harmful to machine vision models. Code and models are publicly available.