Factored Levenberg-Marquardt for Diffeomorphic Image Registration: An efficient optimizer for FireANTs

Researchers from the University of Pennsylvania and University of Texas at Austin have published a new optimization method for the FireANTs medical image registration framework. The paper, "Factored Levenberg-Marquardt for Diffeomorphic Image Registration," addresses a critical bottleneck: FireANTs' default Adam optimizer requires storing momentum and squared-momentum estimates that consume significant GPU memory, limiting its use on large 3D medical images like high-resolution CT scans. The team's modified Levenberg-Marquardt optimizer replaces these multiple state variables with just a single scalar damping parameter, adaptively tuned using a trust region approach.

This architectural change delivers substantial practical benefits. The new optimizer reduces memory consumption by up to 24.6% when processing large volumetric datasets, while maintaining registration performance across all four tested medical imaging benchmarks. Remarkably, a single hyperparameter configuration tuned on brain MRI data transferred without modification to lung CT and cross-modal abdominal registration tasks, matching or outperforming Adam on three of four benchmarks. The researchers also validated the effectiveness of incorporating a Metropolis-Hastings style rejection step to prevent updates that worsen the loss function during optimization.

The work represents a significant engineering advancement for computational anatomy pipelines. By dramatically reducing memory requirements while preserving accuracy, this optimizer enables researchers and clinicians to process larger medical images or run more complex registration tasks on existing GPU hardware. The plug-and-play nature of the solution means it can be directly integrated into existing FireANTs workflows, potentially accelerating research in neuroimaging, oncology, and other fields requiring precise anatomical alignment across imaging modalities.