New study pinpoints where medical AI forgets in continual learning

A new computer vision paper from MBZUAI takes a depth-wise look at why medical image segmentation models suffer from catastrophic forgetting in continual learning settings. The authors, Amal Saqib, Tausifa Jan Saleem, Numan Saeed, and Mohammad Yaqub, focus on gynecological imaging—a domain with high heterogeneity across modalities, anatomical structures, and annotation protocols. They systematically analyze an encoder-decoder network using block-wise ablations and controlled adaptation experiments to identify which network regions are most responsible for performance degradation when new datasets arrive sequentially.

The results are striking: ablating early encoder and late decoder blocks causes the largest performance drops, indicating that these regions are critical for preserving previously learned knowledge. However, when they restrict weight updates to bottleneck-adjacent regions, forgetting remains limited—even as new tasks are introduced. In contrast, allowing even small subsets of parameters in the shallower encoders and decoders to be trainable leads to a sharp increase in forgetting. This suggests that the location of updates across network depth matters more than the number of parameters changed. The findings offer a clear, actionable guide for continual learning in medical imaging: freeze or minimally adapt early encoders and late decoders, and focus fine-tuning near the bottleneck. Full code and analysis pipelines are promised upon acceptance, so practitioners will soon be able to apply this depth-aware strategy to their own models.