New physics-aligned augmentations sharpen scientific imaging AI
Natural-image augmentations distort microscopy data; this arXiv method fixes it.
Self-supervised learning (SSL) relies on data augmentations to teach models invariances, but standard pipelines tuned for natural images often violate the physical constraints of scientific imaging—like electron microscopy's measurement symmetries and acquisition noise. Bashir Kazimi and Stefan Sandfeld tackle this in their new arXiv paper by formalizing a "physics-aligned augmentation set" that combines measurement-consistent symmetries with acquisition-driven perturbations. They propose a label-free workflow: enumerate candidates, annotate via the measurement operator, validate with representation-geometry diagnostics, and confirm with single-factor ablations.
Applying this to real-space electron microscopy and reciprocal-space 4D-STEM diffraction, they tested five SSL paradigms—DINOv2, SimCLR, MAE, VICRegL, and I-JEPA—on classification and crystal-orientation regression. Physics-aligned augmentations significantly boosted downstream performance for cross-view consistency objectives, lowered geodesic error, and strengthened robustness against realistic acquisition variability like detector gain and resolution loss. The authors emphasize the procedure is modality-agnostic, extending naturally to medical and remote-sensing imaging, and positions augmentation design as a controllable source of inductive bias in scientific SSL.
- Proposes a reproducible, label-free workflow for selecting physics-aligned augmentations in scientific SSL
- Evaluated on real-space electron microscopy and 4D-STEM across five SSL paradigms (DINOv2, SimCLR, MAE, VICRegL, I-JEPA)
- Improves classification and crystal-orientation regression while reducing geodesic error and increasing robustness to detector gain and resolution loss
Why It Matters
Gives researchers a principled way to design augmentations for microscopy, medical imaging, and remote sensing—boosting SSL performance without labels.