LADDER paper rethinks domain generalization with style-based reweighting

Domain generalization (DG) has long been dominated by invariance-based methods, which assume that the causal relationship between features and labels remains stable across domains. But what if the domain itself modulates that mapping? A new paper from Yuhang Jiang, Fengchuan Zhang, Sanguo Zhang, and Guojun Zhu argues that chasing invariance can be too restrictive. Their method, LADDER, treats domain style not as noise but as a helpful signal. The pipeline first disentangles causal content from domain style using a latent adaptive approach. It then freezes the encoders, fits separate classifiers for each source domain, and at inference time, reweights those classifiers based on the unlabeled target domain's covariate distribution. No target labels or model updates are needed.

LADDER is validated on simulations, the satellite imagery benchmark FMoW, and a location-grouped version of iWildCam. The method consistently outperforms both standard ERM and state-of-the-art invariance methods like IRM in terms of overall accuracy and worst-group accuracy. The theoretical contribution includes guarantees on the reweighting procedure. This work challenges the assumption that style is always a confounder to be removed — instead, it can serve as a 'ladder' to pinpoint where the target domain lies among source domains, enabling adaptive prediction. For practitioners, this means better generalization to real-world shifts without expensive re-labeling.