Generalized Stein's Lemma enables label-efficient sufficient dimension reduction

This paper by Ye Tian introduces a new approach to sufficient dimension reduction (SDR), a statistical technique for finding the minimal subspace of predictors that captures the full conditional distribution of a response variable. SDR is valuable for high-dimensional data, but existing methods struggle with multivariate responses and limited sample sizes. Inverse regression methods rely on strong distributional assumptions and matrix inversions, while multi-response extensions suffer from slice sparsity. Forward regression methods require computationally intensive iterative smoothing that scales poorly with response dimension, and deep learning approaches demand large labeled datasets.

Tian's framework, based on generalized Stein's lemma, constructs a cross-moment matrix between the multivariate response and the marginal score function of the predictors. The central subspace is recovered via singular value decomposition of this matrix. The method eliminates the need for the linearity condition, avoids matrix inversion and iterative smoothing, and can optionally leverage unlabeled data in semi-supervised settings. The paper provides convergence guarantees under standard regularity conditions and proposes a practical rank-selection algorithm to estimate the subspace dimension. Extensive simulations and a real data application show consistent improvements over existing methods, especially when labels are scarce, noise is high, and the predictor dimension is moderate.