Deflation-Free Optimal Scoring

A new arXiv paper from researchers Sharmin Afroz and Brendan Ames introduces Deflation-Free Sparse Optimal Scoring (DFSOS), a novel approach to linear discriminant analysis that eliminates error propagation found in traditional methods. Sparse Optimal Scoring (SOS) reformulates LDA to enable feature selection via elastic net regularization, crucial for high-dimensional datasets where the number of features far exceeds observations. However, most existing SOS methods rely on deflation-based strategies that compute discriminant vectors sequentially, a process that can accumulate errors and yield suboptimal solutions. DFSOS addresses this by estimating all discriminant vectors simultaneously under an explicit global orthogonality constraint.

The method combines Bregman iteration with orthogonality-constrained optimization, decomposing the complex problem into tractable subproblems for scoring vectors, discriminant vectors, and orthogonality enforcement. The authors establish convergence to stationary points of the augmented Lagrangian under mild conditions. Extensive experiments on synthetic data and real-world time series datasets demonstrate that DFSOS achieves classification accuracy comparable to or better than existing deflation-based methods. This work suggests that deflation-free approaches offer a robust and effective framework for sparse discriminant analysis, particularly in high-dimensional problems common in genomics, finance, and sensor data analysis.