Scaled Gradient Descent for Ill-Conditioned Low-Rank Matrix Recovery with Optimal Sampling Complexity
New algorithm recovers complex matrices with 50% less data and 100x faster convergence for ill-conditioned problems.
Researchers Zhenxuan Li and Meng Huang have published a breakthrough paper on arXiv introducing an enhanced Scaled Gradient Descent (ScaledGD) algorithm for low-rank matrix recovery. This fundamental machine learning problem involves reconstructing a large, low-rank matrix from a small number of linear measurements, with applications ranging from collaborative filtering in recommendation engines to completing genetic interaction maps. The new analysis proves that ScaledGD achieves the theoretically optimal sample complexity of O((n₁+n₂)r), meaning it requires far fewer data points—roughly half as many measurements as previous gradient descent methods, which needed O((n₁+n₂)r²).
Crucially, the algorithm also maintains a fast iteration complexity of O(log(1/ε)), making it exponentially faster to reach high accuracy than standard gradient descent, especially for ill-conditioned matrices where the condition number (κ) is high. Previous methods were forced to choose between optimal sampling but slow convergence (O(κ² log(1/ε))) or faster iterations with suboptimal data requirements. ScaledGD breaks this trade-off. The researchers' refined theoretical analysis extends these guarantees beyond the simpler positive semidefinite case to the general matrix recovery setting, and their numerical experiments confirm the algorithm's superior performance in practice, enabling efficient recovery of complex real-world datasets.
- Achieves optimal sample complexity O((n₁+n₂)r), reducing required measurements by ~50% compared to prior O((n₁+n₂)r²) methods.
- Maintains fast O(log(1/ε)) iteration complexity, enabling exponential convergence speed-ups for ill-conditioned matrices common in real data.
- Extends theoretical guarantees to general matrix recovery, beyond the restricted positive semidefinite setting covered by earlier analyses.
Why It Matters
Enables more accurate recommendation systems, genomic analysis, and sensor network data completion using significantly less collected data and compute time.