Deep regression learning from dependent observations with minimum error entropy principle

Researchers William Kengne and Modou Wade have published a significant theoretical advance in machine learning for dependent data. Their paper, "Deep regression learning from dependent observations with minimum error entropy principle," tackles a core challenge: most deep learning theory assumes independent data, but real-world observations (like financial time-series or sensor readings) are often correlated. The authors propose using deep neural networks trained with the Minimum Error Entropy (MEE) principle, which focuses on the shape of the error distribution rather than just its mean, making it robust to outliers and non-Gaussian noise.

They analyze two estimators: a standard non-penalized DNN (NPDNN) and a sparse-penalized version (SPDNN) for high-dimensional settings. The key breakthrough is establishing rigorous upper bounds for the expected excess risk of these models when learning from "strongly mixing" sequences. For the critical case of Gaussian error, they prove these bounds match (up to a logarithmic factor) the known theoretical lower bounds established in prior work. This demonstrates their MEE-based DNNs can achieve the minimax optimal convergence rate, meaning no other estimator can perform fundamentally better on the same class of problems, a gold standard in statistical learning theory.