Kernel ridge regression insights expanded to product kernels by Zhou et al.
New theory reveals periodic plateaus and multiple descent in kernel regression.
Previous studies of kernel ridge regression (KRR) revealed saturation effects and multiple descent behavior but were restricted to inner product kernels on a sphere or strong eigenfunction assumptions like hypercontractivity. Yang Zhou, Yicheng Li, Yuqian Cheng, and Qian Lin now establish a broad new family of large-dimensional product kernels and derive corresponding convergence rates of the generalization error. This removes previous constraints, allowing the phenomena to be studied in more realistic settings.
Their theoretical results recover three key phenomena: minimax optimality when the source condition s≤1, the saturation effect when s>1, and—critically—a periodic plateau in convergence rates and multiple-descent behavior with respect to sample size n. These findings confirm that these behaviors are not artifacts of restrictive assumptions but intrinsic to a wide class of kernels. The work provides a more solid foundation for understanding when adding more data can unexpectedly hurt performance and how model complexity interacts with sample size.
- Generalization error convergence rates derived for a new family of product kernels in large dimensional KRR.
- Saturation effect confirmed for source condition s>1, limiting achievable accuracy.
- Periodic plateau and multiple-descent behavior observed as sample size increases, mirroring earlier sphere results.
Why It Matters
Deepens theoretical understanding of kernel methods, guiding practical model selection and data scaling strategies in ML.