CAWI boosts randomized neural nets with copula-based weight initialization
New method improves prediction across 83 benchmarks without backpropagation.
Randomized neural networks (RdNNs) offer efficient training by freezing randomly initialized input-to-hidden weights, enabling a closed-form solution for the output layer. However, conventional random initialization ignores correlations, asymmetries, and tail dependence among features, degrading performance. To address this, Akhtar, Tanveer, and Arshad propose CAWI (Copula-Aligned Weight Initialization). CAWI first maps each feature to the unit interval using empirical CDFs, then fits a multivariate copula (elliptical: Gaussian, t; Archimedean: Clayton, Frank, Gumbel) to capture rank-based dependence. Each weight column is sampled from the fitted copula and transformed via a fixed inverse marginal, making the frozen projections dependence-aware without altering the freeze-once paradigm.
Evaluation across 83 diverse classification benchmarks (binary and multiclass) and two biomedical datasets—BreaKHis (breast cancer histopathology) and Schizophrenia (neuroimaging)—shows consistent and significant improvements over standard random initialization. The method works with both shallow and deep RdNN architectures. By preserving the closed-form solution and requiring no changes to the objective or solver, CAWI is a drop-in enhancement. Code is available. The paper, published on arXiv (2605.12580), was accepted at the 29th International Conference on Artificial Intelligence and Statistics (AISTATS) 2026.
- CAWI samples input-to-hidden weights from a copula fitted to empirical feature dependencies, addressing a long-standing limitation in randomized neural networks.
- Tested on 83 classification benchmarks and 2 biomedical datasets (BreaKHis, Schizophrenia), with consistent improvement over conventional random initialization.
- Supports five copula families (Gaussian, t, Clayton, Frank, Gumbel) and works with shallow and deep RdNN architectures without altering the closed-form solution.
Why It Matters
CAWI offers a simple, plug-and-play boost for backpropagation-free neural networks, improving accuracy on real-world data with correlated features.