Portfolio Optimization Proxies under Label Scarcity and Regime Shifts via Bayesian and Deterministic Students under Semi-Supervised Sandwich Training

Researcher Adhiraj Chattopadhyay has published a novel machine learning framework for portfolio optimization that addresses two critical challenges in quantitative finance: extreme data scarcity and market regime uncertainty. The paper introduces a 'semi-supervised sandwich training' method where a traditional Conditional Value at Risk (CVaR) optimizer acts as a 'teacher' to generate supervisory labels. This allows neural network 'students'—both Bayesian and deterministic models—to be trained on a tiny dataset of just 104 real observations, dramatically expanded using synthetically generated data from a sophisticated factor model with t-copula residuals.

The research demonstrates that this hybrid approach enables AI models to effectively learn the complex mapping for portfolio construction. In a rigorous three-part evaluation—controlled synthetic experiments, in-distribution real-market tests (C2A), and cross-universe generalization (D2A)—the student models were deployed using a rolling protocol. This involved periodically fine-tuning a frozen, pre-trained model on recent data before resetting it to ensure stability while allowing limited adaptation. The results are significant: the neural proxies not only matched but in several cases outperformed the CVaR teacher optimizer they were trained to emulate.

Crucially, the AI models exhibited superior robustness during market regime shifts and achieved reduced portfolio turnover compared to the traditional optimization method. This suggests the framework successfully captures the underlying principles of portfolio construction rather than merely memorizing data. The work provides a compelling blueprint for applying advanced ML techniques like teacher-student pipelines and synthetic data generation to domains where high-quality labeled data is prohibitively scarce or expensive to obtain.