Weak-to-Strong Generalization Proven: AI Models Retain Knowledge While Learning from Weaker Teachers

A new arXiv paper from researchers Ryoya Awano and Taiji Suzuki tackles one of AI alignment's most pressing open questions: Can a stronger model learn from a weaker one without losing its broader capabilities? The authors prove that weak-to-strong (W2S) generalization is not only possible but also mechanistically sound. They model the strong model as a two-layer neural network with pre-trained representations organized into low-dimensional subspaces. When fine-tuned on outputs from a weaker, task-specialized model, the strong model "elicits" latent knowledge—it learns the target feature direction from scratch while retaining off-target features. This is a significant advance over previous theoretical work, which either fixed representations or operated in simplified settings. The paper demonstrates that multi-step SGD succeeds in feature learning, something earlier analyses left as an open problem.

Crucially, the authors show that W2S preservation of general capabilities stands in stark contrast to standard supervised fine-tuning, which causes catastrophic forgetting when off-target feature directions correlate with the target. The strong model learns the target task (κ) efficiently while maintaining its pre-trained diversity. Numerical experiments on synthetic data confirm the theoretical results. The work has immediate implications for aligning superhuman AI systems: if a future superintelligent model can be fine-tuned on a weaker, human-aligned model's outputs without sacrificing its general knowledge, it could provide a scalable path to safe AI deployment. The paper is 48 pages with one figure, available on arXiv under reference 2605.12908.