New Study Reveals Forward-Forward Learning Fails on Real Images, Not Synthetic Benchmarks

Yucheng Chen's new paper, 'Synthetic Benchmarks Overstate Forward-Forward Scaling: Real-Data Limits of Layer-Local Training,' systematically debunks the promise of Forward-Forward (FF) learning—a backpropagation-free training method proposed by Geoffrey Hinton in 2022. Chen develops DTG-FF (dynamic temperature goodness, decoupled normalization, multi-layer fusion) as a best-in-class FF instrument, achieving 91.8% on CIFAR-10 and the first FF baseline on ImageNet-100 at 224x224. However, under identical recipe and backbone, a standard backpropagation (BP) baseline with deep supervision beats DTG-FF by 2.40 pp on CIFAR-10 and 5.93 pp on CIFAR-100, with the gap widening as class count increases.

On ImageNet-100 at 224x224, DTG-FF reaches only 49.4% accuracy—the first FF result at that scale—while typical BP tops 75%. The paper reveals a critical confound: on synthetic teacher-student tasks, FF increasingly outperforms BP as class count K grows, but on real images the trend reverses. A within-dataset probe on CIFAR-100 (coarse vs. fine labels) isolates label hierarchy from image distribution, showing synthetic K-sweeps overstate FF transferability. Furthermore, a systems audit shows FF's supposed memory advantage evaporates: on 8 GB hardware, standard BP with gradient accumulation uses 4.18 GB and 157 img/s, while DTG-FF uses 7.90 GB and 138 img/s. The findings strongly caution against using synthetic benchmarks to claim FF's real-world viability.