ADNTN paper shrinks neural networks up to 77,000x without accuracy loss

A new paper on arXiv (cs.LG/2606.00130) from researchers Andrzej Cichocki and Michal Wietczak presents Automatically Differentiable Nonlinear Tensor Networks (ADNTNs), a method that compresses deep neural networks exponentially while maintaining or improving accuracy. The ADNTN framework generates large weight tensors from a compact set of core tensors trained end-to-end via reverse-mode automatic differentiation. The paper explores three architectures: Tree Tensor Networks (TTNs), augmented TTNs with boundary disentanglers (aTTNs), and Multi-scale Entanglement Renormalisation Ansatze (MERA).

Simulations on AlexNet and VGG-16 layers show per-layer compression ratios ranging from roughly 2,000x to 77,000x in studied settings. In several VGG-16 cases, accuracy even improved over the dense baseline, while in all cases it matched or exceeded it. The authors emphasize these results are encouraging but not final, noting that full practical deployment requires co-optimized contraction schedules and deployment kernels. The work represents a mathematically structured, hardware-aware route toward much smaller neural networks, with clear separation between differentiating contraction programs and avoiding large intermediates.