GTF-DEER enables log-time training on sequences >10k steps

Reconstructing nonlinear dynamical systems from sequential data has been limited by the linear runtime complexity O(T) of classical backpropagation through time. Recent breakthroughs in parallel associative scans allow algorithms to achieve logarithmic time complexity O(log T). This paper by Hess, Götz, and Durstewitz studies two classes of parallel-in-time algorithms for dynamical systems reconstruction: modern State Space Models (SSMs) with linear non-autonomous dynamics, and general nonlinear models using the DEER framework.

They find that linear training-time recurrence in SSMs limits their ability to learn accurate nonlinear dynamics. To address this, they augment DEER with Generalized Teacher Forcing (GTF), creating GTF-DEER. This ensures stable and effective learning across arbitrary sequence lengths. Experiments show that training on sequences longer than 10,000 steps dramatically improves reconstruction quality when data exhibits long time scales, establishing GTF-DEER as a robust tool for data-driven discovery and highlighting the untapped potential of long-sequence learning.