Bhand & Joshi reveal lambda knob collapses latent dynamics, but only in simple videos

The paper investigates a common but poorly understood training knob in video predictors and world models: lambda, the weight on multi-step latent agreement. Using an empirical expansion proxy L20 (measured at horizon 20) and prediction error E20, the authors systematically vary lambda across simple and complex domains.

On Moving-MNIST (a passive video dataset with moving digits), raising lambda from 0 to 0.8 dramatically contracts the latent dynamics: L20 drops from 4.96±2.01 to 1.01±0.06, and E20 halves. Four of six seed models achieve L20<1 at lambda=0.8. The authors confirm statistical significance via paired t-test (p=0.005) and Wilcoxon (p=0.031). This suggests soft consistency can nudge latent dynamics toward a near-contractive band—at least for simple, passive video.

The effect does not generalize. On action-conditioned environments (Pendulum-v1, CartPole-v1) and more complex video (KTH Actions), raising lambda does not produce population-level contraction even when prediction error improves. An associational mediation analysis on MMNIST yields r-hat=0.94 (95% CI [0.88, 1.00]), though lambda was not randomized. Defensive checks (architectural baselines, exogenous stress, WorldTest, MPC, scaling) support a narrow claim: soft consistency's geometry-shaping power is domain-limited. The authors propose a stochastic-forcing law L20 ~ 1.23 + 1.82 eta (bootstrap slope CI [1.73, 1.92], R^2=0.96) that unifies control domains via calibrated noise (eta_eff). No DreamerV3 or TD-MPC2 returns are reported; the study is purely diagnostic.