From Euler to Dormand-Prince: ODE Solvers for Flow Matching Generative Models
New benchmark shows high-order solvers cut function evaluations by 60% for flow matching models.
Hao Xiao's new paper systematically compares four classical ODE solvers—Euler, Explicit Midpoint, Classical Runge-Kutta (RK4), and Dormand-Prince 5(4)—for sampling from flow matching generative models. Implemented from scratch in PyTorch and tested on tasks from 2D toy distributions to MNIST digits, the work constructs Pareto frontiers using sliced Wasserstein distance. The headline finding: RK4 at just 80 function evaluations (NFE) achieves sample quality comparable to Euler at 200 NFE, offering a 60% reduction in neural network forward passes.
Beyond raw efficiency, the study reveals two key empirical observations. First, the Jacobian eigenvalue spectrum of the learned velocity field stiffens sharply near t=1, causing the adaptive Dormand-Prince solver to automatically concentrate its step budget at the end of the trajectory—a useful insight for designing custom samplers. Second, the quality gap between low-order and high-order solvers widens for undertrained or smaller models, indicating that solver choice becomes most critical when the model is imperfect. All code is publicly available, making this a practical resource for practitioners deploying flow matching at scale.
- RK4 at 80 function evaluations matches Euler at 200 in sample quality on MNIST flow matching tasks.
- Jacobian eigenvalue spectrum stiffens sharply near t=1, driving adaptive Dormand-Prince to allocate most steps at trajectory end.
- High-order solvers show larger quality gains over low-order ones when models are undertrained or smaller.
Why It Matters
Practical guidance for cutting inference cost in flow matching models without sacrificing quality, especially for imperfect models.