TriVAL: Tri-Validation Framework Boosts LLM Optimization Modeling Accuracy

Optimization modeling is the critical link between natural-language problem descriptions and solver-ready code, but LLM-based pipelines often propagate errors from early stages. To address this, Ziyang Fang, JinXi Wang, Jinghui Zhong, and Yew-Soon Ong developed TriVAL, a tri-validation framework that performs explicit validation at three key stages: semantic specification (is the problem correctly understood?), mathematical formulation (are the equations and constraints accurate?), and code generation (does the code match the formulation?). At each stage, TriVAL executes a construct-validate-revise loop, checking outputs against stage-specific criteria and iteratively refining them until they pass validation. This prevents early mistakes from contaminating later stages, preserving faithfulness throughout the modeling process.

The team also created NL4COP, a benchmark of 150 instances across 50 diverse combinatorial optimization problem types, featuring more complex decision logic, tightly coupled constraints, and higher modeling demands than existing benchmarks like NL4Opt or E-OPT. Experiments on NL4COP and established benchmarks show TriVAL consistently outperforms state-of-the-art methods such as OptiGuide and C-Opt, with the largest improvements on the hardest problems. The paper (13 pages, arXiv:2605.23966) is available under cs.CL and related categories. This work could significantly accelerate the adoption of automated OR modeling by reducing the need for manual debugging of LLM-generated optimization code.