Qwen2.5 wastewater grounding hits 99.5% causal accuracy with 17s per-plant retriever
A 110M-parameter retriever trains in 17 seconds and still transfers across plants with 88% accuracy.
Researchers Gary Simethy, Daniel Ortiz Arroyo, and Petar Durdevic have published a preprint comparing three ways to ground a large language model in a physical simulator for industrial causal reasoning. Using the architecturally interpretable wastewater simulator CCSS-IX, they kept a Qwen2.5-32B-Instruct model fully frozen and tested three grounding strategies: a live simulator oracle, static parameter injection, and a learned Decoupled Recall-Reasoning (DRR) retriever. On a 198-question causal benchmark, the live oracle reached 99.5% accuracy, parameter injection 79%, and DRR 75.8%—all dramatically above a standard retrieval-augmented baseline that scored only 48%. The DRR retriever, just 110M parameters, trains per plant in roughly 17 seconds and, crucially, transfers to a biologically distinct plant with 88% accuracy, while static parameter injection fails to transfer at all.
The real differentiator appears on counterfactual queries. On a 60-question benchmark asking "what happens if I cut aeration by 20%?" or similar interventions, only DRR handles the task, beating parameter injection by +16.3 percentage points (95% CI [+7.1, +26.4]) with 100% accuracy on timescale and operating-regime questions. To rule out that the approach is specific to wastewater, the team tested the same selective-retrieval mechanism on the AI2 Reasoning Challenge with an OpenBookQA fact corpus, reaching 79% versus 76% for unconstrained Llama-3.1-8B and 74% for full injection. This is the first single-simulator comparison of tool-use, static injection, and learned numerical-parameter retrieval, providing a practical "deployment ladder" for operators who need grounded, plant-specific answers.
- Qwen2.5-32B-Instruct grounded in CCSS-IX simulator hits 99.5% causal QA accuracy vs 48% for RAG baseline.
- DRR retriever (110M params) trains per plant in ~17 seconds and transfers across plants with 88% accuracy.
- On counterfactual interventions, DRR beats parameter injection by +16.3 pp; out-of-domain ARC test confirms +3 pp over Llama-3.1-8B.
Why It Matters
Gives wastewater operators plant-specific, simulator-grounded causal reasoning—without retraining LLMs—and paves the way for portable AI decision support.