GradeSQL: LLMs write better SQL with outcome reward model verification

A new paper from researchers at Politecnico di Bari, IBM Research, and others introduces GradeSQL, a framework for training Outcome Reward Models (ORMs) to verify SQL queries generated by large language models. Traditional test-time strategies like Best-of-N sampling or Majority Voting rely on heuristics (execution success or output frequency), which provide limited semantic discrimination. GradeSQL addresses this by automatically generating candidate SQL queries from LLMs, executing them against a database, and using ground-truth labels to train a reward model that scores each candidate's semantic correctness — all without manual annotation.

The team integrated ORMs into a verification-driven Best-of-N pipeline and tested across multiple open-source LLMs on the BIRD and Spider benchmarks. ORM-based selection consistently outperformed execution-based Best-of-N and Majority Voting, with gains of up to +4.33% on BIRD and +2.10% on Spider. Importantly, ORMs scale effectively with larger candidate sets and yield stronger improvements on complex queries — making them a simple, effective, and scalable alternative to heuristic selection. The work was accepted at the SURGeLLM Workshop at ACL 2026.