MPMMine benchmark suite tackles constraint acquisition's reproducibility crisis

A new paper from Rafał Stachowiak and Tomasz P. Pawlak addresses a critical gap in constraint acquisition (CA) research: the lack of proper benchmarks. Existing datasets were designed for evaluating solvers — not CA algorithms that learn or validate mathematical programming (MP) models from domain knowledge artifacts like natural language or example solutions. This mismatch creates reproducibility issues and makes cross-study comparisons nearly impossible.

Enter MPMMine, a benchmark suite built for consistency, standardization, completeness, and extensibility. It uses open formats (MiniZinc, CommonMark, JSON) and offers multiple models per problem, tens of instances per model, and thousands of labeled solutions and non-solutions covering both integer and continuous domains. Natural-language descriptions are included to support emerging text-to-model methods. MPMMine's uniform structure and version control promise to accelerate CA method maturation and enable fair, reproducible evaluation across the AI and operations research communities.