New Test Shows AI Struggles to Predict Database Query Results

is increasingly being used to run databases—generating queries and executing them on live data. But a new study shows that just because an AI can write a query doesn't mean it can predict what that query will do. Researchers built a test called MongoBench with 300 tricky database queries. They asked four leading AI models to predict exactly which records each query would return. The models scored between 70% and 92% accuracy, a big gap. Even when the AI wrote the query itself, it mispredicted the results 10-31% of the time. That's like a chef who can cook a meal but can't tell you what it will taste like.

The errors weren't random. They clustered around specific database concepts like range comparisons and null values—things that often trip up humans too. Surprisingly, giving the AI a cheat sheet of rules helped only unevenly. And when one model tried to delegate by actually running the query, it often failed to match the intended query, getting about 22 answers wrong. The other three models did better. This shows that AI still needs human oversight when handling critical data.

So what? If you're using AI to manage customer records, financial data, or inventory, a wrong prediction could mean lost sales, incorrect reports, or privacy breaches. The study's authors released the test publicly so others can build safer AI database tools. For now, treat AI-generated database queries like a new employee: check their work before trusting them with anything important.