AI Can Answer Your Database Questions With Surprisingly Little Detail

Companies increasingly want AI that can answer plain-English questions by digging through their databases — "which product sold best in March?" — instead of waiting on a data analyst. The problem is that big companies have enormous databases: thousands of columns spread across many systems, far more than an AI can read at once. So engineers must choose: show the AI more tables, or show it fewer tables in richer detail? And every bit of data you feed it costs money.

This study built a test with 80 databases and asked 1,279 questions, then measured how often the AI got the right answer. Two search methods were compared. With the simpler keyword-based search, giving the AI much more of the database map improved accuracy by 18 percentage points. But with the smarter, meaning-based search, that improvement shrank to just 3 points — because the smart search already finds the right tables even when shown very little. How the tables were formatted into text barely mattered at all, changing results by at most a couple of points.

The most interesting — and slightly worrying — finding came from a test where the researchers deliberately removed the tables the AI needed. Even then, 94.6% of correct answers named one of those missing tables exactly. That suggests the AI was filling in gaps from what it learned during training, not from the company's actual data. Handy when it's right, dangerous when a column was renamed or deleted last year.

The takeaway for businesses: spending more on feeding AI your full database map is often wasted money. Investing in better search is what pays off. And anyone relying on AI answers should verify them against the real database, because the AI may be confidently quoting a table that no longer exists. The researchers released their testing tool publicly so others can check their own setups.