AI Test Scores May Be Misleading, Hugging Face Expert Warns

When companies say their AI is the best, they often point to test scores. But according to Irene Solaiman, policy chief at Hugging Face, those tests are not yet reliable. She spoke at TechCrunch Disrupt last week, explaining that the methods behind AI evaluations are still evolving and not standardized. This means the scores you see might not accurately reflect real-world performance.

Why should you care? If you're choosing an AI tool for your business or personal use, you might rely on these test results to make a decision. But if the tests are inconsistent, you could end up with a product that doesn't live up to its promises. Solaiman also raised concerns about "embedded evaluators"—third-party experts placed inside AI companies to monitor safety. While this sounds good, there are doubts about how neutral or effective these evaluators can be, especially after Anthropic's choice of Accenture as its first evaluator was mocked by critics.

Another issue is that evaluations are becoming more complex. It's not just about the AI model itself but also the tools and systems around it. A weaker model with a better user interface could outperform a stronger one, making comparisons even harder. Solaiman noted that evaluating AI agents (AI that can take actions) is particularly difficult and expensive.

In short, the AI testing landscape is messy. Without a uniform approach, it's hard to know which AI is truly best for your needs. This could lead to wasted money on subpar products or missed opportunities to use the best tools. As AI becomes more integrated into our lives, having reliable evaluations is crucial for making informed choices.