New AI Test Checks If Chatbots Understand Nigerian Pidgin

Nigerian Pidgin is one of Africa's most widely spoken languages, yet AI systems barely understand it. Most language tests only measure translation or simple sentiment, like whether a sentence is positive or negative. A new tool called Wazobia Eval, created by researcher Stephanie Okoye, tries to fix that by testing AI on the things that really matter: picking up emotions, spotting sarcasm, and understanding cultural references that don't translate neatly.

Why should you care? If a chatbot or voice assistant can't tell when someone is being sarcastic, it will give wrong, frustrating, or even offensive answers. For millions of Pidgin speakers, this means AI tools often fail them in everyday situations. Wazobia Eval is a benchmark — a standardized test — that shows developers exactly where their AI falls short. It comes with over 550 carefully annotated examples and 16 emotion categories designed to capture how Pidgin speakers actually express feelings.

The catch: this is an early pilot, and 550 examples is a small starting point. It's not a complete solution. But making the dataset publicly available means researchers and developers around the world can use it to improve their AI. It also sets a pattern for how to build similar tests for other underrepresented languages.

In short, Wazobia Eval is a step toward AI that doesn't just translate words, but truly understands the people using them. That matters for customer service, healthcare bots, translation apps, and any technology meant to serve everyone — not just English speakers.