Understanding LLM Evaluator Behavior: A Structured Multi-Evaluator Framework for Merchant Risk Assessment

Researchers tested five leading AI models on their ability to evaluate merchant payment risk. The models showed significant bias, with some consistently scoring themselves too high and others too low. When the models' identities were hidden, this bias decreased by about 26%. The study found that models with a 'negative bias' were actually more aligned with human expert judgment, and four models showed strong correlation with real payment network data.