RankLLM: Weighted Ranking of LLMs by Quantifying Question Difficulty

Researchers have introduced RankLLM, a novel framework that fundamentally changes how AI models are ranked. Instead of treating all test questions equally, it quantifies each question's difficulty and each model's competency through a bidirectional scoring system. The method evaluated 30 large language models on 35,550 questions, achieving 90% agreement with human judgments and outperforming traditional benchmarks. It offers a more nuanced, stable, and computationally efficient way to compare model capabilities at scale.