Researchers' EM algorithm learns rewards and worker reliability to filter crowdsourced spam

Pairwise comparisons fuel everything from recommendation systems to RLHF (reinforcement learning from human feedback) for large language models. But when those comparisons come from crowdworkers on platforms like Amazon Mechanical Turk or Scale AI, they often contain spam or biased judgments from annotators with limited domain knowledge. A new paper from Kaustubh Shivshankar Shejole, Tanish Agarwal, Arpit Agarwal, and Avishek Ghosh tackles this head-on by modeling both the underlying item rewards and each worker's reliability simultaneously.

The researchers adopt a Boltzmann-rational model that extends the classic Bradley-Terry-Luce framework by introducing a worker competency parameter. To make learning tractable, they derive an EM algorithm that leverages Polya-Gamma latent variables, transforming the logistic likelihood into a conditionally Gaussian form. This clever trick simplifies the E-step and reduces the whole formulation to a matrix sensing problem, for which the authors provide solid theoretical convergence guarantees. Experiments on real-world and synthetic datasets demonstrate that the method significantly outperforms baselines, especially in the presence of spammers and adversarial workers — making it highly practical for realistic crowdsourcing and reward learning setups. The code and data are publicly available, and the paper is accepted at UAI 2026 in Amsterdam.