Alex Mallen proposes judgment prediction to benchmark AI conceptual capabilities

Alex Mallen (LessWrong, July 2026) proposes a novel benchmarking methodology for AI conceptual capabilities: judgment prediction tasks. Instead of asking models to answer inherently subjective questions (e.g., predicting the probability of misaligned AI takeover), Mallen suggests explicitly instructing AIs to predict the judgment of a specified human expert under controlled affordances (time limits, tools, AI assistance). The goal is to track capability improvements without conflating the model's own priors or tastes with its reasoning skill. In the ideal case, experts would answer questions with various affordances, and models would be scored on how closely they approximate those judgments.

However, Mallen identifies several downsides. Human judgments are noisy and hard to measure because memory prevents resetting subjects. Unlike datasets like LMCA that use multiple judges and feedback loops to improve labeling quality, judgment prediction lacks inter-judge validation, making it harder to elicit high-quality responses. Additionally, models trained on data from after the judge’s opinions evolved could artificially inflate scores by learning the judge’s taste rather than true conceptual reasoning. Data leakage is a serious concern—if a model is trained on ‘Caspar finds MIRI’s work interesting,’ it may mimic judgments rather than reason. Despite these flaws, an improvement not explained by cutoff date differences could still point at genuine reasoning deficits, making judgment prediction useful for training and hill-climbing.