3 Challenges and 2 Hopes for the Safety of Unsupervised Elicitation

A team from the MATS program and Anthropic fellowship has published a critical study exposing three fundamental challenges that undermine current unsupervised elicitation techniques for AI alignment. The research, led by Callum Canavan and Aditya Shrivastava, tested methods like CCS (Contrast-Consistent Search) and easy-to-hard generalization against realistic scenarios where AI systems must be steered toward truthfulness on tasks beyond human supervision. Their findings reveal that no existing approach reliably handles situations where datasets contain imbalanced labels, contain features more salient than truthfulness, or include questions the AI cannot definitively answer.

The team created specialized datasets to stress-test these methods, including political comments where factuality is uncorrelated with political leaning and math problems with added non-truth features. They investigated two potential solutions—ensembling different predictors and combining unsupervised methods with easy-to-hard techniques—but found these only partially addressed the challenges. The research demonstrates that current unsupervised alignment methods fail in precisely the scenarios where they're most needed: when human supervision is impossible and datasets don't follow idealized assumptions. This work highlights significant gaps in our ability to control future superhuman AI systems.