ICML paper argues ground truth datasets are human constructs, not objective facts

A new position paper accepted at ICML 2026 challenges a foundational assumption in machine learning: that ground truth datasets are objective, neutral references. Authors Charlotte Högberg, Ericka Johnson, and Kiri L. Wagstaff argue that every ground truth is a human construction, produced by specific arrangements of people, tools, and institutional contexts. They contend that these datasets carry invisible choices—what to measure, how to label, whose perspective to prioritize—that shape the models trained on them. The paper introduces the concept of 'situated reliability,' which calls for explicitly articulating the limits and strengths of both datasets and the models they produce, rather than treating them as universal truths.

The ML community, the authors say, will benefit from openly debating these often-unreported decisions. Doing so can improve model transparency, accountability, and cross-disciplinary collaboration. The paper does not argue that ground truths are useless, but that their contingent, context-dependent nature must be acknowledged to guide appropriate use. For practitioners, this means asking not just 'how accurate is my model?' but 'where, when, and for whom is this ground truth valid?' The work adds to a growing conversation about bias, reproducibility, and the social dimensions of AI, offering a practical framework for building more robust and honest systems.