"What don't you understand?" Language games and black box algorithms

A new academic paper is challenging a foundational goal of modern AI safety and ethics: making complex models explainable. In "What don't you understand? Language games and black box algorithms," researcher Rémy Demichelis applies heavyweight philosophy to computer science, arguing that the field of Explainable AI (XAI) is chasing an impossible dream. The core claim is that we should stop seeking 'explainability'—a complete, unambiguous account of a model's reasoning—and settle for 'interpretability,' which is always partial and context-dependent.

Demichelis grounds this argument in two key philosophical concepts. First, he invokes Willard Van Orman Quine's thought experiment of 'radical translation,' where a linguist can never be sure if a native's word 'gavagai' means 'rabbit,' 'undetached rabbit parts,' or a momentary stage of rabbithood. Similarly, we can never be certain what internal concepts in a model like Llama 3 truly 'refer' to. Second, he uses Ludwig Wittgenstein's idea of 'language games,' where meaning comes from use within a form of life, not from fixed rules. An AI's 'reasoning' is a language game we interpret, not a rulebook we decode. The paper concludes that regulatory demands for algorithmic transparency are fundamentally limited; we must build trust through robust, practical interpretations, not mythical perfect explanations.