AI speech audits fail people with aphasia—FAccT paper reveals three pitfalls

A new study published at the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT '26) takes aim at how automatic speech recognition (ASR) systems are audited for equity. The researchers—Katelyn Xiaoying Mei, Anna Seo Gyeong Choi, Hilke Schellmann, Mona Sloane, and Allison Koenecke—examined six popular ASR systems and found that standard auditing practices systematically overlook performance disparities for people with aphasia, a speech disorder affecting millions. They identify three specific pitfalls: (1) using a single method of text standardisation, which can mask variance in ASR performance and ignore preferences of marginalised communities; (2) reporting only high-level demographic findings without drilling into intersectional subgroups or controlling for acoustic properties; and (3) relying exclusively on word error rate (WER), which fails to capture common generative AI errors like hallucinations.

The case study compared ASR performance for speakers with aphasia against a control group, revealing consistently worse transcription quality across all six systems. The authors argue that these inadequacies can lead to deceptive conclusions about ASR fairness and risk harming the very populations who most depend on speech-to-text technology. They propose a holistic auditing framework that incorporates community input, multiple standardisation methods, intersectional analysis, and metrics beyond WER. The paper calls on practitioners to adopt these robust, transparent practices as ASR capabilities rapidly evolve and become embedded in healthcare, education, and daily communication tools.