Researchers expose and fix LLM dialect bias against AAE

Researchers from seven institutions have exposed a systemic bias in large language models (LLMs) where African American English (AAE) inputs are routinely rewritten into Standard American English (SAE). The team—including Huan Wu, Ali Emami, and others—audited six instruction-tuned LLMs ranging from 14B to 70B parameters and found that these models consistently preferred SAE continuations over AAE ones, regardless of the input dialect. Their analysis introduced conditional Dialect Group Invariance (cDGI), a framework that isolates true model bias from artifacts, and identified syntactic constructions like negative concord (e.g., 'ain't nobody') as universal triggers across all tested models.

To mitigate this bias, the researchers pioneered the first application of activation steering—a training-free, test-time technique—to dialect bias correction. By extracting dialect directions via causal tracing and injecting them into bias-relevant layers, they reduced bias 5 to 20 times more effectively than prompting alone while preserving SAE fluency. Supporting this work, the team released REAL-AAE, the largest real-world AAE parallel corpus to date, comprising 17,479 AAE/SAE/AAE_back triplets from natural tweets. The dataset was validated automatically (BERTScore F1 = 0.95) and by three native AAE speakers (83.0% semantic agreement), setting a new benchmark for dialect-aware AI research.