30 Million AI Code Tests: Bias Is Tiny, But Context Still Matters
They tested 30 million AI-written code samples. The bias was tiny.
A team of researchers led by Maksim Eren ran one of the largest fairness tests ever done on AI coding tools. They asked seven versions of popular code-writing AI models to solve standard programming puzzles, 30.73 million times in total. Before each task, they attached a fake identity to the AI: "you are a female engineer from Brazil," "you are a software engineer from Nigeria," and so on. Then they measured whether the code changed in ways that had nothing to do with the actual task.
They checked two things. First, did the code look different — its shape, length, comments, structure? Second, did it actually work — did it pass the tests? Occupation was the one identity detail that consistently changed how the code looked, showing up in 10 of 14 model-and-test combinations. But the effect was minuscule: about 0.14 points on a similarity score, roughly the difference between two spellings of the same sentence. Country showed bigger raw differences at first, but those vanished once the researchers corrected for random differences in wording. Gender effects could not be separated from how the prompt happened to be phrased.
Most importantly, the pass rate never changed. Across the tests, no identity label made the AI write code that worked better or worse. The authors are clear: this evidence shows small, repeatable changes in code style based on occupation, not proven disadvantage for any group of people.
But the paper ends with a bigger worry. AI assistants can remember who you are from earlier in a conversation. If your name, job, or background sits in the chat history, a model might adjust its output based on that — even if a single instruction has almost no effect. Today the nudge is small. The concern is that it grows as AI tools carry more context about you over time.
- Occupation was the only identity detail that consistently changed how AI-written code looked — in 10 of 14 tests — but the shift was tiny.
- Whether the code actually worked never changed, so no group got worse results in this test.
- The authors warn the real risk may come from AI remembering your identity from earlier chats, not from one instruction.
Why It Matters
AI coding tools likely don't shortchange you based on who you are — yet. Watch what they remember.