AI Writes Better Code in English Than Other Languages, Study Finds
If you ask AI to code in your native language, it may quietly do worse.
Researchers put seven large language models — GPT-4o, o3-mini, DeepSeek-V3.2, Llama-3, two Qwen2.5-Coder models, and GitHub Copilot — to the test, presenting each programming problem on AtCoder, LeetCode, and BigCodeBench in English, Japanese, and Chinese. Same problem, different language, different results: the natural language of the problem statement measurably affects code-generation accuracy. The highest median accuracy showed up in the languages each dataset officially supports. Translation improved accuracy, but not consistently — its effectiveness varied by dataset and model type. The researchers also found AtCoder contained a particularly high proportion of narrative-style and longer problem statements. The lesson: language matters, translation can help in some settings, and the narrative style and context length of a prompt are tied to both language bias and how well translation works as a fix.
- Seven AI coding tools were given identical programming problems in English, Japanese, and Chinese — and the language changed how well the code worked
- Models scored highest in English, the language most of their training material came from, with the biggest gaps on long, story-style questions
- Translating your request into English can help, but only sometimes — the benefit varied by model and by type of problem
Why It Matters
Non-English speakers may get worse code from AI tools — try English prompts for better results.