Stacking Instructions in One Prompt Makes AI Coders Worse
How you phrase an AI request changes the quality of its code — dramatically.
If you've ever asked an AI to do something and got a bad answer, researchers just found a big reason why. A new study from computer scientists tested exactly how combining different instructions in one prompt affects AI code generation. They took five OpenAI models, gave them 164 coding problems, and tried all 27 ways of mixing three instruction types: output format (JSON, XML, or plain), persona (treating the AI as an expert or not), and urgency (urgent, moderate, or none). That's 22,140 attempts in total.
The results are surprising and a little worrying. Some combinations made specific AI models fail far more often than you'd expect from testing each instruction alone. The worst case: on a model called GPT-4o-mini, asking for JSON output plus an expert persona plus moderate urgency dropped correct answers by 12.2 percentage points more than the sum of individual effects. That's not just noise — it's a super-additive failure, meaning the instructions amplified each other's damage. JSON caused more trouble than XML. But the effect was not universal: GPT-4.1 models were mostly resistant, while another model, o3-mini, actually improved when given structured output requirements.
Why should you care? More and more people — not just professional programmers — rely on AI assistants for coding, spreadsheets, and data tasks. If the way you phrase a request matters this much, then cramming too many demands into a single prompt can silently ruin your result. The study also warns companies building software with AI: testing one instruction at a time is not enough. A pair or trio of harmless-looking requirements can team up and create bugs that pass unnoticed. Vulnerabilities are built into a model's architecture, not its size, so you can't assume a bigger, smarter model is automatically safer.
The honest caveat: this experiment focused on code generation and one family of OpenAI models. Still, the lesson probably generalizes to everyday uses of AI. When you ask for too much in one go — "act as a lawyer, output a table, and be quick about it" — the quality of what you get back may quietly drop. And now researchers know to look for those nasty combinations.
- Combining prompts like "expert persona + JSON + urgent" caused AI coding failures up to 12% more than expected.
- The effect was model-specific, not size-specific: GPT-4o models were vulnerable, GPT-4.1 was resistant, and o3-mini sometimes improved.
- Testing single instructions isn't enough — companies and users need to test real-world prompt combinations to avoid silent quality loss.
Why It Matters
The way we phrase AI requests affects output quality — stacking too many demands can silently produce worse results.