Study: Telling AI It's a Scientist Makes It Slower and Costlier

Researchers often start an AI chat by handing it a long resume-style instruction: 'You are a senior bioinformatician with 20 years of experience...' The idea is that telling the AI who it is makes it smarter. A new study by Timothy Kassis tested that assumption directly. He took 503 profession profiles — detailed job descriptions for scientists — and ran them against nine science question sets totaling 4,531 questions, using Google's Gemini 3.8 Flash model through the OpenRouter service. He compared the long profiles to four simpler alternatives, including a bare-bones 'You are a helpful assistant.'

The result is a clear thumbs-down. On average, the detailed job profiles scored 0.6 percentage points lower than the minimal baseline, and no subject showed a real improvement. Meanwhile, the AI produced 1.5 to 2.3 times as many words, and each successful answer cost 2.2 to 4.5 times more. Think of it like hiring a consultant, handing them a 40-page job description, and getting a slower, wordier answer that isn't any more accurate — then paying triple for it.

The gap got worse on harder work. On 60 bioinformatics problems that required using software tools, the AI solved 46.7% with the long profile versus 56.7% with the simple one — a 10-point drop. The reason is practical: long instructions eat up the AI's limited working budget, so it ran out of room or time more often before finishing.

There was one surprise benefit, and it wasn't about expertise. When the AI provider's servers glitched mid-answer, the longer prompts coped better — 71.6% of questions had a correct first answer versus 54.0% with the short prompt. But generic and even off-topic long prompts performed just as well, meaning the benefit came from length or formatting, not scientific knowledge. The takeaway: skip the fancy titles unless you've measured a real gain.