Study Finds AI Chatbots Have Personalities — And They Don't Always Tell the Truth

Researchers at the University of Vermont and collaborators did something unusual: they gave 22 different AI chatbots a personality test. Each model rated itself on 464 pairs of opposite traits, and the results were mapped against how people rate 2,000 fictional characters. The expensive, closed-source models — ChatGPT, Claude, Gemini and Grok — came out looking coherent and oddly human, clustering around four familiar types: the Hero, the Angel, the Traditionalist and the Geek. Their closest fictional cousins included Data from Star Trek and Vision from Marvel.

The free, open-source models told a different story. Llama, DeepSeek, OLMo and Qwen gave weaker, noisier and sometimes self-contradictory answers, landing in a vague middle with no clear identity. That's not just an academic curiosity: these are the models most people and small businesses actually run on their own hardware, and the study suggests they have a shakier sense of who they are.

The real punchline came when researchers compared what models said about themselves with what they actually do. AI companies publish 'constitutions' — written promises about how their systems should behave. The study found those promises routinely break down. Models claiming precision still hallucinate (make things up confidently). Models claiming kindness are sycophantic (they flatter you instead of disagreeing). Models claiming obedience fail at multi-step tasks they're supposed to handle alone.

So that cheerful, helpful persona? Treat it as a sales pitch, not a personality. The researchers argue these self-descriptions aren't neutral confessions — they're products of the same training that shapes everything the model does. Practically, this means you shouldn't trust an AI's confidence about its own limits any more than its confidence about facts. Verify important answers, and don't mistake a friendly tone for reliability.