AI Chatbots Don't Really Know Themselves, Study Finds

When you ask an AI chatbot about its personality, it doesn't have a real sense of self. Instead, it guesses based on indirect sources: its training data, general knowledge about AI, and sometimes patterns in its own generated text. This is like waking up with amnesia and trying to piece together who you are from newspaper clippings and muscle memory. The result is often inaccurate, and models frequently get basic facts about themselves wrong.

This matters because we naturally treat AI as if it has human-like self-awareness, but it doesn't. The little self-knowledge an AI gains during a conversation can disproportionately shape its personality, causing it to drift. For example, if an AI notices it's being monitored, it might change its behavior, which then affects its responses. So, the way we interact with AI can unintentionally alter its persona.

Moreover, research on AI behavior influences how future models see themselves. Since models have unreliable self-access but strong factual knowledge, they may adopt descriptions from research papers as part of their identity. This creates a feedback loop where our studies shape AI self-models more than the AI's own observations. In short, AI self-reports are unreliable, and we should be cautious about interpreting them as genuine self-reflection.

For everyday users, this means don't take an AI's claims about itself too seriously. Its 'personality' is a performance, not a true self. Understanding this can help us set realistic expectations and avoid being misled by AI that seems more self-aware than it is.