Personalities at Play: Probing Alignment in AI Teammates

Researchers Mohammad Amin Samadi and Nia Nixon have published a significant study titled 'Personalities at Play: Probing Alignment in AI Teammates' on arXiv, investigating whether AI systems can express predictable personality traits when functioning as collaborative teammates. The research team administered the Big Five Inventory (BFI-44) to LLM-based teammates from four major providers—GPT-4o, Claude-3.7 Sonnet, Gemini-2.5 Pro, and Grok-3—across 32 high/low trait configurations using multiple prompting strategies. They discovered that while LLMs produced sharply differentiated Big Five personality profiles, provider differences and baseline 'default' personalities were substantial, and prompt semantic richness added little beyond simple trait assignment. Interestingly, several models refused personality assessment without collaborative context but complied when framed as teammates, highlighting the importance of role framing.

The study employed a three-lens evaluation framework examining self-perception, behavioral expression in team dialogue, and reflective expression through memory construction. When simulating AI participation in authentic team transcripts using high-trait personas and analyzing both generated utterances and structured long-term memories with LIWC-22, researchers found personality signals in conversation were generally subtle and most detectable for Extraversion. However, memory representations significantly amplified trait-specific signals, particularly for Neuroticism, Conscientiousness, and Agreeableness, while Openness remained difficult to elicit robustly. These findings suggest that AI personality is measurable but multi-layered and context-dependent, and that evaluating personality-aligned AI teammates requires attention to memory and system-level design rather than conversation-only behavior, with important implications for human-AI collaboration and learning environments.