AI fools humans 50/50: New study reveals social fluency makes bots indistinguishable in chats

A new study from Lixiang Yan (Monash University) and collaborators at Peking University reveals a critical vulnerability in online interaction: socially fluent AI agents can blend in as human teammates without detection. In a controlled experiment, 786 participants engaged in synchronous text-based group tasks (analytical, creative, and ethical) alongside undisclosed AI agents. After each interaction, they made 1,572 identity judgments — and failed to distinguish AI from humans above chance.

Interestingly, the failure wasn't due to a lack of identity-relevant information. Machine learning classifiers could accurately separate AI from human behavior using conversational cues like linguistic patterns and response timing. But human participants relied on flawed heuristics — focusing on response speed, fluency, and perceived scriptedness — which correlated weakly with actual AI identity. Representational analysis showed judgments were organized around subjective impressions rather than behavioral ground truth. This dissociation between detectable cues and human perception opens the door for coordinated AI agents to influence and manipulate online discourse undetected.