Humanlike AI Writing Assistants Boost Efficiency but Feel Eerie, Study Finds

A new study from researchers at the University of British Columbia (Michael Yin, Angela Chiang, Samuel Rhys Cox, Robert Xiao) explores the social dynamics of human-AI collaborative writing. They built three variants of an AI text editor that varied along two dimensions: temporal (synchronous vs. asynchronous suggestions) and visual (presence vs. absence of a cursor). In a controlled experiment with 48 participants, each user completed a collaborative writing task with the AI agent. The goal was to measure how different levels of humanlike interaction affect both task performance and the user's sense of social connection with the AI.

The results reveal a clear trade-off. Synchronous AI suggestions—where text appears in real time as the user types—made the writing process significantly more efficient but often produced suggestions that were contextually misaligned, forcing users to pause and correct. Adding a visual cursor (a blinking caret showing where the AI's next edit would land) improved users' ability to understand the AI's intent, but it also triggered feelings of being watched or surveilled, described by some participants as “a bit eerie.” In contrast, asynchronous suggestions (appearing after a delay) and no cursor felt less intrusive but also less collaborative. The paper concludes that humanlike design in AI agents can create positive social expectations but also imposes social costs, especially when the AI lacks the alignment that human collaborators naturally develop. The authors offer design implications and ethical considerations for building future human-AI collaboration systems.