Study finds users strongly prefer human-like AI agents over text bots
90-person study shows human-like agents win with large effect size over text-based
Get AI news that actually matters
One email a day. Zero fluff. Join 10,000+ professionals.
Danai Korre’s CHI 2026 workshop paper presents empirical data from a 90-participant within-subjects study comparing a highly human-like spoken embodied conversational agent (ECA) against a low-human-like text-only agent (no embodiment). The agents operated within a Unity-developed mobile serious game about pre-decimal UK currency, featuring two distinct roles: an Instructor (Alex) and a Shopkeeper/Collaborator. Users interacted via voice and mouse input. Quantitative data was collected using the CCIR MINERVA usability questionnaire and the Agent Persona Instrument, analyzed via paired t-tests, repeated measures ANOVA, and multiple linear regression.
The results show a statistically significant preference for the highly human-like agent version, with a large effect size. Qualitative data from observations and exit interviews further reinforce this, revealing how roles, mixed-initiative dialogue, and breakdown/repair patterns emerge in goal-oriented tasks. Korre emphasizes that the study does not propose new frameworks but rather reports empirical findings and raises questions about timing, user expectations, and role-specific interactions for the community to explore.
- 90 participants compared a highly human-like spoken ECA vs. text-only agent in a mobile Unity game
- Quantitative analysis (paired t-test, ANOVA, regression) showed significant preference for human-like agents with large effect size
- Qualitative insights highlighted role dynamics, mixed-initiative dialogue, and repair strategies in goal-oriented collaboration
Why It Matters
Voice-driven human-like AI agents could redefine user trust and collaboration in mobile serious games and beyond.