Microsoft study: Bing Copilot users' AI habits are surprisingly sticky
Longitudinal analysis of 12,000 users reveals individual behavior barely changes over time.
A new paper from Rebecca Hicke and Kiran Tomlinson tracked the conversational trajectories of roughly 12,000 randomly sampled Microsoft Bing Copilot users over time, alongside data from the WildChat-4.8M dataset. Despite clear population-level trends—like growing task complexity or shifting conversational styles—individual users showed remarkably little change. The authors describe user habits as 'overwhelmingly sticky,' suggesting that once a person figures out how to use an LLM, they largely stick to that pattern.
Stark differences emerged between activity levels: more active users had more successful conversations and used the LLM for more complex, professionally oriented tasks. Meanwhile, WildChat-4.8M was found to be significantly skewed toward highly proficient 'power' users, meaning it does not represent typical user-AI interactions—an important caveat for researchers relying on that dataset. The findings challenge assumptions that LLM usage naturally evolves or that existing datasets capture the full spectrum of user behavior.
- 12,000 Bing Copilot users analyzed longitudinally; individual behavior barely changed over time.
- Active users engage in more complex, professional tasks than less active users.
- WildChat dataset is skewed toward power users, not representative of typical interactions.
Why It Matters
Dataset biases and sticky user habits mean AI behavior predictions must account for individual inertia.