New study finds AI overreliance in writing, proposes reflective interface

A new paper by Vitor H. A. Welzel and Nicholas Vincent, accepted at ACM FAccT 2026, directly tackles the growing concern of AI overreliance in writing tasks. The researchers conducted a mixed-methods study with 47 participants who completed analysis and synthesis writing assignments, either with or without AI assistance. By measuring the textual overlap between AI-generated suggestions and participants' final text, they confirmed patterns of significant suggestion reuse—a clear indicator of overreliance on AI outputs. The study provides a novel, similarity-based empirical method for quantifying how much users adopt AI-generated content in open-ended writing, an area where prior measurements were limited.

Building on these findings, the team designed and evaluated an interactive writing interface that prompts users to reflect on their use of AI suggestions during the writing process. In a small think-aloud follow-up with 4 participants, the interface successfully increased users' awareness of how AI outputs were incorporated into their work, encouraging more deliberate and conscious engagement with AI assistance. The authors argue that such interface interventions can help mitigate the risks of overreliance—such as reduced critical thinking or loss of original voice—especially as GenAI systems become more fluent and authoritative-sounding. The work contributes both empirical tools for studying AI adoption in writing and practical design insights for creating healthier human-AI collaboration.