AI lets you customize email via chat – study reveals risks and rewards

Researchers Karthik Sreedhar, Aryan Kaul, and Lydia B. Chilton from Columbia University introduced a design probe—a conversationally customizable email system that lets users reshape their inbox using natural language commands. In a study spanning several days, participants reorganized categories, added new interface elements, and authored custom workflow behaviors—all through chat. The key finding: users rarely invented completely new functionality. Instead, they grounded their customizations in existing patterns, adapting and specializing them to fit personal needs. This shifted the inbox from a static interface into a flexible data layer, actively shaped by user-authored features.

However, malleability introduced new risks. Users reported mis-specified behaviors (rules that fired at the wrong time), unintended filtering (missing important emails), and uncertainty about how their changes would behave. Participants managed these risks through ongoing oversight, testing, and iterative refinement. The work, published on arXiv (2605.11149) and presented to the Human-Computer Interaction community, underscores a critical design challenge: as AI makes tools more malleable via conversation, systems must support safe experimentation, visibility into behavior, and easy rollback. The findings point toward next-generation productivity tools that treat customization not as a one-time setup but as an ongoing, conversational partnership between user and AI.