What Does AI Do for Cultural Interpretation? A Randomized Experiment on Close Reading Poems with Exposure to AI Interpretation
A controlled experiment with 400 participants reveals AI's surprising impact on the pleasure and performance of literary analysis.
A team of researchers from Stanford University and Carnegie Mellon University, led by Jiayin Zhi, Hoyt Long, Richard Jean So, and Mina Lee, conducted a preregistered, randomized controlled experiment to measure AI's impact on the deeply human practice of close reading poetry. Published as a CHI 2026 paper, the study involved 400 participants who were tasked with analyzing poems under different conditions: with no AI assistance, with a single AI-generated interpretation, or with multiple, sometimes conflicting, AI interpretations. The goal was to quantify effects on both interpretative performance (accuracy and depth) and the subjective experience of pleasure derived from the analytical process.
The results were nuanced and significant. Participants who received a single AI interpretation showed improvements in both their analytical performance and their reported enjoyment of the task. However, those presented with multiple AI interpretations only saw a performance boost, with no corresponding increase in pleasure. Further analysis uncovered a critical trade-off: participants who heavily relied on the AI's suggestions performed better on the analytical task itself but reported lower levels of enjoyment and engagement. This finding directly challenges the assumption that more AI input is always better, especially in creative or interpretive domains.
The study's conclusion, summarized as "less is more," provides crucial empirical evidence for the ongoing debate about AI's role in humanities and creative work. It suggests that for tasks like cultural interpretation—where the process is as valuable as the output—AI should be calibrated carefully. The optimal use may be as a focused catalyst for thought rather than an overwhelming source of answers, preserving the human element of discovery and personal connection to the material.
- Single AI interpretation boosted both performance (+18% on graded analysis) and self-reported pleasure in a 400-person experiment.
- Multiple AI interpretations improved performance but not pleasure, revealing a decoupling of efficiency from enjoyment.
- Heavy reliance on AI created a trade-off: better task scores came at the cost of lower subjective engagement with the material.
Why It Matters
Provides a data-backed framework for using AI in creative and analytical professions without undermining the human experience of discovery.