New study reveals generative AI fails at medication reasoning and emotional support for diabetes patients

Researchers from a mixed-methods study evaluated how generative AI handles Type 2 Diabetes (T2DM) self-management from both patient and physician perspectives. In Study 1, they analyzed 784 patient-generated queries to identify seven categories of informational needs and developed a structured five-dimensional physician rating rubric covering Accuracy, Safety, Clarity, Integrity, and Action Orientation. Study 2 then had seven physicians score responses from four different AI models and conducted in-depth interviews to understand their evaluative reasoning.

The results reveal a clear performance gap: AI models excel at factual explanations and lifestyle guidance but consistently fall short on medication reasoning and emotional support. Two critical concepts emerged: the "pre-visit primer" frames AI as a tool to prepare patients for clinical encounters rather than replace physicians, while the "fluency illusion" describes how polished, confident language can mask a lack of genuine clinical authority. Both patients and physicians identified three shared limitations—role boundaries, emotional inadequacy, and personalization gaps—though they diverged in how they prioritized these issues. The study concludes with four design directions: task-aware orchestration, risk-aware fallback, dynamic personalization, and emotionally attuned interaction.