Research finds 'what-if' analysis better than questions for AI-assisted clinical decisions
New study reveals that 'what-if' scenarios promote reflection, but data-driven questions fall short.
A new study from researchers led by Simon Fischer on arXiv (cs.HC) tackles a critical challenge in human-AI interaction: how to prevent clinicians from blindly trusting decision-support systems (DSS). The team tested two friction methods—data-driven questions and 'what-if' scenario analysis—on a replicated DSS used in real clinical practice. Through in-situ interviews with 7 medical experts, they gathered qualitative feedback on whether these interventions actually promote cognitive engagement and reflection, or simply add friction without benefit.
The findings reveal a clear split: clinicians found data-driven questions unhelpful for reflective thinking, though they acknowledged them as useful reminders to consider relevant information. In contrast, inspecting 'what-if' hypotheticals was seen as genuinely valuable for improving patient care and for training novice clinicians. The prototype was praised more as a training tool than a live decision aid. Based on the feedback, the authors offer design recommendations for implementing friction in clinical workflows. This work contributes to the growing field of human-AI interaction research that aims to mitigate overreliance on AI through thoughtful interface design.
- Data-driven questions were perceived as unhelpful for reflection but useful as reminders to consider relevant information.
- 'What-if' hypotheticals were found useful for improving patient care and as a training tool for novice clinicians.
- Study based on a real-world decision task with 7 clinicians, replicating an actual decision-support system used in clinical practice.
Why It Matters
Insights for designing AI tools that keep clinicians engaged and reflective, reducing automation bias in healthcare.