AI Safety

Agentic AI boosts physician trust 89% in clinical decisions

Doctors trust AI that shows its reasoning steps—but still need oversight.

Deep Dive

Medical AI is evolving from static reasoning to agentic AI—a paradigm where systems autonomously invoke external tools and render intermediate reasoning steps transparent to users. A new study published on arXiv (accepted at AMIA 2026) explored whether this transparency actually builds physician trust. Three physicians evaluated 315 multimodal clinical cases, comparing agentic AI against non-agentic baselines. The study measured both process-oriented cognitive trust (how much physicians believed the AI's reasoning) and outcome-oriented behavioral reliance (whether they followed the AI's recommendations).

Results showed a dramatic preference: physicians exhibited significantly higher cognitive and behavioral trust for the agentic model (P < 0.001). In treatment planning tasks, physicians chose the agentic AI's reasoning in 89.57% of cases. Importantly, process-oriented trust was strongly associated with outcome-oriented reliance, suggesting that showing work builds confidence. Yet the study also found measurable over-reliance on incorrect agentic outputs, indicating that decision-logic transparency alone is insufficient. The authors emphasize the continued need for rigorous clinician oversight, highlighting a key limitation of even the most transparent AI systems.

Key Points
  • Three physicians evaluated 315 multimodal clinical cases comparing agentic vs. non-agentic AI.
  • Physicians showed significantly higher trust (P < 0.001) and preferred agentic AI in 89.57% of treatment planning tasks.
  • Despite higher trust, over-reliance on incorrect agentic outputs still occurred, requiring human oversight.

Why It Matters

Transparent AI reasoning can boost clinical trust, but doctors must remain vigilant against automation bias.

📬 Get the top 10 AI stories daily