Clinicians reject autonomous AI prescribing without confidence-based veto power

A new paper from LaRocco et al. (arXiv:2606.25108) examines the regulatory and technical requirements for autonomous AI prescribing systems, in light of recent U.S. legislation H.R. 238 and Utah's prescription-renewal pilot that authorize AI to prescribe medications autonomously. The authors argue that current regulations only require aggregate model performance metrics, but fail to mandate three critical architectural features: calibrated per-prediction confidence thresholds for escalating decisions to humans, differentiated communication of uncertainty (model ignorance vs. genuine clinical ambiguity), and inferential transparency at the moment of decision to enable liability allocation. A survey of 136 U.S. prescribing clinicians validates these requirements.

The survey results show that clinicians would not permit autonomous prescribing without a calibrated confidence-based escalation mechanism. When uncertainty is aleatoric (genuine clinical ambiguity), clinicians preferred a competing-options summary; but when uncertainty is epistemic (model ignorance), they demanded abstention. They were only willing to accept additional liability when inferential transparency allowed substantive judgment under acknowledged uncertainty. The study concludes that systems meeting these requirements would function less as true autonomous agents and more as heavily supervised decision-support tools. The findings offer clear guidance for regulators: ethical autonomy in AI prescribing must be constrained, with liability aligned to the institutional actors who control system design and deployment.