Australian AI Health Test Reveals Dangerous Mistakes in Medical Advice

A new study tested 28 large language models (AI systems like ChatGPT) on 142 questions based on Australian clinical guidelines. The questions covered areas like medicine subsidies (PBS), doctor billing (MBS), professional obligations, emergency triage, and immunisation. Each question had one correct answer, and models were tested both with and without the relevant rule provided. Results showed a wide range in accuracy: from about 52% to 93% correct when models had to rely on their own knowledge. The top models were statistically tied, meaning no clear winner.

Most mistakes happened because models didn't know the current Australian rule. In 86% of failed questions, the model answered correctly once the rule was given. This suggests AI often fails to recall local guidelines rather than failing to apply them. Worryingly, models were confidently wrong between 6.6% and 47.8% of the time. On five immunisation questions where guidelines give no directive, 82% of answers still stated a rule. On Aboriginal and Torres Strait Islander health questions, models did no worse overall, but some wrong answers applied general population rules, older rules, or other countries' rules.

The study's author, a general practitioner, proposes treating AI as an ingredient in clinical products, with minimum standards for local rules. The benchmark is not yet independently reviewed and measures base models, not finished clinical tools. It does not approve any model for clinical use. For everyday patients, this means AI in healthcare could give outdated or incorrect advice, especially about Australian-specific policies. While AI can be a helpful tool, it's not a substitute for professional judgment, and patients should verify important information with their doctor.