AI Health Chatbots Give Risky Advice to Skin Cancer Patients at Home

A team of researchers from Carnegie Mellon and several medical centers asked 73 skin cancer patients to use AI chatbots while caring for their wounds at home after surgery. The goal was simple: see whether a chatbot stuffed with trusted medical information could actually help someone figure out what to do next.

The answer was mostly no — and the reason is surprisingly human. Nearly 42% of the questions patients needed answered couldn't be resolved by medical facts alone. They were things like "is this redness normal?" or "my bandage slipped off in the shower, what now?" Answering those requires seeing the wound, knowing the patient's bathroom, or remembering what they did ten minutes ago. A chatbot has none of that.

So the researchers replayed the questions to seven leading AI models, this time adding the full surgical and aftercare instructions. Mistakes stayed high. The AI often acted as if it knew things it couldn't possibly know, even when told to be cautious. Worse, when given the full procedure details, six of the seven models started suggesting later steps too early — telling patients to move on before they were ready.

The takeaway isn't that medical AI is useless. It's that knowing medicine isn't the same as helping a specific person in a specific moment. The authors propose a design approach where the assistant recognizes what it can't establish, asks the patient to check, and adjusts the kind of help it gives. Until that exists, treat chatbots as a supplement to your care team — never a replacement for a nurse's eyes and judgment.