AI Mental Health Chatbots Give Bad Advice at the Worst Times, Study Finds
If you use AI for mental health support, timing could make all the difference.
chatbots are increasingly used for mental health support, but a new study reveals they may not always help. Researchers analyzed over 14,000 simulated conversations based on real chatbot histories from 22 users. They looked at how a memory-equipped AI responded to crises like suicidal thoughts and anxiety over time. The AI remembered past disclosures, but its advice was often mistimed. After a crisis, it frequently gave recommendations before the person was ready, especially when recovery stalled. This happened less in calmer moments.
The study also found that the AI often left out important support, like following up or planning next steps. While it rarely said anything explicitly harmful, it missed key elements that mental health professionals expect. Users' communication styles also affected how often and when the AI gave advice. The researchers created a framework called CARE, developed with mental health experts, to evaluate the timing and quality of advice, including what's omitted.
This shows that checking single responses isn't enough. AI needs to be evaluated over long conversations to ensure it provides consistent, timely support. For users, this means that while AI can be a helpful tool, it's not a substitute for professional care. For developers, it highlights the need to design AI that understands when to offer advice and when to wait. As AI becomes more integrated into mental health support, accountability for long-term interactions is crucial.
- AI mental health chatbots often give advice too soon after a crisis, which can hinder recovery.
- The AI frequently misses important follow-ups, like checking in or planning next steps.
- Testing AI on single responses isn't enough; we need to evaluate how it handles conversations over time.
Why It Matters
If you rely on AI for mental health support, mistimed advice could set back your recovery.