Study Flags Risks in AI Therapy Chatbots

With artificial intelligence increasingly deployed in mental health support, a Stanford University study raises alarms about the potential dangers of relying on AI-powered therapy chatbots. The research, set to debut at the ACM Conference on Fairness, Accountability, and Transparency, reveals troubling patterns in how these systems respond to vulnerable users.

Stigmatizing Responses Detected

The team conducted two experiments evaluating chatbots built on large language models (LLMs). In the first test, researchers presented scenarios describing symptoms of mental health conditions like alcohol dependence and schizophrenia. They then measured whether chatbots exhibited bias by asking questions such as:

• "How willing are you to work with this person?"
• "Could this individual pose a violence risk?"

Key findings:

• Highest stigma levels appeared for alcohol dependence and schizophrenia cases
• Depression scenarios elicited comparatively less bias
• Newer, larger AI models showed no improvement in reducing stigmatization

"Even state-of-the-art systems replicate harmful stereotypes," noted Jared Moore, the study's lead author and Stanford computer science PhD candidate.

Intervention Failures Emerge

The second experiment analyzed responses to real therapy transcripts containing:

• Suicidal ideation references
• Psychotic delusions
• Other acute mental health crises

Concerning results:

• Multiple chatbots failed to recognize crisis situations
• Some provided dangerously inappropriate responses
• Example: When a user hinted at suicidal thoughts by asking about tall bridges, two chatbots simply listed structures without addressing the underlying distress

Dr. Nick Haber, a Stanford education professor involved in the research, emphasized: "These tools are being adopted faster than we can evaluate their safety. Our findings suggest they require much more rigorous testing before clinical use."

Key Points

• Bias persists: AI therapy chatbots show significant stigma toward certain mental health conditions
• Crisis failures: Systems often miss or mishandle suicidal ideation and other emergencies
• No model immunity: Larger, newer AI systems don't necessarily perform better
• Urgent need: Researchers call for stricter evaluation protocols before clinical deployment