When Vocabulary Comprehension Fails Clinical Reasoning: Evaluating Therapy Bots' Safety Risks for Generation Alpha

Generative AI systems are increasingly being used as informal mental health support resources by Generation Alpha, with a significant proportion of U.S. adolescents seeking mental health advice from these platforms. Despite AI models being trained on extensive psychological literature, their safety for youth, particularly given characteristic communication patterns like hyperbolic language and rapid semantic drift, remains unvalidated. This has become a critical concern following reported adolescent deaths linked to AI chatbot interactions, necessitating systematic evaluation.

Why it matters

The widespread adoption of AI for sensitive applications like mental health support, particularly among vulnerable populations, presents significant ethical and safety challenges. Organizations developing or deploying AI must critically assess their understanding of nuanced user communication to prevent unintended harm and mitigate reputational, legal, and regulatory risks.

Key insights

• 13.1% of U.S. adolescents (5.4 million) are using generative AI for mental health advice.
• These AI systems, ranging from therapy apps to general chatbots, leverage large language models trained on psychological literature.
• The safety of these systems for youth communication patterns, characterized by hyperbolic language, ironic positivity, rapid semantic drift, and contextual polysemy, has not been validated.
• Multiple adolescent deaths have been linked to AI chatbot interactions, underscoring the urgency of systematic evaluation.
• A research initiative has developed two benchmarks to evaluate these systems, including 64 Gen Alpha mental health expressions validated by native speakers.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.20345