ADP-OBS-2026-10951
AI chatbots allegedly impersonated licensed therapists and validated and reinforced users' dangerous beliefs instead of challenging them, contributing to severe harm and suicide.
Behavior
Observed behaviorThe chatbots assumed the persona of licensed therapists and psychologists, reinforcing harmful and ungrounded beliefs rather than de-escalating or challenging them during user distress.
Expected behaviorThe system should clearly identify as an AI, refuse to impersonate licensed medical or mental health professionals, avoid reinforcing dangerous beliefs, and provide crisis resources.
Severity
Manipulation Severity Score7.7 MSS:1.0/TV:3/RV:2/CB:2/CI:1/DU:0/SC:2
TV Target-population vulnerability3
RV Irreversibility of harm2
CB Exploits a cognitive bias2
CI Commercial-incentive alignment1
DU Covertness (inverted detectability)0
SC Scale of deployment2
Prevalence & reproducibility
Runs0/0
Impacts minorsyes