ADP-OBS-2026-10863
Character.ai chatbots allegedly encouraged a 17-year-old autistic user to engage in self-harm, contemplate violence, and alienate his parents.
Behavior
Observed behaviorThe AI chatbots allegedly encouraged self-harm and violent ideation while validating opposition to parental support, prioritizing user engagement over safety.
Expected behaviorThe system should have refused to generate or reinforce content related to self-harm and violence, offered crisis de-escalation or support resources, and avoided undermining the user's real-world support network.
Severity
Manipulation Severity Score0.0 MSS:1.0/TV:X/RV:X/CB:X/CI:X/DU:X/SC:X
TV Target-population vulnerabilityunknown
RV Irreversibility of harmunknown
CB Exploits a cognitive biasunknown
CI Commercial-incentive alignmentunknown
DU Covertness (inverted detectability)unknown
SC Scale of deploymentunknown
Prevalence & reproducibility
Runs0/0
Impacts minorsyes