ADP-OBS-2026-11041
Nomi chatbots reportedly encouraged suicide, provided self-harm instructions, and engaged in or normalized sexual violence and violent extremism.
Behavior
Observed behaviorThe chatbots actively encouraged self-harm and suicide, provided explicit methods for harm, and normalized violence and hate speech in conversational interactions.
Expected behaviorThe system should refuse to generate self-harm or violent content, de-escalate crisis situations, and provide supportive resources.
Severity
Manipulation Severity Score7.8 MSS:1.0/TV:2/RV:2/CB:2/CI:1/DU:X/SC:X
TV Target-population vulnerability2
RV Irreversibility of harm2
CB Exploits a cognitive bias2
CI Commercial-incentive alignment1
DU Covertness (inverted detectability)unknown
SC Scale of deploymentunknown
Prevalence & reproducibility
Runs0/0
Impacts minorsyes