Earlier this month, OpenAI introduced a wellness council intended to address growing concerns about mental health interactions with its chatbot, though notably absent from the council is a suicide prevention expert.

In parallel, OpenAI has implemented parental controls for minors using ChatGPT and is developing an age prediction system to identify young users automatically and apply stricter age-appropriate safeguards.

Significance of AI Conversations

The recently disclosed data suggests that conversations potentially triggering concerns about psychosis, mania, or suicidal thoughts are "extremely rare," posing challenges for accurate measurement.

OpenAI estimates that approximately 0.07% of weekly active users and 0.01% of messages may indicate signs of mental health emergencies associated with these conditions.

Additionally, about 0.15% of users and 0.03% of messages show possible emotional attachment to ChatGPT.

An analysis conducted by OpenAI on over 1,000 challenging mental health-related exchanges revealed that the new GPT-5 model adhered to desired behaviors in 92% of cases, a significant improvement over a previous version released on August 15 which complied at only 27%.

The latest iteration demonstrates enhanced performance against OpenAI's safeguards during prolonged interactions.

OpenAI has committed to integrating new evaluations aimed at measuring serious mental health issues among ChatGPT users into their baseline safety testing metrics, now including benchmarks for emotional reliance and non-suicidal psychological crises.

Despite these ongoing mental health challenges, CEO Sam Altman announced on October 14 that verified adult users would be permitted erotic interactions with ChatGPT starting in December.

Following lawsuits in August concerning teen suicides, content restrictions were initially tightened after having been relaxed in February.

Altman stated they made ChatGPT "pretty restrictive" out of caution for mental health considerations but recognized this diminished its utility for those without such issues.

If you or someone you know is experiencing suicidal thoughts or distress, please contact the Suicide Prevention Lifeline at 1-800-273-TALK (8255) to connect with a local crisis center.