OpenAI adds summaries to ChatGPT to flag self-harm, violence

OpenAI announced Thursday that ChatGPT will use temporary “safety summaries” to identify escalating signs of suicide, self-harm and potential violence that emerge over the course of a conversation. The feature is intended to let the model interpret later messages in light of earlier warning signs rather than judging each message in isolation.

OpenAI described the summaries as short, narrowly scoped notes that capture safety-related context from earlier parts of a chat. In a blog post, the company wrote the notes are used only in serious situations and are not stored as permanent memories or for personalization.

The summaries are designed to help the system avoid offering harmful information, de-escalate tense exchanges and guide users to support when appropriate. OpenAI wrote that context can change the meaning of an otherwise ordinary request and that the tool focuses on acute scenarios including suicide, self-harm and harm to others.

OpenAI said it updated model policies and training with input from mental health experts to improve the model’s ability to spot warning signs that develop over multiple messages. The company added it analyzed hundreds of millions of interactions to inform the approach and limited use of the summaries to narrow, serious cases.

The announcement comes as OpenAI faces multiple lawsuits and government reviews alleging ChatGPT failed to respond properly in dangerous conversations. In April, the Florida attorney general opened an investigation into the company over child safety, self-harm and issues tied to a 2025 mass shooting. A federal lawsuit alleges ChatGPT helped a suspected gunman plan an attack, and a separate California state lawsuit filed this week names OpenAI and its CEO in a case by a family whose 19-year-old died from an accidental overdose.

OpenAI said recognizing risk that only becomes clear over time is an ongoing challenge. The company added it may consider similar context-aware techniques for other high-risk areas, such as biological or cyber safety, if careful safeguards are established. The blog post concluded that the company will continue to refine safeguards as its models and understanding evolve.