OpenAI on Monday introduced a set of safeguards aimed at stopping child sexual abuse material from being created or spread through its tools, affecting both the people who use ChatGPT and the outside organizations that screen images for abuse. The announcement comes as the company faces lawsuits over alleged harm to users, particularly children.
The centerpiece is a new image classifier that can identify CSAM, including images produced by AI. OpenAI is providing the classifier free through its API to a select group of companies. According to the company, the tool looks for visual indicators of abuse in images it has not seen before and assigns a score that gives reviewers an indicative signal for handling CSAM cases.
OpenAI said it built the classifier with the National Center for Missing and Exploited Children, or NCMEC, and is working with Tech Coalition and ROOST to broaden access. In early partner testing, the classifier flagged 48% fewer images while finding the same number of abusive images, which the company said reduces reviewer workload. Flagged images were also twice as likely to be abusive.
On the consumer side, ChatGPT will display warning screens when users attempt prompts that sexualize children, pointing them toward specialized support to stop the behavior. Accounts behind such attempts will be suspended for 24 hours as a warning, and repeat offenses could result in permanent removal. The app already blocks CSAM content and related requests. OpenAI said it developed the messaging with Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan at the University of Tasmania, who independently evaluated a trial of the deterrent messages.
OpenAI is also publishing guidelines for developers who use its tools, covering how to prevent, detect and help users report CSAM. Early testing showed the guidelines helped several developers adopt child safety tooling and register to report CSAM to NCMEC. For law enforcement, the company is offering AI-powered features including dedicated plug-ins, investigative tools and complex data analysis, with access managed by program administrators, plus training and a field guide for investigators.
The push follows mounting scrutiny of OpenAI's safety record. The nonprofit Common Sense Media rated its technology an unacceptable risk for teens, and other chatbots, specifically Grok, have been used to generate nonconsensual sexualized imagery, including of children. Over the past 12 months OpenAI has released a safety blueprint, updated its content guidelines to prohibit inappropriate material, retuned its model to prevent harmful output, and launched a version of ChatGPT for teens.
More AI news from TechManNews.






