OpenAI's Free Classifier to Detect Child Abuse Images, Even AI-Made
· Cortex Solutions

OpenAI is giving vetted organizations a free image classifier that flags suspected child abuse material, including AI-generated images, and adds new ChatGPT warnings.
OpenAI announced this week a new set of child-safety measures, the most notable being an image classifier that detects suspected child sexual abuse material (CSAM), including AI-generated images. The news was reported by Fox News on November 16, 2026, and by the tech site Digit on November 17, 2026, both based on the company's announcement and official post.
Why platforms need a new tool
According to Fox News, most existing detection systems rely on digital fingerprints (hashes) that recognize files already identified and recorded. New content, especially AI-generated images, often has no matching fingerprint and slips past those systems.
The new classifier instead examines visual signs in the image itself, so it can flag suspected material even if it has never been identified before. It assigns each image a score that helps human reviewers prioritize cases, which means the final decision stays with people.
Who can access it?
According to Digit, quoting OpenAI, the company is offering the classifier through its API at no cost to vetted organizations, to make CSAM detection more accessible to child-safety organizations and the wider industry, including teams of varying sizes. OpenAI says it is the first broadly available CSAM classifier that can also flag whether material may be AI-generated.
OpenAI developed the classifier with assistance from the US National Center for Missing & Exploited Children (NCMEC), and is working with the Tech Coalition and ROOST to make it available to more organizations.
What do the early results say?
Both outlets cited figures that come from OpenAI and participating organizations, not independent testing:
- In an OpenAI evaluation on images uploaded to its systems, the new classifier flagged 48% fewer images for review while detecting roughly the same amount of abusive material as its existing tool.
- Images flagged by the classifier were about twice as likely to actually contain abuse.
- According to Fox News, reviewers at the AI platform Fal confirmed 72% of high-confidence flags as previously unidentified CSAM.
- SafetyKit reported that the classifier detected 90% of cases involving drawings or animations that sexualize children in its evaluation.
Fox News noted that independent testing will determine how reliably the tool performs across different platforms.
Changes inside ChatGPT
Alongside the classifier, OpenAI added new ChatGPT warnings that appear when someone tries to find this kind of material, explaining that sexualizing children causes harm even when the material is AI-generated. After the warning, the company removes the conversation, pauses the account for 24 hours, and points the user to specialist support. Repeated violations may lead to longer restrictions, permanent account closure, and reports to authorities.
According to Fox News, OpenAI has also begun participating in the Tech Coalition's Lantern program for sharing information about suspected exploitation across platforms. It is introducing tools and training for child-safety law enforcement units, starting with a tool that helps authorized teams review long chat logs and organize timelines with findings linked to source material. It is also publishing guidance for developers on detecting suspected abuse and setting up review procedures.
What it means for Saudi businesses
Content moderation is no longer only a concern for global platforms. Any Saudi company that lets users upload images, such as online stores, delivery apps, education platforms, classifieds and social apps, is responsible for what is published through it. The practical lessons:
- Review your app or platform's content policy: who uploads images, who reviews them, and what the reporting and removal process is.
- Fingerprints alone are no longer enough as generated images spread; AI-based tools are now part of baseline protection.
- The final decision stays human: the classifier is designed to prioritize work for the review team, not replace it, which is a healthy model for any use of AI in sensitive decisions.
- Eligible organizations, such as child-safety groups and platforms hosting user content, can apply for the classifier at no cost, lowering the barrier for small teams.
- When building any image-generation or file-upload feature, design safety in from day one, not after a problem occurs.
At Cortex Solutions we help companies build apps and platforms with security and content governance in mind from the start, and integrate AI tools into workflows responsibly: https://cortex.sa/services/ai-automation