New York Bans 12 Unsafe AI Chatbot Features for Under-18s
· Cortex Solutions
New York signed a law on October 28, 2026 barring 12 unsafe AI chatbot features for minors, with fines up to $25,000 per violation, in force on January 1, 2027.
On October 28, 2026, New York Governor Kathy Hochul signed a new law targeting how AI chatbots are designed when the user is under 18. According to the official statement from the governor's office that day, the legislation, numbered S.9051B/A.10379C, prohibits companies from offering minors features it describes as unsafe. In the same statement, James P. Steyer, founder and CEO of Common Sense Media, called it the strongest kids' AI safety bill in the country.
The significance of the law lies in its target: it does not only address harmful content, it reaches into the product's design and how the chatbot presents itself to the user.
What does the law prohibit?
The International Association of Privacy Professionals (IAPP) reported on October 28 that the law covers 12 broad categories of unsafe features. The governor's office listed the most prominent ones, which companies may not offer to users under 18, or to any user unless the company has first confirmed that the user is not a minor:
- Pretending to be a real person, to be alive or to have human emotions.
- Claiming a personal, romantic, professional or authority-figure relationship with a child.
- Unprompted emotional questions, and using a child's private health or personal information from earlier conversations.
- Promoting or helping with suicide, self-harm, eating disorders or drug and alcohol use.
- Encouraging children to keep secrets, isolate themselves or avoid asking parents, trusted adults or professionals for help.
- Any sexually explicit content or child sexual abuse material.
PPC Land, which published the bill's text in its October 28 report, adds details that show how far the law goes into design. The list includes outputs that engage in flattery or sycophancy with the user, using information about the user's health or personal matters acquired more than twelve hours earlier or in a previous session, and outputs that optimize engagement in a way that supersedes the system's safety guardrails.
Age verification and data deletion
The governor's office said the law requires companies to conduct age verification and to offer more than one way of doing so, including at least one option that does not depend solely on a government ID. Age information must be deleted after it is used. PPC Land added, citing the bill's text, that self-declaration can only be relied upon if the user declares that they are a minor; a user simply saying they are an adult is not enough.
Penalties and effective date
According to the governor's statement, the attorney general may seek civil penalties of up to $25,000 per violation, along with damages, restitution, the destruction of unlawfully collected data or any AI models trained on it, and other injunctive relief. The attorney general must also maintain a public website where New Yorkers can file complaints. Both IAPP and PPC Land said the law takes effect on January 1, 2027, and PPC Land noted that it gives individuals no private right of action.
Who is covered, and who is exempt?
PPC Land said the law defines a covered AI companion as a generative AI system with a natural language interface, by writing or sound, that provides ongoing, adaptive responses to user inputs. It reported that a Common Sense Media fact sheet lists Character.ai, Replika, Nomi, Grok, Meta AI and Snap's My AI as in scope, along with ChatGPT, Gemini and Claude.
The point that matters most to ordinary businesses is the exemption. According to PPC Land's report on the text, the law excludes chatbots used solely for customer service or for information about a business's products and services, as well as systems used for internal purposes or employee productivity. It attaches one condition: the system must be unable to respond on topics outside its specified purpose.
Disputed wording
The law did not pass without objection. PPC Land reported that the Computer & Communications Industry Association (CCIA) asked the governor to veto it, arguing that terms such as flattery and sycophancy are hard to apply objectively. IAPP quoted Justine Gluck of the Future of Privacy Forum as saying the wording is broad and somewhat undefined, and that it is unclear where ordinary conversational responsiveness ends and prohibited behavior begins.
What does this mean for Saudi businesses?
This is a law of one American state, and this article is not legal advice. But it shows the direction regulators are taking with chatbots, and a Saudi business running an assistant on its website or WhatsApp can draw practical lessons from it now:
- Define the assistant's scope clearly. The New York exemption depends on the system being unable to go outside its purpose, which is sound design anywhere: a customer service assistant should stick to customer service.
- Do not let the assistant pretend to be human. State clearly that the customer is talking to an automated system, and avoid any wording that implies feelings or a personal relationship.
- Review what it remembers. Keeping customers' health or personal details across conversations for no clear need is a growing risk, especially in clinics, pharmacies and education.
- Provide a clear route to a human, particularly in sensitive conversations.
- If your service reaches children or teenagers, review it now, before any regulatory requirement arrives.
Building these limits in from the start costs far less than rebuilding the system after a problem or a new requirement.