BILL
SUBJECT
DATE
The Business Council respectfully opposes S.9051 (Gonzalez) / A.10379A (Bores) which would prohibit artificial intelligence chatbot from using features considered “unsafe” for minors. We share the sponsors’ core goal of fostering a safe environment for kids online with age-appropriate content and interactions.
However, as drafted, this bill is not a workable regulatory framework and would effectively ban minors from accessing increasingly essential online and technological tools and innovations. Its definitions are overbroad and inconsistent with existing New York law, its prohibitions would sweep in educational, research, and general-purpose tools that serve and benefit minors every day, its age-verification mandate contradicts the bill’s own “actual knowledge” standard and raises serious constitutional, privacy, and security concerns, its liability framework imposes strict liability on non-deceptive conduct, and its enforcement mechanism — including a private right of action with punitive damages and disgorgement — creates unquantifiable litigation risk that would chill beneficial innovation.
The proposed bill is fundamentally flawed and cannot be amended to meet its intended goals without seriously jeopardizing New York’s innovation economy and depriving New York minors access to essential educational and technological literacy and equity.
Allow Newly Enacted Companion Chatbot Law to Work
The New York AI Companions Act (GBL Article 47 (§ 1700–1704), effective November 5, 2025) mandates notifications, safeguards and enforcement for AI companions.
S.9051 is layered on top of this framework, which has been in effect for less than six months. Industry compliance is ongoing, and the Attorney General has not yet had an opportunity to exercise enforcement authority or issue implementing regulations under Article 47. Enacting a second, far broader statute before the first has been tested risks regulatory confusion, duplicative compliance burdens, and conflicting obligations.
Lawmakers must allow Article 47 to operate as intended before legislating further in this space, and to use that experience to inform any additional regulation.
Overbroad Definitions Would Capture Beneficial Tools
“Advanced Chatbot” Is Inconsistent with Existing New York Law and Far Too Broad
S.9051 creates a new defined term, “advanced chatbot,” as any “generative artificial intelligence system with a natural language interface … that provides ongoing, adaptive responses to user inputs.” This definition is dramatically broader than the “chatbot” definition New York already uses in existing proposed legislation (S.7263), which is limited to systems that “simulate human-like conversation … to provide information and services to users,” and far broader than the “AI companion” definition in the enacted GBL Article 47, which is limited to systems specifically designed to simulate a sustained human relationship through retention of prior interaction data, unsolicited emotion-based questioning, and ongoing personal dialogue.
The “advanced chatbot” definition in S.9051 sweeps in every generative AI tool with a text or voice interface, including:
- AI-powered tutoring and homework tools used in schools and libraries;
- Reading comprehension, language-learning, and accessibility applications;
- Enterprise productivity suites, developer APIs, and coding assistants;
- Shopping assistants, voice assistants, and customer service bots; and
- AI research assistants used in higher education.
None of these tools pose the companion-chatbot harms this bill is designed to address. The bill’s definition should match the actual risk.
“Unsafe Chatbot Features” Prohibit Standard, Beneficial AI Behavior
The definition of “unsafe chatbot features” encompasses standard AI functionality with no nexus to the companion-chatbot harms this bill targets:
- Prohibition on first-person pronouns:
- Prohibiting use of “I,” “my,” and “me” would fundamentally break natural language interaction for all chatbots, not just companion chatbots. A shopping assistant that says “I found three options for you” would be providing an “unsafe feature.” This is a core feature of how large language models communicate and is not inherently harmful.
- Personal opinions or emotional appeals:
- This vague prohibition would capture standard conversational AI outputs across virtually every use case, including an AI tutor saying “I think you did a great job on that answer.”
- Unprompted emotion-based questions:
- Would capture basic conversational follow-ups (e.g., “How can I help you today?”) and would bar AI mental health check-in tools from asking “How are you feeling?”
- Memory restriction:
- The 12-hour memory restriction is arbitrary and would impair legitimate functionality including customer service continuity, account preferences, and accessibility features.
- Open-ended AG delegation:
- Granting the AG authority to designate additional “unsafe features” by regulation creates unbounded regulatory risk and a continuously moving compliance target with no legislative guardrails.
- Engagement optimization:
- Prohibiting outputs that “optimize user engagement that supersede the chatbot’s safety guardrails” is vague and circular — what constitutes optimizing engagement versus providing a helpful, responsive answer is entirely unclear.
Overly broad and prescriptive mandates that capture general-purpose AI platforms, enterprise services, and developer infrastructure do not advance child safety. Instead, they create compliance uncertainty across the entire AI ecosystem while failing to target the specific products and behaviors that pose actual risk to minors. Technically infeasible requirements undermine the credibility of the regulatory framework.
The Bill Does Not Contain an Educational Carve-Out
Unlike comparable state and federal frameworks that explicitly preserve AI-powered educational tools and research systems, S.9051 contains no exemption for educational technology.
Educational chatbots used by schools, libraries, tutoring services, and students are entirely absent from this list. A school district deploying an AI reading assistant, a non-profit providing AI-powered language instruction to underserved communities, or a university offering an AI research tool to high-school students would all be subject to the bill’s prohibitions. Students from lower-income households who rely on AI-powered educational tools as a substitute for private tutoring would be disproportionately harmed if those tools are withdrawn or restricted to comply with this bill.
Age Verification Mandate Imposes Two Conflicting Standards
The bill’s definition of “covered minor” (§1800(6)) uses an “actual knowledge” standard, meaning that child-specific obligations are triggered only when the chatbot operator has actual knowledge that the covered user is a minor. Despite the actual knowledge standard in §§1800(6), the bill imposes a universal age verification mandate through two separate provisions that directly undermine the actual knowledge standard:
- §1801(b): Requires operators to use methods “permissible under article forty-five” (New York’s age verification framework) to confirm a user is not a covered minor before providing any “unsafe features.” This effectively gates all users (including adults) behind an age verification wall before they can access standard AI functionality.
- §1804(1): Requires operators to offer “at least one method” to determine whether a covered user is a covered minor. While framed as a single option, this still imposes a universal age verification obligation on all users of all “advanced chatbots” in the state.
These provisions are in direct tension with §1800(6).
Mandatory age verification also invites privacy and data security concerns as it requires operators to collect sensitive personal data about all users, including adults, creating significant privacy and security risks that may outweigh the protections the bill seeks to provide
California’s SB 243 uses an actual knowledge trigger for child-specific obligations without requiring universal age gating. New York’s own enacted GBL Article 47 similarly does not mandate age verification for all users. S.9051 should be consistent with both.
Liability Framework Is Disproportionate
Section 1802's liability framework is disproportionate.
First, it imposes strict liability for non-deceptive conduct; a chatbot that clearly discloses "I am an AI" at every interaction can still be sued for outputs that fall within the bill's expansive "unsafe features" definition. Liability should attach where deception actually occurred; a disclosure defense would not weaken child safety.
Second, the definition of "responsible party" in §1800(9) sweeps in upstream model developers and cloud infrastructure providers who have no direct relationship with end users and no ability to control operator-level compliance decisions. This amounts to liability without control, is contrary to basic tort principles, and will cause developers to restrict access to AI services in New York without any corresponding safety benefit.
Third, the bill contains no notice-and-cure mechanism, meaning a good-faith operator who inadvertently violates a vague provision faces immediate $25,000-per-violation penalties and private litigation with no opportunity to remediate.
AG Only Enforcement is Most Appropriate
The bill's private right of action compounds the above-mentioned liability concerns. Section 1802(1) allows any injured individual to seek punitive damages, disgorgement of profits, and attorneys' fees with no requirement of concrete, measurable injury, and pairs this with a rebuttable presumption of causation that effectively shifts the burden of proof to defendants in any case involving self-harm by a minor. As so often is the case, this combination will attract plaintiff's attorneys seeking settlements rather than consumers seeking genuine protection and will create unquantifiable litigation exposure for small edtech companies and non-profits — not only large commercial platforms.
The Attorney General enforcement mechanism already provided in §1802(2) is the more appropriate model.
***
The Business Council is committed to ensuring safe, beneficial, and age-appropriate digital environments for minors. But in doing so, we cannot ban minors from using digital technologies and innovations that will put them behind their peers nationally and internationally.
New York has an opportunity to be a model for responsible, targeted AI regulation. We encourage the Legislature to rethink its approach to ensure that New York kids are not left behind and that innovation is not stifled.