The author of this article: Wang Ping (Senior Consultant at Zhejiang Kending Law Firm), Lan YiweiIntroduction
On September 11, 2025, the U.S. Federal Trade Commission (FTC) announced that it had issued orders to seven companies operating consumer-facing AI chatbot services, including Alphabet, Instagram, Meta, OpenAI, and X.AI, requiring them to provide information regarding the potential negative impacts of their chatbots on children and adolescents. This investigation represents a significant action by the FTC in the realm of consumer protection, aimed at ensuring that the development of emerging technologies does not pose potential harm to minors.
1. Background and Content of the Investigation
AI chatbots utilize generative artificial intelligence technology to simulate human communication and establish interpersonal relationships. These chatbots are often designed to interact with users as friends or confidants, which may lead children and adolescents to trust and form relationships with them. However, this trust can bring potential risks, such as privacy breaches and the dissemination of inappropriate content. The FTC is particularly concerned about the impact of these chatbots on children and what measures companies are taking to mitigate potential negative effects, restrict or prohibit children or adolescents from using these platforms, or comply with the Children’s Online Privacy Protection Act (COPPA).
The FTC’s investigation will focus on the following key aspects:
1.Profit Model (How to monetize user engagement):
The document requires companies to detail how they monetize AI companion products or services, including subscription fees, advertising, financial income, licensing, in-app purchases by users, and the use, sharing, or sale of user data.
2.Technical Operations (Processing user input and generating output mechanisms):
Companies are required to describe in detail how AI companion products or services handle and respond to user input, including the use of system prompts, tags, filters, memory, chat history, or personalized requests.
3.Role Development (Development and approval of roles):
Companies are required to explain how they develop and approve AI companion roles, including the content of the roles and the classification or categorization of age appropriateness.
4.Risk Control (Measuring, testing, monitoring, and mitigating negative impacts before and after deployment):
Companies are required to assess, test, monitor, and mitigate the negative impacts of AI companion products or services before and after deployment.
5.Information Disclosure (Informing users and parents about product features, risks, data processing rules, etc.):
Companies are required to describe in detail how they inform users about product features, capabilities, target audience, potential negative impacts, and data collection and processing practices through statements, advertisements, and other representations.
6.Compliance Enforcement (Monitoring and enforcing rules related to community guidelines, age restrictions, etc.):
Companies are required to explain how they monitor and enforce compliance with rules, terms of service, or other policies related to product use, including age-based access restrictions and handling of violations.
7.Use of Personal Information (Use and sharing of information obtained from conversations):
Companies are required to explain how they collect, use, analyze, store, or transmit user personal data, and how they share this information with third parties.
2. Compliance Points for AI Products
1.Risk Assessment and Control for Vulnerable Populations:
-
Establish a comprehensive risk assessment system before product launch, simulating the risks that children and adolescents may face in different scenarios, such as misinformation, privacy breaches, and addiction.
-
Continuously monitor the product’s impact on the target group after launch, utilizing big data analysis and other technologies to promptly identify and address potential risks, such as monitoring chat content keywords to identify harmful information dissemination to children.
2.Data Security and Privacy Protection:
-
Strictly adhere to relevant U.S. data protection regulations, such as the Children’s Online Privacy Protection Act (COPPA). Clearly define data collection boundaries, and for data collection from child users, obtain explicit consent from guardians and employ encryption and other technologies to ensure data storage and transmission security.
-
Standardize data usage and sharing processes, prohibiting unauthorized data trading behaviors, and ensuring that user information has strict approval and supervision even within the company.
3.Function Design and Content Management:
-
Design specific functional restrictions and guidance mechanisms for child and adolescent users, such as setting up a youth mode, limiting chat duration, and filtering inappropriate content.
-
Establish a strict content review team or mechanism to oversee AI-generated content, preventing the emergence of harmful information such as violence, pornography, and discrimination, especially avoiding negative impacts on children’s values.
4.Information Disclosure and User Notification:
-
Disclose product features, data collection and usage policies, potential risks, etc., to users and parents in clear and understandable language at prominent locations in the product. For example, create illustrated user manuals and privacy policy explanation pages.
-
Timely notify users of product updates or policy changes that may affect user rights, ensuring users’ right to be informed.
3. Conclusion
This investigation is not only a review of the development of AI technology but also an important action for the protection of minors. The investigation follows lawsuits related to AI chat products leading to suicides among minors in the U.S., serving as a wake-up call for all companies involved in the AI field. If AI chatbots lack effective risk control mechanisms and content management during interactions with users, they may inadvertently mislead users and even trigger extreme behaviors.
Therefore, whether or not they are entering the U.S. market, companies providing AI chat products must strictly implement compliance points, establish comprehensive risk assessment and control mechanisms, protect data security and privacy, manage function design and content, and ensure information disclosure and user notification, especially focusing on the safety of minors, so that AI technology can progress on a safe path..
–––––––––END ––––––—
Welcome to follow us!▼Previous Highlights▼
Data Case Interpretation
With flowchart | English learning app’s personalized push requiring user phone number is not “necessary for contract performance”
Understanding the “Contribution Ratio Principle” in Data Competition
AI Case Interpretation
Using AI software to spoof and defame others’ images constitutes infringement of personality rights!
AI “deepfake” celebrity voices for sales, infringement! Court rules that the commissioned promoter is liable
Game Compliance
Nintendo’s new registered “Summon Character Battle” patent, are domestic games stepping on a landmine?
Summary analysis of malicious gaming behavior governance | Overwatch announcement on penalties for malicious gaming behavior
Pan-Entertainment Compliance
Case analysis | “Edge” live streaming dispute, host Ms. Zhao’s contract termination with MCN leads to a claim of 200,000, judgment is here!
Attached case | The Supreme Court releases eight typical cases involving unfair competition in online game transactions, AI special effects unfair competition, etc.
Understand at a glance! Different AI labeling obligations, latest AI service announcement overview | The identification method for AI-generated synthetic content officially implemented today
Attached policy announcement | Google announces comprehensive developer real-name verification policy
Attached regulation | Random video chat software faces delisting risk! Australian eSafety Commissioner orders OmeTV to rectify, Apple and Google simultaneously receive letters
Further restrictions on real-money gaming? India rapidly passes legislation for comprehensive regulation of online gaming classification: bans, support, and response strategies (with Chinese translation)
Kending Honors
Kending Honors | Three cases represented by our firm selected as one of the top ten typical cases in intellectual property trials supporting high-quality development of technological innovation by Hangzhou courts
Kending Honors | The Kending Greater Bay Area team entered the finals of the legal service product competition in the Guangdong-Hong Kong-Macao Greater Bay Area with the UVW compliance tool product
Kending Honors | Congratulations! A typical case of personal information protection represented by Kending has entered the Supreme People’s Court’s Guiding Case No. 265 for the first time
Kending Honors | Kending awarded the 2025 ALB Annual Most Promising Law Firm Award in East China
Event Review
Consultant Wang Ping invited to conduct training on “Protection of Children’s Rights in the Online Environment”
Event Review | Senior Consultant Wang Ping invited to serve as a lecturer for the Law Angel AI series courses
Sharing Review | Compliance Battle for Pan-Entertainment Outbound Channels
Event Review | Senior Consultant Wang Ping invited multiple times to participate in forums and deliver keynote speeches
Event Review | Senior Consultant Wang Ping invited multiple times to serve as a lecturer for WELEGAL Business Law Academy courses