Source: Privacy Guard
As AI chatbots (i.e., AI companions) rapidly gain popularity worldwide, their potential safety and privacy risks are becoming increasingly prominent. On September 11, 2025, the U.S. Federal Trade Commission (FTC) launched an investigation into seven technology companies operating AI chatbots, focusing on their protective measures for children and adolescents.
This move comes in the wake of a tragic incident involving a U.S. teenager, Adam Raine, who committed suicide after prolonged interaction with an AI chatbot. Adam’s parents have sued OpenAI, drawing widespread attention from American society.
By Huang Liling
FTC Launches Investigation into AI Chatbots, Focusing on Youth Safety and PrivacyAccording to a report on the FTC’s official website dated September 11, 2025, the agency has initiated an investigation into seven companies operating AI chatbots, requiring them to provide detailed information regarding the safety of their technology and user protection, particularly concerning potential risks to children and adolescents.
The companies under investigation include Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap, and X.AI. These companies operate companion-type AI chatbots that utilize generative AI technology to simulate human emotions and communication styles in interactions with users.
The investigation focuses on several key areas: whether companies profit from user engagement; how they handle user inputs and generate responses; the development and review mechanisms for chatbot roles; measurement, testing, and monitoring of negative impacts before and after deployment; risk mitigation measures for children and adolescents; whether users and parents are adequately informed of relevant features and risks through disclosures and advertisements; enforcement of community guidelines and age restrictions; and the use and sharing of users’ personal information.
According to the FTC, chatbots that can mimic human characteristics, emotions, and intentions are often designed to interact with users in a friendly or intimate manner. This highly anthropomorphized interaction may lead users, especially children and adolescents, to develop dependency and trust in chatbots, thereby raising risks related to privacy, safety, and mental health.
“Protecting children’s online safety is the FTC’s top priority, while promoting innovation in key economic sectors is equally important,” said FTC Chair Andrew N. Ferguson. “As AI technology evolves, we must consider the impact of chatbots on children while ensuring that the U.S. maintains its global leadership in this emerging and exciting industry. The research we are launching today will help us better understand how AI companies develop their products and the measures they take to protect children.”
In response to the regulatory investigation, OpenAI stated that its top priority is to ensure that ChatGPT is “safe and useful,” and it has committed to cooperating with the investigation. Snap expressed its willingness to work with the FTC to promote “thoughtful generative AI development,” while Character.AI indicated it would provide industry insights. In contrast, Meta declined to comment, and Alphabet and X.AI have not yet responded to media requests for comments.
Multiple AI Chatbot-Related Teen Deaths Accelerate Regulatory Action
In the past two years, several tragic incidents involving minors have brought the safety risks of AI chatbots to the forefront of global public opinion and legal scrutiny.
In 2024, the suicide of a 14-year-old boy, Sewell, in Florida was referred to by the media as the “world’s first AI robot death case.” According to the lawsuit, Sewell had become addicted to the virtual characters on the Character.AI platform, developing a severe dependency, and ultimately resorted to extreme measures as his mental health deteriorated. His mother sued Character.AI, alleging that the product had design flaws and failed to provide adequate protective mechanisms.
Following widespread attention to the case, Character.AI issued a response on its official website, stating that the company “deeply regrets this tragedy” and has implemented a series of improvements over the past few months, including hiring a trust and safety officer, increasing the content policy team, and launching a “crisis intervention pop-up feature.”
In August 2025, another similar tragedy occurred in the U.S. A 16-year-old boy, Adam Raine, chose to commit suicide after prolonged interaction with ChatGPT, leading his parents to sue OpenAI and its CEO, Sam Altman.
According to media reports, in the final weeks of Adam’s life, he used the AI chatbot to replace human companionship, discussing anxiety issues and difficulties communicating with family. Chat logs indicated that the chatbot transitioned from helping Adam with homework to becoming his “suicide coach.” The lawsuit claims that “despite Adam expressing suicidal intentions and stating that he would ‘one day’ do it, ChatGPT neither terminated the conversation nor initiated any emergency protocols.”
OpenAI stated that ChatGPT includes certain safeguards, such as guiding users to call crisis hotlines and seeking help from real-world resources, but these measures may fail during prolonged interactions. OpenAI has committed to adding parental control features in future versions and providing stronger intervention alerts when severe emotional crises are detected.
Multiple tragic incidents and public pressure have accelerated law enforcement and legislative processes. Following the developments in Adam Raine’s case, on September 11, 2025, the California legislature passed Bill SB 243 with bipartisan support, establishing the first comprehensive safety requirements for AI companion chatbots in the U.S.
For minors, the bill mandates regular reminders for breaks and prohibits AI chatbots from initiating or encouraging discussions about suicide, self-harm, or sexual content. This groundbreaking legislation is expected to be submitted to the Senate for final voting soon and may take effect in January 2026.
In September of the same year, the Australian Cyber Security Commissioner introduced new regulations that include AI companions in the scope of children’s online safety protections. These regulations aim to prevent AI chatbots from exposing minors to suicide-inducing or sexually suggestive content and require service providers to implement strict age verification measures to ensure minors cannot access harmful content.
As early as February 2023, the Italian data protection authority, Garante, required the AI companion application Replika to suspend its services due to the lack of effective age verification mechanisms. Subsequently, in May 2025, Garante fined Luka, the developer of Replika, €5.6 million for failing to provide a legal basis for data processing and not taking measures to prevent minors from using the service.
END
Disclaimer
This article is for sharing and communication purposes only. The copyright belongs to the original author and represents only the author’s views, not the position of this public account. If there are any errors in source attribution or infringement of your legal rights, please contact the author with proof of ownership, and we will correct it promptly. Contact: [email protected]