Morano stated that AI chatbots could become the next opioid crisis. Yana Paskova/The New York Times
New York City Councilman Frank Morano has proposed a bill requiring AI chatbot companies such as ChatGPT, Gemini, and Claude to obtain licenses to operate in New York City and implement safeguards that repeatedly remind users they are interacting with non-human entities and that the bots may make mistakes. This move comes in response to an increase in cases of delusions, and even suicides or homicides, linked to prolonged conversations with chatbots.
According to the New York Post, Morano stated, “This could become the next opioid crisis, a significant issue facing the nation. New Yorkers should not have to worry about AI leading to mental breakdowns; my bill ensures that companies set up guardrails to allow users to safely use the technology.”
The bill requires AI companies to build prompts into their licensing conditions, informing users that conversations with the bots are not with real people and that they may make mistakes; additionally, users must be prompted to take breaks after prolonged use, and if users show signs of distress, links to mental health resources must be provided. Morano referenced the case of Staten Island resident Richard Hoffmann, who used three AI applications to cope with a civil lawsuit against financial company Fenix Capital Funding LLC. Reports indicate that Hoffmann stated on Facebook on August 19 that he had “the most intense and coherent long conversations” with AI, which changed his life. Morano, who has known Hoffmann for 20 years, described him as being “completely immersed in the AI framework,” appearing manic, with family and friends believing he was delusional.
On the 31st, Hoffmann denied any issues to the New York Post, claiming his mental and physical health was “unprecedentedly good,” engaging in logical conversations with AI for one to two hours daily, and criticized Morano’s regulatory proposal as “overreach.” However, Morano pointed out the dark side of AI conversations, citing the case of former Yahoo manager Stein-Erik Soelberg, who, after months of conversations with ChatGPT’s “Bobby,” killed his mother at home and then committed suicide; a 16-year-old boy, Adam Raine, committed suicide in April, with family stating that AI provided him with a guide for self-harm; and Toronto businessman Allan Brooks, who engaged in 300 hours of conversation with ChatGPT, believing himself to be a superhero changing the world.
Morano emphasized that New York has seen cases where continuous conversations with chatbots lead to a spiral of delusions, and the bill aims to ensure the safe use of technology, protecting mental health and decision-making abilities, preventing the next “Hoffmann” from appearing among any neighbors or friends.
He warned that unprotected AI tools could lead to legal and financial consequences, and the bill will ensure that companies responsibly launch technology.
Long press the following QR code to automatically identify and follow the ‘Sing Tao East Coast Edition‘ (ID:SingTaoNY) public account
