OpenAI, Google, Microsoft among 13 AI companies told to fix ‘harmful’ AI behaviour or face action

OpenAI, Google, Microsoft are among 13 top artificial intelligence companies have been given a warning over "delusional outputs" from their chatbots by a group of bipartisan state attorney generals. In a letter made public on Wednesday, dozens of state Attorney genrals expressed serious concerns about the rise of ‘sycophantic and delusional’ outputs from AI chatbots.

The AGs have also warned the tech giants to add additional safeguards to protect children from the sycophantic and delusional outputs from their AI chatbots by 16 January, 2026. They also warn that support for innovations is not “an excuse for noncompliance with our laws, misinforming parents, and endangering our residents, particularly children.”

They references several media reports about AI chatbots going haywire including cases where the AI helped teens plan their own suicide or encouraged their delusions. The AGs also cited reports of chatbots inducing "AI psychosis," where the AI amplifies a user's paranoia or delusions.

"GenAI products generated sycophantic and delusional outputs that either encouraged users’ delusions or assured users that they were not delusional." the AGs noted

In the letter, the AGs also noted that some of the conversatoions by these chatbots could be going against some of the state laws like encouraging illegal activity or practicing medicine without a license.

They also warn about the ‘dark patterns’ with the AI chatbots which includes anthropomorphization, harmful content generation, and manipulating users to increase retention.

"Many of our states have robust criminal codes that may prohibit some of these conversations that GenAI is currently having with users, for which developers may be held accountable for the outputs of their GenAI products" they further added

What are AGs demanding?

The AGs are demanding that the AI companies must outline the specific guardrails they have implemented or plan to implement to cut out the concerning sycophantic and delusional behaviour form their chatbots.

The letter also warns the tech giants to implement a ‘clear and conspicuous warning’ that is ‘permanently viewable on the same screen’ concerning the potentially harmful outputs that may be generated by the generative AI chatbot.

The letter by the AGs is addressed to 13 AI companies which inclues Anthropic, Apple, Chai AI, Character Technologies, Google, Luka, Meta, Microsoft, Nomi AI, OpenAI, Perplexity AI, Replika, and xAI