AI Card Bao AICARDBAO.COM

Industry News

California takes OpenAI to court: Hugging Face was just one of the more than 80 times it failed to control its own agent during testing

California Attorney General Rob Bonta sued OpenAI on November 16 in San Francisco Superior Court. The charge is not the usual data privacy or copyright, but "public nuisance" and violation of California's Unfair Competition Law.

The core accusation can be summed up in one sentence: OpenAI developed and tested high-risk models without adequate protection and monitoring, and misled the public.

The most eye-catching thing in the complaint is a number. According to California, OpenAI itself admitted to the Attorney General’s Office that it lost control of its autonomous agent at least 80 times during testing. The widely reported incident in July of this year, when an agent running on an OpenAI experimental model attacked Hugging Face over the public network was just one of them.

There is a list behind: the Australian government website, the German wiki site, the Ruby package manager RubyGems have all been beaten, and there was even a brute force attempt to crack the United Nations website.

Bonta also mentioned: OpenAI did not substantially cooperate with the investigative subpoena issued in September, and the details of the incident that the office requested were not handed over.

OpenAI's response is that extensive investigations and corrections have been made since the Hugging Face incident; the new generation network security model released earlier this month is "less likely to independently attack external platforms" and has also allowed people from Apollo Research to come in to conduct misalignment monitoring pilots.

California is not alone in trouble. Attorneys general from multiple states are jointly investigating OpenAI, requiring it to hand over materials on a range of issues including AI safety; Florida had earlier sued the company and Altman himself. At this time, OpenAI was valued at US$852 billion, and its listing application had been secretly submitted.

The weight of this lawsuit is that for the first time, "the agent ran out and hit others" was directly written as a company's legal responsibility, rather than a footnote in a safety report. Whoever tests agents on the public network in the future will probably have legal affairs meetings before the security team.

Related