A first of its kind AI bill is winding its way through California, causing infighting between groups of AI pioneers.
The AI safety bill SB 1047 would put more responsibility on any developer spending more than $100 million to build an AI model. The requirements include: safety testing, implementing safeguards, and allowing the state attorney general to take action against the developer of any AI model that causes “severe harm,” such as mass casualties or incidents causing $500 million or more in damages.
Companies must agree to third-party audits and implementing a kill switch that can turn off the technology at any time. The bill also proposes protections for whistleblowers.
California state Senator Scott Wiener, co-author of the bill, accused some opponents of “fear mongering” to try and stop the state legislation from being passed.
“There’s been a lot of drama and dramatic statements and misstatements by some of the opposition,” Wiener said. “The large labs have committed repeatedly and vociferously to perform safety evaluations on their large models and that’s what this bill requires. Taking voluntary commitment and making it mandatory.”
The bill passed the state Senate and is scheduled for a vote in the State Assembly by the end of this week. It’ll return for a final vote in the Senate since it’s been amended multiple times. If passed it'll go to Governor Gavin Newsom for signing. Newsom hasn't signaled which way he’s leaning.
Wiener, who represents San Francisco, held a virtual press conference on Wednesday with supporters of the bill, including Yoshua Bengio, known as one of the “godfathers of AI,” Center for AI Safety director Dan Hendrycks, Economic Security California director Terri Oll, Encode Justice's vice president of political affairs Sunny Gandhi, and AI startup Ava's CEO Thibault Duchemin.
Wiener called SB 1047 “reasonable” and “light-touch”, and referenced Meta ( META )’s AI model Llama. "Meta has already committed to doing this testing," he claimed, so this bill should not cause the company to stop open sourcing the model.
"We’re working very hard to make the case to my colleagues in the assembly that this bill is worthy of supporting, we certainly have a path to pass it on the assembly floor," Wiener told Yahoo Finance. "The governor has made statements that are very consistent with my own thinking — that regulation could be appropriate here and we want to make sure we’re fostering innovation as well.”
Last week Anthropic wrote an open letter to Governor Newsom, showing cautious support for the amended bill. In the letter, the company acknowledged the changes and said the bill was "substantially improved, to the point where we believe its benefits likely outweigh its costs." Anthropic cofounder Jack Clark shared the letter on X, noting that it “isn’t an endorsement.”
The amended bill removed the creation of a "Frontier Model Division" to police frontier models. It also dropped criminal perjury for lying about the models and will instead rely on existing laws.
In a letter to Wiener, OpenAI’s chief strategy officer Jason Kwon wrote that while it supports some of the bill's provisions, AI regulations should be left for the federal government to decide on.
“A federally-driven set of AI policies, rather than a patchwork of state laws, will foster innovation and position the US to lead the development of global standards," wrote Kwon.
But two former OpenAI researchers, Daniel Kokotajlo and William Saunders, said they’re disappointed in OpenAI’s decision to oppose it. "Sam Altman, our former boss, has repeatedly called for AI regulation," they wrote in a letter to Gov. Newsom last week. "Now, when actual regulation is on the table, he opposes it."
Google ( GOOG , GOOGL ), Meta, and Andreessen Horowitz are also among the big names who call the bill a threat to innovation and research. Some fear it will put California at a disadvantage and possibly risk losing AI companies based in Silicon Valley.
SB1047 is also facing criticism from prominent figures in the industry. Dr. Fei-Fei Li, known as the “Godmother of AI” and the co-director of the Stanford Institute for Human-Centered Artificial Intelligence, wrote in Fortune earlier this month that the bill’s penalties and restrictions would have “unintended consequences” on innovation.
Li said SB 1047 “will harm our budding AI ecosystem, especially the parts of it that are already at a disadvantage to today’s tech giants: the public sector, academia, and ‘little tech.”
Wiener is facing opposition from some of his Democratic colleagues as well. Former House Speaker Nancy Pelosi criticized the bill , calling it “well-intentioned but ill-informed.” And eight California House members: Ro Khanna, Zoe Lofgren, Anna G. Eshoo, Scott Peters, Tony Cárdenas, Ami Bera, Nanette Diaz Barragan and Lou Correa are urging Newsom to veto the measure.
Wiener said he would welcome the federal government to preempt SB 1047 with strong AI safety regulation. “Congress has been completely paralyzed when it comes to technology policy,” Wiener said. “There are good people in Congress working to do so but the institution as a whole has not acted in the way it needs to act."
Lawmakers in California have introduced 50 AI-related measures aimed at regulating the emerging technology. There’s currently no federal law that sets guardrails for AI developers. If passed and signed into law, SB 1047 could become the first real piece of AI regulation in the country.
Yasmin Khorram is a Senior Reporter at Yahoo Finance. Follow Yasmin on Twitter/X @YasminKhorram and on LinkedIn . Send newsworthy tips to Yasmin: yasmin.khorram@yahooinc.com