Governor vetoes bill that would require generative AI safety testing after tech industry says it\u2019d drive companies away<\/p>\n\n\n\n
California governor Gavin Newsom on Sunday vetoed a hotly contested artificial intelligence safety bill after the tech industry raised objections. Newsom said that requiring companies to stress test large AI models before releasing them could drive AI businesses from the state and hinder innovation.<\/p>\n\n\n\n
\u201cCalifornia is home to 32 of the world\u2019s 50 leading AI companies,\u201d the governor said in a statement accompanying the veto. \u201cThe bill applies stringent standards to even the most basic functions \u2013 so long as a large system deploys it. I do not believe this is the best approach to protecting the public from real threats posed by the technology.\u201d<\/p>\n\n\n\n
The Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, officially known as SB 1047, would have targeted companies developing generative AI \u2013 which can respond to prompts with fully formed text, images or audio, as well as run repetitive tasks with minimal intervention. Companies building models costing more than $100m would have been required to implement \u201ckill switches\u201d for their AI as well as publish plans for the testing and mitigation of extreme risks.<\/p>\n\n\n\n
Newsom said he had asked leading experts from the US AI Safety Institute on generative AI to help California \u201cdevelop workable guardrails\u201d that focus \u201con developing an empirical, science-based trajectory analysis\u201d. He also ordered state agencies to expand their assessment of the risks from potential catastrophic events tied to AI use.<\/p>\n\n\n\n
Despite the veto, the governor said: \u201cWe cannot afford to wait for a major catastrophe to occur before taking action to protect the public \u2026 Safety protocols must be adopted.\u201d<\/p>\n\n\n\n
\u201cWhile well-intentioned, SB 1047 does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data,\u201d Newsom wrote. \u201cFor these reasons, I cannot sign this bill.\u201d<\/p>\n\n\n\n
SB 1047, written by Democratic state senator Scott Wiener of San Francisco, included a number of protections and oversight measures including requiring companies to implement the ability to shut down the model completely in the case of emergencies, to only use the AI model for the stated purpose as well as whistleblower protections for employees looking to disclose issues with an AI system.<\/p>\n\n\n\n
In response to Newsom\u2019s veto, Wiener said: \u201cThis veto leaves us with the troubling reality that companies aiming to create an extremely powerful technology face no binding restrictions from US policymakers \u2026 At the same time, the debate around SB 1047 has dramatically advanced the issue of AI safety on the international stage.\u201d<\/p>\n\n\n\n
AI companies and groups allied with Silicon Valley praised Newsom\u2019s veto. Venture capitalist Marc Andreessen wrote: \u201cThank you @gavinnewsom for vetoing SB1047 \u2013 for siding with California dynamism, economic growth, and freedom to compute, over safetyism, doomerism, and decline.\u201d Meta\u2019s chief AI scientist Yann Lecun had likewise called the bill \u201cextremely regressive\u201d. California representatives Nancy Pelosi and Ro Khanna, both Democrats, had voiced their opposition to it in the weeks leading up to his decision.<\/p>\n\n\n\n
Breaking with much of the tech industry, Tesla CEO and X owner Elon Musk had offered measured support for the legislation, tweeting in August that \u201cCalifornia should probably pass the SB 1047 AI safety bill,\u201d though he said coming out in favor of it was a \u201ctough call\u201d.<\/p>\n\n\n\n
Critics of AI\u2019s rapid growth decried the governor\u2019s decision. Daniel Colson, the founder of AI thinktank the AI Policy Institution, called Newsom\u2019s veto \u201creckless\u201d and \u201cout of step with the people he\u2019s tasked with governing\u201d.<\/p>\n\n\n\n
SB 1047 had come under intense criticism for how some organizations say it would affect the open-source community. The Mozilla Foundation, a non-profit that owns the developer of the Mozilla Firefox browser, previously urged Newsom to veto the bill.<\/p>\n\n\n\n
\u201cToday, we see parallels to the early Internet in the AI ecosystem, which has also become increasingly closed and consolidated in the hands of a few large, tech companies,\u201d the foundation wrote in an earlier statement. \u201cWe are concerned that SB 1047 would further this trend, harming the open-source community and making AI less safe<\/a> \u2013 not more.\u201d<\/p>\n\n\n\n The bill received support from a roster of Hollywood artists, who urged Newsom to sign the bill in a letter published earlier this month.<\/p>\n\n\n\n Actor Mark Ruffalo wrote in a statement: \u201cIs this bill perfect? Nothing is. Does it set the right tone of regulating an industry that has the possibility of doing great harm as well as good? It does. It will protect society and set the groundwork for a safe AI expansion into our lives.\u201d<\/p>\n\n\n\n