The novel and equally controversial bill — also known as Senate Bill 1047 — was at the center of months and months of debate before coming to an anticlimactic end on the governor’s desk Sept. 29.
“This veto leaves us with the troubling reality that companies aiming to create an extremely powerful technology face no binding restrictions from U.S. policymakers, particularly given Congress’ continuing paralysis around regulating the tech industry in any meaningful way,” the bill's author, state Sen. Scott Wiener (D-San Francisco), said in a written response to the governor’s veto.
Among the many goals outlined in the bill was an attempt to set requirements around training and auditing of large AI models under the threat of potential civil action from the state’s attorney general.
The reason Newsom gave for pulling the trigger on the legislation was its potential to curtail the innovation in the space while simultaneously failing to address gaps related to smaller AI models.
“While well-intentioned, SB 1047 does not take into account whether an Al system is deployed in high-risk environments (or) involves critical decision-making or the use of sensitive data,” the governor wrote in his veto decision. “Instead, the bill applies stringent standards to even the most basic functions — so long as a large system deploys it. I do not believe this is the best approach to protecting the public from real threats posed by the technology.”
Wiener denounced the governor’s implication that the bill was not based on empirical evidence.
“The governor’s veto message lists a range of criticisms of SB 1047: that the bill doesn’t go far enough, yet goes too far; that the risks are urgent but we must move with caution. SB 1047 was crafted by some of the leading AI minds on the planet, and any implication that it is not based in empirical evidence is patently absurd,” the senator wrote.
The governor’s veto is likely only a temporary setback for more comprehensive regulation in the rapidly evolving space, though exactly how long and what form it takes remains to be seen. Newsom said in his veto letter that he remains committed to working with lawmakers and experts to strike a balance between innovation and regulation.
California has been a national role model where AI and GenAI are concerned. In November 2023, the governor signed an executive order directing the state’s technology agencies to examine the risks and opportunities surrounding GenAI, which was followed by a handful of pilot projects to study real-world applications across as many agencies.
“Let me be clear — I agree with the author — we cannot afford to wait for a major catastrophe to occur before taking action to protect the public. California will not abandon its responsibility. Safety protocols must be adopted,” he continued in his veto letter. “Proactive guardrails should be implemented, and severe consequences for bad actors must be clear and enforceable.”
While Wiener’s bill didn’t make it through this session, the governor signed several others related to AI technology in the last month. They include (as written by the Governor’s Office):
- AB 1008 by Assemblymember Rebecca Bauer-Kahan (D-Orinda) — Clarifies that personal information under the California Consumer Privacy Act (CCPA) can exist in various formats, including information stored by AI systems (previously signed)
- AB 1831 by Assemblymember Marc Berman (D-Menlo Park) — Expands the scope of existing child pornography statutes to include matter that is digitally altered or generated by the use of AI
- AB 1836 by Assemblymember Rebecca Bauer-Kahan (D-Orinda) — Prohibits a person from producing, distributing or making available the digital replica of a deceased personality’s voice or likeness in an expressive audiovisual work or sound recording without prior consent, except as provided (previously signed)
- AB 2013 by Assemblymember Jacqui Irwin (D-Thousand Oaks) — Requires AI developers to post information on the data used to train the AI system or service on their websites (previously signed)
- AB 2355 by Assemblymember Wendy Carrillo (D-Los Angeles) — Requires committees that create, publish or distribute a political advertisement that contains any image, audio or video that is generated or substantially altered using AI to include a disclosure in the advertisement disclosing that the content has been so altered (previously signed)
- AB 2602 by Assemblymember Ash Kalra (D-San Jose) — Provides that an agreement for the performance of personal or professional services which contains a provision allowing for the use of a digital replica of an individual’s voice or likeness is unenforceable if it does not include a reasonably specific description of the intended uses of the replica and the individual is not represented by legal counsel or by a labor union, as specified (previously signed)
- AB 2655 by Assemblymember Marc Berman (D-Menlo Park) — Requires large online platforms with at least 1 million California users to remove materially deceptive and digitally modified or created content related to elections, or to label that content, during specified periods before and after an election, if the content is reported to the platform; provides for injunctive relief (previously signed)
- AB 2839 by Assemblymember Gail Pellerin (D-Santa Cruz) — Expands the time frame in which a committee or other entity is prohibited from knowingly distributing an advertisement or other election material containing deceptive AI-generated or manipulated content from 60 days to 120 days, amongst other things (previously signed)
- AB 2876 by Assemblymember Marc Berman (D-Menlo Park) — Require the Instructional Quality Commission (IQC) to consider AI literacy to be included in the mathematics, science and history-social science curriculum frameworks and instructional materials
- AB 2885 by Assemblymember Rebecca Bauer-Kahan (D-Orinda) — Establishes a uniform definition for AI in California law (previously signed)
- AB 3030 by Assemblymember Lisa Calderon (D-Whittier) — Requires specified health-care providers to disclose the use of GenAI when it is used to generate communications to a patient pertaining to patient clinical information (previously signed)
- SB 896 by Sen. Bill Dodd (D-Napa) — Requires the California Department of Technology (CDT) to update report for the governor as called for in Executive Order N-12-23, related to the procurement and use of GenAI by the state; requires the Governor's Office of Emergency Services (OES) to perform a risk analysis of potential threats posed by the use of GenAI to California’s critical infrastructure (with high-level summary to Legislature); and requires that the use of GenAI for state communications be disclosed
- SB 926 by Sen. Aisha Wahab (D-Silicon Valley) — Creates a new crime for a person to intentionally create and distribute any sexually explicit image of another identifiable person that was created in a manner that would cause a reasonable person to believe the image is an authentic image of the person depicted, under circumstances in which the person distributing the image knows or should know that distribution of the image will cause serious emotional distress, and the person depicted suffers that distress (previously signed)
- SB 942 by Sen. Josh Becker (D-Menlo Park) — Requires the developers of covered GenAI systems to both include provenance disclosures in the original content their systems produce and make tools available to identify GenAI content produced by their systems (previously signed)
- SB 981 by Sen. Aisha Wahab (D-Silicon Valley) — Requires social media platforms to establish a mechanism for reporting and removing “sexually explicit digital identity theft” (previously signed)
- SB 1120 by Sen. Josh Becker (D-Menlo Park) — Establishes requirements on health plans and insurers applicable to their use of AI for utilization review and utilization management decisions, including that the use of AI, algorithm or other software must be based upon a patient’s medical or other clinical history and individual clinical circumstances as presented by the requesting provider and not supplant health-care provider decision-making (previously signed)
- SB 1288 by Sen. Josh Becker (D-Menlo Park) — Requires the superintendent of public instruction to convene a working group for the purpose of exploring how AI and other forms of similarly advanced technology are currently being used in education (previously signed)
- SB 1381 by Sen. Aisha Wahab (D-Silicon Valley) — Expands the scope of existing child pornography statutes to include matter that is digitally altered or generated by the use of AI
