Senators, Expert Witnesses Compare Notes on AI Safety Concerns

Few would argue with the idea that California has been leading the charge on commonsense guardrails on the technology that some believe will eventually kill us all.

Both sentiments were on full display Wednesday morning during a state Senate Privacy, Digital Technologies and Consumer Protection Committee hearing titled "Understanding the Contours of the AI Threatscape."

In his opening remarks, Committee Chair Sen. Christopher Cabaldon called out several concerns in this space; chief among them was a broad lack of preparedness for future model iterations and new capabilities.

“Today we're here because California has led the way with respect to artificial intelligence regulation and governance in many, many ways, and it is wholly insufficient to meet the moment that is now before us,” the senator said.

The nearly three-hour hearing highlighted the pace of the technology's evolution, with experts testifying that the coming years will see what is called “recursive self-improvement” — when AI models are able to build new and improved versions of themselves without human help.

“So, just as a matter of timeline, if we're thinking about recursive self-improvement happening within a time frame of roughly two years, we're already seeing superhuman capabilities in a number of domains,” former OpenAI Chief Futurist Josh Achiam said. “We're seeing the potential for impacts on all of the theoretical domains that can happen on a computer and many of the areas in the physical world.”

That rapid expansion of capabilities will, by 2030, pose an outsized challenge for lawmakers and regulators as they try to balance safety and security with the new realities the technology creates. This, of course, is all set against the backdrop of growing social discord.

“It won't just be one thing to pay attention to; it'll be a very wide range of things at once,” Achiam added.

Complicating the whole issue of regulation and safety is the “black box” companies work in with regard to their models. Researchers are often at the mercy of the major players, who offer a glimpse behind the curtain, but never a full picture.

Model Evaluation and Threat Research (METR) AI Research Scientist Rif Saurous noted that there is a delicate balance at play between his organization and the AI labs creating the models. Their relationships are voluntary, and companies are able to redact information at will.

”Overall, these kinds of assessments are designed to surface scientific evidence of loss of control risks. They are not no-go decisions; they are not guarantees this is safe; they are not audits. It's really up to companies, governments and the public to decide how to act on the evidence,” Saurous explained.

In addition to the numerous cyber-related threats posed by artificial intelligence, there are other potential harms expert witnesses highlighted. Stanford Privacy and Data Fellow Jennifer King pointed to increased data privacy risks, potential for consumer harms, as well psychological and emotional harms.

King added that these risks are compounded by companies rushing to win, rather than iterating carefully around potential risks.

“The Silicon Valley startup culture is just not a culture of safety whatsoever. And I can tell you that having worked on the ground there … 20 years ago, working specifically on trust and safety issues. This is not an industry that rewards safety at all,” King said.

For all the discussion about these risks, Cabaldon closed the hearing by saying that a “comprehensive” and “substantial” package of legislation is being worked on for introduction in the upcoming session.