Little Hoover: Progress, Remaining Perils Central to AI Testimony

California is long past the point of no return with AI and generative systems. The focus now, for technologists, policymakers and worker groups is how to maximize the benefit while watering down the damage.

During a hearing hosted by the Little Hoover Commission Thursday, officials talked about this work in relation to 2024 commission recommendations and what still needs to be done.

From the public policy perspective, state Sen. Jerry McNerney, D-District 5, called out concerns about the federal government’s ability to adopt a holistic national regulatory framework, adding that California has to take up that mantle in the interim.

“There’s a lot that can be done and has to be done in the fairly short term because the risk is there,” he said.

While the evolving technology promises efficiencies and benefits across multiple economic sectors, McNerney said those benefits will be tempered with the sort of job displacements that fuel social unrest and political upheavals.

“We know where we are; we are a little behind the eight ball, in my opinion, but what do we do to get on top of that?” he said.

To combat the issues of bioterrorism and cyber threats, the senator underscored the need for humans in the loop as well as kill switches to stop any model that might go off script.

“There’s a lot of challenge. There are some members of the Legislature who are up to that challenge, and we need to be let loose so we can actually make some progress,” he said, adding that passing legislation that codifies CalCompute could be useful in “squeezing” money out of tech firms to fund it.

Amy Tong, former adviser to Gov. Gavin Newsom as well as former state CIO and secretary of the Government Operations Agency, testified that work on AI predated the governor’s 2023 order by several years, but noted that the executive order gave that work the legs it needed to explore the technology in a controlled way.

With relation to the state’s workforce, Tong noted that the technology is being seen as a way to increase employee capacity rather than cut the workforce.

Commissioner David Beier pointed out the disparity between English second-language households, 45 percent of California households, and the state’s public-facing bilingual workforce, somewhere between 12 and 13 percent. Tong agreed that areas like this are prime for AI innovation and have already been the focus of department pilots.

“It’s very evident that the conversation about AI is still at too much of an abstract level,” Tong said, adding that the discussion needs to shift to be more tangible and focused on technology application.

“What we don’t want to repeat is to have this technology out there and everybody adapts on their own as opposed to a very concerted effort to train people step by step and then give them a safe space to use it …,” Tong said.

State CIO and California Department of Technology Director Chris Given offered his perspective, noting that the commission’s recommendation around empowering state workers to guide adoption and use is a priority of his.

“I believe that the most impactful applications of AI are not going to come from the top down, but rather from the bottom up. How might we enable the workforce to use these tools to solve their own problems, bringing to bear the firsthand experience that only they possess,” Given said.

So far, the California Department of Human Resources (CalHR) and CDT have partnered to train 165 departments and more than 15,000 employees on available tools like Poppy, the CIO said.

He also highlighted the department’s work to streamline AI procurement — namely by shifting focus from general risk assessment to areas of highest risk in consultation with other departments — and honing the use of shared services contracts that better leverage the state’s buying power across all departments.

The CIO said that while there is progress being made to secure state systems for the increasing risks posed by generative AI, there is more work to be done. The state is leveraging AI to mitigate risks wherever possible.

“The things we’ve got to get done to defend state systems and assets, they’re no different than the things everyone should have been doing last year or the year before, but there is no longer any room to kick the can,” Given said.

Earlier this month, CDT released its updated Cal-Secure 2.0 security plan and has also been tasked by the governor with creating an AI Cyber Defense Program.

While technologists took a positive tone around the progress made in AI training and workforce development, SEIU Local 1000 President Anica Walls said not enough has been done to include union members in AI-related conversations.

“We’re not against the AI sandbox; we’ve just got to figure out how and where we’re going to intentionally allow workers’ voices to be in that conversation and play nicely in that sandbox,” Walls said.