Utopian Promise, Dystopian Risk: Oversight Commission Tackles Government AI

The independent government oversight group known as the Little Hoover Commission heard testimony last week about the various kinds of artificial intelligence and how the state is responding to them.

Commissioners approached the topic with a mix of optimism and reluctance, one even pointing out that the two common viewpoints of the technology seem to be a “utopian Nirvana versus dystopian hellscape.”

Among the experts who testified was Government Operations Agency (GovOps) Secretary Amy Tong, who emphasized a holistic and strategic approach to the disruptive and pervasive technology.

“We want to have a responsible adoption of generative AI in the state of California,” she said. “... We’re trying to not follow the rush to say use the latest and greatest because it’s a fun thing to do. …”

Tong, who took over as GovOps secretary in February 2022 after having served as the state CIO and in other positions, told commissioners that there are several efforts underway to better understand and utilize AI and generative AI (GenAI) in state government. In September 2023, Gov. Gavin Newsom signed an executive order mandating careful study of the risks and benefits of AI, and in March, procurement guidelines were published, setting new standards for the tools and tech state agencies could deploy.

These efforts, Tong noted, are being watched by other states and the federal government — especially where procurement and related policy is concerned. The state’s considerable influence and purchasing power make it an obvious bellwether, she told commissioners.

The U.S. General Services Administration (GSA) — the federal equivalent of California’s Department of General Services (DGS) — has shown interest in working with the state in this area and is already using the state procurement guidelines to develop more detailed procurement standards on AI and GenAI, Tong told commissioners.

Another area of focus for GovOps is the touchpoints and impacts to the state’s workforce. Tong said the technology should be seen as a supplement to the some 250,000 state workers, rather than a potential replacement, citing the possibility of GenAI to assist human workers in areas like call centers.

While the input from commissioners was mostly complimentary of action being taken by the administration, there was some concern about the possibility of failed implementations and risks in areas like critical infrastructure.

Tong acknowledged the risks, but was quick to point out that several ongoing pilots are being carefully monitored and sandboxed to ensure that the systems are operating safely and as expected.

Six-month pilot projects are currently being conducted in partnership with Accenture, Deloitte Consulting, Ignyte Group, INRIX and SymSoft Solutions. The projects include technologies from OpenAI, Anthropic, Google and Meta, among others, to study health-care inspections, traffic safety, business tax processes and more.

Experts from C3.ai, Meta, Google, the Stanford Institute for Human-Centered Artificial Intelligence, and SEIU Local 1000 also testified.