California’s Child Welfare Digital Services (CWDS) project has been working to replace the previous case management system with agile software that creates a suite of tools to help case workers.
One tool being considered — predictive analysis — was discussed Tuesday at CWDS' Noon Speaker Series by Emily Putnam-Hornstein, who is working with a team of researchers to harness data that social workers are required to collect anyway.
The Predictive Risk Modeling Project is a proof-of-concept research project that is studying an algorithm’s ability to quantify the likelihood that a child will be reported to child welfare authorities more than once over a two-year period.
“This is really a proof of concept, so there is nothing being piloted,” Putnam-Hornstein made clear.
Putnam-Hornstein’s team, the Children’s Data Network at the University of Southern California, is testing the “data analytics tool to help child-abuse investigators gauge the risk of maltreatment when a report of child abuse or neglect is made,” says a CWDS flier about the event.
The tool is a “possibility — not plan” cautioned Digital Services Director Kevin Gaines.
The project would not replace any part of the current welfare system, especially because it can capture only data, not "softer" information such as social workers' relationships or services. The system could indicate which children might need the most services over time. The concept is similar to using facial recognition in a camera, in that it can make a reasoned guess or prediction but is still controlled by a worker making the decisions.
“The thinking here was that workers … are pretty good at assessing the immediate needs of the child, what is far harder to do ..." she said. “What the algorithm is doing is trying to pull together data we have about this call plus all the prior interactions this family has had with the child-welfare system, to look out over the two years.”
The California version is modeled on a similar project in which child-welfare workers of Pennsylvania's Allegheny County are cross-referencing child data with other data, such as health records.
California’s research project is based on data from 2010-2014, giving "a full two-year window, where we could look at that data over that horizon," Putnam-Hornstein said. That information included discussions with Monterey, San Francisco and Los Angeles counties.
“We were very exploratory. We said what can we observe, in the administrative data, that reflects future system involvement,” Putnam-Hornstein said.
About 400 predictors were collected, “as conceptual blocks of predictors,” including demographic, accusation, historical data and past reports. Data is also included about when the calls come in and the people around the victim, including other siblings and caregivers. This led to more than 3.5 million observations, according to Putnam-Hornstein.
The model was then validated against itself but it has not been tested on live data, or any "forward-looking information."
The California team is still testing the algorithm, through regression analysis and other quantitative methods, and has no plans to use it yet.
But Putnam-Hornstein said she still has some questions — primarily: "How would this be eventually be used to improve and support decisions, and how do we know that this would actually make a difference?"
