IE11 Not Supported

For optimal browsing, we recommend Chrome, Firefox or Safari browsers.

Infrastructure, Operations Chief: ‘Embrace Change and Work Through It With Tools ... ’

II23-Quote-Carlson-940x490.jpg
As part of Industry Insider — California’s ongoing efforts to educate readers on state agencies, their IT plans and initiatives, here’s the latest in our periodic series of interviews with departmental IT leaders.

A longtime state staffer, Cheryl Carlson is infrastructure and operations branch chief at the California Department of Child Support Services (DCSS), a role she has had for about one year. Collectively, she has more than 20 years’ experience in technology that includes more than five years as a consultant to state departments and more than 15 years as a state employee. Her first state employee role dates to 2007, when she joined the Franchise Tax Board to work on its California Child Support Automation System project; before joining DCSS, Carlson worked for about three-and-a-half years at the California State Lottery.

Her education includes a management information systems degree and a master’s in organizational management, both from California State University, Sacramento.

Industry Insider — California: Take me through your time in the public sector. How long have you been at the state, how long have you been at the Department of Child Support Services and how long have you been in your current role?

Carlson: I have over 20 years of experience in the technology field, working over five years as a consultant to state departments and over 15 years as a state employee. My education background is a management information systems degree and a master’s degree in organizational management. I started working with the state as a student assistant for the California Department of Transportation and, later, the California Department of Justice. Out of college, I worked for Orion Scientific Systems which later became SRA International Inc. I provided consulting services for almost six years to various local, state and federal entities. I became a state employee in 2007 with the Franchise Tax Board on the California Child Support Automation System (CCSAS) project. In 2009, the project team merged into the Department of Child Support Services, where I remained until 2015 when I had an opportunity to work for the California Lottery. I worked there for about three-and-a-half years before returning to DCSS in 2018. Shortly after returning to DCSS, I became the project manager for the CSE Cloud Migration Project. I have been in the role of the infrastructure and operations branch chief for approximately one year.

IICA: How would you describe your current role, infrastructure and operations branch chief at DCSS, in terms of responsibilities and duties? How large is your current team?

Carlson: My team consists of 97 positions. As the branch chief, I look to provide reliable and secure services to support our customers using cross-functional and multidiscipline teams. I have full responsibility and oversight of the DCSS infrastructure including providing goals, strategies, visioning, budgets and plans to implement processes used for service delivery and management. This includes the areas of network operations, development operations, both Windows and Linux system administration, systems operations including database administration and system performance monitoring and workload automation, and the technology service desk that includes service e-management, asset management, and the statewide and IT service desks. Our goal is to continuously strengthen and secure the infrastructure to protect the organization. To support the mission, my team provides the foundational consistency, support and structure for the technology within DCSS. Integrity, commitment, consistency and collaboration are the keys to success. Using agile methodologies and processes, we build upon incremental successes to create a strong, sturdy and reliable infrastructure.

IICA: What big IT initiatives or projects are you working on currently? What sorts of projects might be coming your way in 2023?

Carlson: It is an exciting time to be working at DCSS. With the child support enforcement (CSE) application in the Microsoft Azure cloud, we are continuing to mature our cloud capabilities as well as continue to drive a cloud-first strategy. We will continue to optimize and automate where possible to improve day-to-day efficiencies, with the goal of obtaining more time to focus on supporting the business as much as possible. For example, we recently took advantage of other disaster recovery (DR) features offered in the cloud by successfully implementing a geographically redundant DR framework. Next, we will be building upon the success of DR and working to implement high availability (HA) not only for near-instantaneous recovery capabilities, but also to reduce downtime during software upgrades.

IICA: You accepted a CIO Leadership Award at last month’s California Public Sector CIO Academy. Can you share a bit about the work that led to you receiving that award, the needs it resolved, its goals and your role in it?

Carlson: As part of statewide oversight and reporting (State Administrative Manual (SAM) 5325 and the Statewide Information Management Manual (SIMM) 5325-A), DCSS is required to have a documented technology recovery plan. Prior to being in the Microsoft Azure cloud, DCSS was faced with a minimum of three to six weeks for recovery of CSE to get back to business operations and serve the public. When DCSS moved into the MS Azure cloud in November of 2021, it now had the infrastructure capable to successfully recover from a disaster in less than two weeks, but it had not been fully built or proven and that was not good enough for DCSS or our customers! The DR project kicked off after the system stabilization period post-migration in the summer of 2022 and completed in December. Implementation included an updated technology recovery plan and documented standard operating procedures to ensure sustainable practices as well as increased cloud expertise for staff, live simulation and analysis and a fully tested DR plan to validate the process and infrastructure. Although implementing a DR framework may not seem innovative at the surface, DCSS is one of the first child support programs not only to use cloud infrastructure, but is also the first to have a working DR that has been successfully proven with a return-to-operation in less than 72 hours! We are prepared for a major disaster, should it occur. With this, the DCSS mission and vision can continue to be fulfilled. Continuing the leadership demonstrated as the project manager of the cloud migration project, I was also given the opportunity to be the project manager of the DR project. I am grateful for the team, our partners, and for the support of our director and chief information officer.

IICA: What best practices might you offer as a result of having completed that work?

Carlson: There are two best practices that were effective when influencing major change within the organization when migrating to the cloud and creating the disaster recovery framework.

  • Cross-Functional Collaborative Communication – Having a great team is critical to successful change, and we have some of the best teams at DCSS. It is important to not only ensure that the staff participating in the project are excited and motivated, but it is also important to ensure your peers, your customers and your executives are equally excited. By creating cross-functional teams, each area was represented and had a voice in the project outcome. With this ownership, all are invested in the overall success of the project. Customers were kept informed with monthly progress presentations, weekly status calls and daily meetings post-implementation to allow them to stay informed and be part of the process. With the support of the executive team, it was important to inform them of the attention and resources needed for the project to be successful. With the cloud migration project listed as the No. 1 project within the organization by the director and the CIO, there was grace and understanding alongside grit and hard work that motivated all to work together toward a successful outcome.
  • Embrace the Unknown Discoveries When Project Planning – This is especially important with high-priority projects. It is vital to manage expectations and let stakeholders know that with significant change, such as a migration of a mission-critical system to the cloud, we may not know everything, and unforeseen issues can occur. But with a safe space to explore and identify those discoveries, issues can be identified early, before go-live, which in the end is cheaper and easier to course-correct than after the changes are in production. Plans are not meant to be perfect. However, they are meant to establish an expectation. Embrace change and work through it with tools such as agile reassessment, communication and collaboration to manage expectations.
IICA: In your time at DCSS – or at another state entity – what IT project or achievement are you most proud of?

Carlson: I am very proud to be working at DCSS. The people have so much expertise and heart, they are amazing. This combined with the confidence of innovation from the cloud migration and DR projects really makes DCSS a great place to work. We are ready to take on the next chapter. As one of California’s largest applications and the country’s largest child support enforcement system, with 1.2 million customers, 7,000-plus child support professionals, and millions of children who rely on child support, the CSE cloud migration was a huge undertaking. DCSS had never migrated an application so large, but migration was needed because the Y2K-era AIX/UNIX OS was costly, inflexible to change, slow to recover in a disaster, and required security improvements. In the past, the system would take weeks for disaster recovery, and it was unresponsive to business needs, requiring excessive time to plan maintenance or improvements, and unable to take advantage of the cloud’s self-service on-demand model. Moving CSE to the Microsoft Azure government cloud has benefited the department. In the cloud, DCSS has the capability to take advantage of innovative technologies that will have direct impacts to the organization. Not only did the organization experience immediate impacts such as an improved security framework, increased employee expertise, cost savings and a disaster recovery framework, it also provided the capability for future modernization such as automation, business process improvements and business intelligence. Even with the multitude of challenges, DCSS was able to successfully modernize the CSE infrastructure and provide the business with impacts and benefits in the areas of:

  • Improved ability to serve the needs of California children and families through the adoption of new tools and processes, resulting in faster payment processing
  • On-demand cloud environments to support development and training needs
  • Lowered risk of operating in an unsupported environment
  • Prepared to leverage the latest in cloud technology and roll out updates seamlessly
  • Elimination of technical debt and environment management autonomy
  • Reliable and available infrastructure needs for a mission-critical application (confidence with Microsoft technologies and expertise)
  • Extend AI to edge devices by running machine learning and advanced analytics close to the users and the data. Resulting in gaining real-time insights and delivering an immersive experience.
With these projects, the foundational future of technology for the organization has been built and the ability to preserve the program was provided. It is rare in a person’s career that one can achieve one, let alone both. I feel very fortunate to have been a part of these efforts and to work with such outstanding people that embrace teamwork with trust and respect.

IICA: What has surprised you most this year in government technology?

Carlson: To provide efficient services for the people of California, modern technology is vital. Many departments look to modernize their infrastructure to provide the capability to improve business practices, enhance security and promote faster service delivery. Once in the cloud, it is surprisingly easy to purchase resources such as virtual machines, storage and CPU in the cloud. This can be a good thing as it allows you to be agile and move quickly, such as purchasing additional CPU to improve performance in an emergency. However, it is equally as easy to purchase more than you planned and overextend your budget. This can be surprising if you are not prepared. As with anything, there needs to be balance. For example, this year marks one year of CSE being in the cloud. Concentrated due diligence in the area of cost management was top of mind. We worked side by side with our vendors to ensure our licensing, system performance and cloud resources were adequate to stabilize the CSE system in the cloud. It is easy to have “runaway” costs as there are essentially infinite resources offered in the cloud. It is through the discipline of education and governance that we worked to create a culture of cost consciousness. We analyzed the infrastructure to take advantage of cost savings opportunities such as reserved instances and savings plans to lower costs while maintaining stability. We also analyze our resource utilization to reduce our resources that are not being used. We work together as a technology division to understand where cost can influence planning. And we share our strategies with other departments within CalHHS. When working on the cloud migration project, DCSS looked to create a repeatable framework for other departments within CalHHS. We created a monthly informal infrastructure discussion forum with interdepartmental subject matter experts to share ideas across multiple departments within the agency. DCSS quickly became a leader within CalHHS to assist other departments with their cloud journey and share lessons learned. To date, we have remained within budget and continue to be able to proactively plan our costs for next year and are starting to see real cost savings.

IICA: What do you read to stay abreast of developments in the govtech/SLED sector?

Carlson: Government Technology Today* has great articles that are relevant to industry trends. I also enjoy reading about the progress the state is making with the articles in Industry Insider. Recently Gartner demonstrated audio articles; I will be looking into that. I enjoy attending the CIO Academy* with the motivational speakers, great panel topics and the ability to catch up with partners and peers. I have also been a mentor for the past three years for the California Department of Technology’s Project Management Leadership Academy and encourage those who are interested to attend. It is a great way to expose technology specialists to the art of project management as well as see how other departments approach the discipline.

IICA: What are your favorite hobbies and what do you enjoy reading?

Carlson: Married for nearly 18 years with two children who keep me busy. Love exercising with the family as much as possible, walking, running, hiking, swimming, dancing it out, staying active and keep moving especially when sitting and working virtually all day. I love to learn, and I enjoy reading books on leadership strategies. This is an area that seems to be ever-changing, can be circumstantial and there is no “one size fits all.” IT also can help both personally and professionally. One of my mentors suggested the book The Ideal Team Player by Patrick Lencioni. The book is a quick read with relatable examples. The concepts of being humble, hungry and smart are universal when creating or becoming a part of a high-performing team. For the past year, I have been working to optimize both a critical system in the cloud as well as the infrastructure and operations at DCSS. Through healthy relationships, positivity, creativity and collaboration, we continue to strengthen the technological foundation of DCSS and prepare it for what the future may bring.

*The Government Technology Today newsletter is produced by Government Technology magazine, a publication of e.Republic, which also produces Industry Insider — California. GT also hosts the California Public Sector CIO Academy.

Editor’s note: This interview has been lightly edited for style and brevity.