As part of Techwire’s ongoing efforts to inform readers about state agencies, their IT plans and initiatives, here’s the latest in our periodic series of interviews with departmental IT and cybersecurity leaders.
George Khalil is the inaugural chief innovation officer at the city of Riverside, a position he has held since August 2018. During his tenure, Riverside claimed sixth place in its population category in the 2020 and 2021 editions of the Center for Digital Government’s* (CDG) Digital Cities Survey. In 2019, Riverside won first place in Overall City Government Experience at CDG’s third annual Government Experience Awards. A recommendation for other governments, Khalil said, is “for us to leverage each other’s efforts,” quoting former city colleague Bill Bunger’s observation that “in government, ‘R&D’ is not research and development, it’s rip and duplicate.”
“Our work is not intellectual property; to some extent, it’s taxpayer-paid items, and if we can help each other and reuse and reinvest the time and effort done by other agencies, we can help our constituents dramatically,” Khalil told Techwire.
Khalil was previously the city’s interim deputy CIO/CISO from June to August 2018; he joined Riverside in May 2015 as its chief information security officer. Elsewhere in the public sector, he served as senior information systems specialist at the city of Rancho Cucamonga from March 2010 to October 2013; and as systems administrator/network engineer at Redlands Community Hospital from February 2009 to April 2010.
He has a bachelor’s degree in IT from American Intercontinental University and a Master’s of Science in Information Security Engineering from SANS Technology Institute. Among his professional credentials, he is a SANS/GIAC (SysAdmin, Audit, Network and Security/Global Information Assurance Certification) security expert and a systems and network auditor.
Techwire: As CIO of your organization, how do you describe your role; and how have the role and responsibilities of the CIO changed in recent years?
Khalil: CIOs, for a long time, it started as keeping the lights on, managing infrastructure, classic compute and data center and storage and backup, and things along these lines. CIOs and the city created an innovation division to try and transform a lot of legacy business processes, and we’ve been doing that. We’ve been focusing on trying to gear towards online services, digital twins of whatever digital services we’re providing. But if you look across the industry, it was a slow transformation that was going on for the past five years. Then when COVID hit, everybody realized that it’s needed. This is not something that needs to get dragged on. And COVID has been, to some extent, the biggest transformation driver that I’ve seen throughout my 25-year career in technology. Things that were explored that could take two to three years to get done were getting done in two to three days. It’s very eye-opening to see how everybody embraces change in an emergency situation ... . If there’s anything that we’re starting to see, evolution of the role, it’s the delivery speed is really becoming crucial to transform organizations. That the classic way of two- to three-year complex projects is how we’ve operated as organizations in the past and we’re seeing more and more that there’s very little patience, if that’s the right word to use right now, for this. And agile development and agile deployments and improving as you go along is really the way to move forward as a technology leader. ... The fast development is labor-intensive. Because you’re crashing projects, if you’re familiar with project management, you’re throwing (in) all the resources you have. Accepting more tolerance for errors, because agile is about having that feedback loop and going back and doing corrections within small sprints. That’s versus doing extensive (quality assurance) QA and being sure of a completely polished product. It’s a two-way street. The biggest challenge is labor now, because for us to crash work and deliver results faster, it means we need to throw all we have at it, to start having a minimum viable product out there quickly. And labor is becoming harder and harder to manage and the shift in mindset of the workforce from that legacy two- to three-year project planning and QA and testing, etc., is very labor-intensive and it really requires retraining the workforce and changing the mindset right now.
Techwire: How big a role do you personally play in writing your organization’s strategic plan?
Khalil: We haven’t had a formal strategic plan for at least eight to 10 years. We’re focusing at the moment, short-term, on modernizing infrastructure. We’re building a private cloud at the moment. ... We just finished upgrading a 25-year-old in-house-developed GIS system that was attempted two to three times in the past and didn’t succeed and we’re wrapping it up right now. So, once the infrastructure modernization is done, we’re going to be switching gears into upgrading a long-term five- to seven-year strategic plan. We’re going to start, I would say, probably in the next one to two years, when we finish most of the big infrastructure modernization.
Techwire: What big initiatives or projects are coming up? What sorts of RFPs should we be watching for in the next six to 12 months?
Khalil: I think we have our hands full for probably the next year and a half already. Our network infrastructure is probably about 14 years old at this point, so we’re doing a complete overhaul. We put an RFP out a year and a half ago and it was awarded to Aruba to build a software-defined network, to start building more flexibility into our environment, to embed disaster recovery, fail-over, extension and hyper-segmentation, and security device posture assessment capabilities that we’ve always wanted. We’re trying to deal with legacy technology and that’s taking a lot of our time at the moment. Implementation is in progress.
We’re also working on building a private cloud to support multiple tenants. We’re seeing more and more that we can use it as a city, as a municipality. We have multiple businesses under one roof, unlike the private sector, from law enforcement with criminal justice data, to utilities that manage power grids, water, sewer, you name it. To public-facing services, to payment processing to ... our utility, we collect about $300 million a year in payments alone for our utility billing system. So, we have a lot of diverse needs and trying to treat everybody the same as municipalities have in the past ... is good and bad. I’ll pause there for a second. You can’t create customization and create custom solutions for everyone, in trying to create scalable infrastructure and infrastructure that can accommodate this from day one. With the private cloud, we’re trying to build multi-tenant environments to reduce the duplication of services following the Microsoft Azure and AWS (Amazon Web Services) models. Building infrastructure that can scale, hyper-converged infrastructure, automatic failover for primary and backup data center, replication, micro-segmentation to be able to isolate different business units and different data types from each other is what we’re also working on in the data center. The same software-defined network that we’re implementing, we’re basically implementing a software-defined data center as well. To be able to be as flexible as we can and accommodate business needs for the next five-plus years. ... Most of it is already out (procurements). We’re at the point where we have our network refresh, that was a $3 million-plus project, data center refresh was a $3 million-plus project. We have a phone system upgrade that was a half-million dollars that we already awarded. We have a video infrastructure upgrade that was almost a three-quarters of a million-dollar project. All of them have been out and awarded, and now we have to get the work done. That’s going to keep us busy for a while.
Techwire: How do you define “digital transformation,” and how far along is your organization in that process? How will you know when it’s finished?
Khalil: We focused about two to three years ago in trying to have a virtual City Hall. As a transformation of business processes. Where we tried to mirror anything that someone can do in person — maybe ‘any’ is a broad word — but trying to get the vast majority of what people can do in-person and trying to do it remotely. We’ve taken a couple of approaches, actually, and we ended up winning first place in the digital governments a couple of years ago for that. We’ve even taken the approach of environmental impact, beyond digital transformation. We’ve submitted some studies of the carbon footprint of people having to come into City Hall to do services and how many visits do we get to get permits and fees and inspections and plans? And all these trips, we calculated, I think, in one year more than 50,000 trips. And adding carbon footprint on top of that, I think people can do that service remotely. There’s monetary savings from transportation, gasoline, and then adding it through a ... carbon footprint that’s submitted by that transportation. Digital transformation is a buzzword but, I think, what are we trying to achieve? And really, it’s reducing expenses for our residents and the city, being more efficient. Increasing turnaround time on the service that we’re delivering and being good custodians of the environment as well, and finding a more efficient way of doing business without having heavy carbon footprints. That’s really what we’re trying to achieve. ... Everybody will take an automation perspective or whatever but ultimately, it’s to deliver a service, and that service is reaching out to our residents, providing quality customer service and being good custodians of financial and environmental resources that we have.
Techwire: What is your estimated IT budget and how many employees do you have? What is the overall budget?
Khalil: The IT budget, I believe, is about $12 million a year right now, and that includes our contractual costs, subscriptions and software renewals, and maintenance and labor. We have about $10 million in capital project upgrades in addition to that — those are the data center and the other items I talked to you about earlier. We’re budgeted for 60 FTEs. I think we’re currently at about 50 or 51 filled and we’re having a little bit of a hard time filling those but we’re continuing to try to recruit.
Techwire: How do you prefer to be contacted by vendors, including via social media such as LinkedIn? How might vendors best educate themselves before meeting with you?
Khalil: I like reading the (LinkedIn) articles and engaging in discussions. However, I’m getting on an average 50 to 60 private messages per day saying, ‘Buy our stuff and you’ll solve all your problems.’ And I got to the point where I just don’t read them anymore, there’s so many messages coming in. I engage in posts and responses and things along these lines but I’m at the point ... of finding a way to shut down private messages all together. I think the volume (of email) right now has reached the exhaustion point, and you see that across the industry. At this point, I set up an automated response for most direct sales outreach, to just recommend that they go on our portal and register as a provider. That way, we’re not getting 20 vendors, everybody asking for a 30-minute demo of their product that we may not even have a budget for. And I’ve been advising most people to go on our RFP portal and register the type of service they have. And that way when there is a need, we can notify and engage. I think from future planning, capacity-wise, with the workload that all of us have and the staffing challenges and the dramatic transformation that we’re all going through, education is helpful to know what else exists out there, but the bandwidth does not exist. My preferred way of getting introductions to capabilities and road-mapping items is honestly through a conference that has multiple vendors. That’s the best use of everybody’s time, rather than direct email outreach or LinkedIn messages.
Techwire: In your tenure in this position, which project or achievement are you most proud of?
Khalil: It’s tough to pick one. There’s a lot of transformation initiatives that we’ve taken on. Most recently, I think building and launching our rental assistance program in the middle of a pandemic, in the middle of urgent timelines, when we had the CARES Act and we needed to get money in people’s hands quickly to help them with their needs. Our team ended up building a platform from the ground up in six weeks, from zero to launch in six weeks. And they put in a tremendous amount of time; I think we put in more than 1,400 hours in six weeks. That was a monumental effort and a very worthy cause to help our community. We’d attempted to look at commercial products and we had numbers estimating up to a million dollars just to build a system or to give us access to a system that they have. And alternatively, we ended up developing it in-house and the money got put into residents’ hands instead.
Techwire: If you could change one thing about IT procurement, what would it be?
Khalil: So, one thing that I chit-chatted with our colleagues about, creating RFPs is a six- to eight-month type of effort. One item that I spoke to some of my L.A., San Diego, San Bernardino and Riverside colleagues about is, we leverage piggyback contracts and piggyback on other agreements that have been competitively bid by other agencies. And one suggestion that I made was ... if we’re investing in creating an RFP, why don’t we include those municipalities with a piggyback clause, that the vendor offering us this pricing will offer the same pricing to other agencies? And that would dramatically expand the pool of piggyback contracts and help all other agencies become more efficient if they see something that works — I know there’s a lot of collaboration in the industry, but when it comes to procurements, we tend to think about ‘us.’ I think if it could literally be a one-liner or one paragraph added to all of our RFPs, to make that procurement be available to others and the time and effort spent in creating the RFP would be a savings for others. I don’t think anybody has done it.
Techwire: What do you read to stay abreast of developments in the gov tech/SLED sector?
Khalil: Industry magazines, I’m subscribed to a bunch of industry daily emails from everyone. LinkedIn is a good educational opportunity on what people are sharing and what organizations are sharing, excluding direct messages.
Techwire: What are your hobbies, and what do you enjoy reading?
Khalil: There’s very, very little time actually left, outside of everything we’re doing. At this point, we’re almost working weekends and days off and whatnot. I’m actually trying to disconnect a little bit and reset my brain, so gardening is my way out, of not having to think about technology or a lot of complex things. Just enjoying the beauty of nature and getting your hands dirty resets your mind quite a bit.
*The Center for Digital Government is part of e.Republic, parent company of Techwire.
Editor’s note: This interview has been lightly edited for style and brevity.
Techwire One-on-One: Riverside CIO on Collaboration, IT Refresh
