On July 19, 2024, CrowdStrike experienced a significant global outage triggered by a sensor configuration update that impacted Windows machines worldwide. Although this incident was not a security breach, it caused widespread operational challenges. The sensor update led to disruptions, including the "Blue Screen of Death," affecting numerous organizations that rely on CrowdStrike for endpoint security.
During this outage, the California Department of Child Support Services (DCSS) partnered with Tek Yantra's Security Operations Center (SOC) team to manage the situation effectively. Despite not being directly impacted by the outage, DCSS faced a surge in phishing and notification alerts, underscoring the importance of maintaining robust security measures.
Tek Yantra’s SOC team is instrumental in providing 24/7 monitoring for DCSS, ensuring that all systems remain secure and operational. During the CrowdStrike outage, Tek Yantra’s vigilance and expertise were crucial in managing the increased phishing alerts that arose during this time.
Support in Tools Integration and Vulnerability Management
A critical aspect of Tek Yantra's support for DCSS was facilitating tools integration with ServiceNow, which streamlined the incident management process and enhanced response efficiency. This integration allowed for seamless data flow and automation of security workflows, enabling the SOC team to prioritize and respond to incidents more effectively.
Additionally, the team was involved in reviewing Tenable scans as part of the Security Incident Response (SIR) process. By analyzing these scans, Tek Yantra identified potential vulnerabilities and ensured that any weaknesses in DCSS’s systems were promptly addressed, minimizing the risk of exploitation.
The SOC team was responsible for continuously monitoring various tools and platforms, including CrowdStrike Falcon, Azure Sentinel, ExtraHop, Microsoft Defender, and Abnormal Security. Their role was not only to identify potential threats but also to keep track of events and alerts, providing a comprehensive view of the security landscape.
Continuous Monitoring and Active Involvement
As the CrowdStrike outage unfolded, Tek Yantra’s SOC team was on high alert. They engaged in continuous monitoring of all tools and systems, focusing particularly on the Abnormal Security tool to identify any suspicious inbound emails. This vigilance was crucial in preventing phishing attempts that could exploit the situation.
Throughout the incident, the SOC team remained on standby, ready to respond to any threats. They maintained their monitoring efforts from 12 a.m. on the night of July 19 until 11 a.m. on July 20, ensuring that DCSS systems were not compromised. This continuous coverage allowed them to detect and address any issues promptly, safeguarding DCSS’s sensitive data.
Coordination and Communication
Tek Yantra’s team demonstrated exemplary collaboration by maintaining open communication with DCSS and CrowdStrike throughout the incident. Although the outage was not a security incident, the team’s role in monitoring and supporting recovery efforts was vital.
The SOC team actively participated in calls with DCSS and CrowdStrike, providing insights and updates on the situation. They shared all relevant logs and information, ensuring that everyone involved had a clear understanding of the ongoing events. This proactive communication helped facilitate a swift response and contributed to the effective management of the situation.
Providing CrowdStrike's Recommendations
While Tek Yantra’s SOC team was not directly involved in the technical implementation of service recovery, they played a critical support role. They provided the server administration team with CrowdStrike's recommendations, which included steps for removing the faulty sensor configuration that had caused the outage.
Their responsibilities included:
- Analyzing Alerts: Keeping track of alerts and events across multiple platforms to ensure that no threats went unnoticed.
- Providing Recommendations: Relaying CrowdStrike's recommendations to the server administration team, helping to guide the recovery process and ensure timely implementation of the necessary fixes.
- Ensuring Continuous Security: Monitoring for any potential threats during the recovery process, including phishing attempts and other suspicious activities.
Shift Handover Excellence
Tek Yantra’s dedication to excellence was further demonstrated through their well-structured shift handover process. As part of the 24/7 SOC operations, each team member was responsible for preparing detailed shift handover reports. Naveen Goud (SOC lead), along with Suresh Pasula (SOC analyst) and Shakawat Hossain Tusher (SOC analyst), ensured seamless transitions between shifts and maintained consistent oversight of all security operations.
The team took the initiative to develop their own shift handover template when a client-provided version was delayed. This proactive approach impressed DCSS, highlighting Tek Yantra’s commitment to delivering clear, concise, and timely information to the client. The SOC team’s ability to maintain a high level of professionalism and organization during such a challenging period was crucial in maintaining DCSS’s trust and confidence.
Client Appreciation and Conclusion
The collaborative efforts of Tek Yantra’s SOC team and DCSS during the CrowdStrike outage exemplified the importance of having a responsive and skilled cybersecurity partner. DCSS expressed gratitude for the SOC team’s unwavering support, professionalism, and the detailed shift handover reports that kept everyone informed and aligned.
The lessons acquired from the 2024 CrowdStrike outage highlight the significance of having dependable and adaptable cybersecurity strategies as the cybersecurity landscape continues to change. Tek Yantra’s role in supporting DCSS highlights the power of collaboration and expertise in navigating complex cybersecurity challenges, ensuring the resilience and integrity of digital infrastructure.Join Tekyantra today.
