Worldwide IT Outage Linked to Faulty Cybersecurity Update

A flawed update for cybersecurity software from the company CrowdStrike caused many Windows machines to crash Friday, creating problems for hospitals, airlines, 911 services, banks and others across the globe.

Impacts range from airlines grounding flights to hospitals canceling non-emergency procedures. Several states are reportedly losing 911 services as well, while people in Australia and New Zealand have said on social media that they can’t access online bank accounts. Some stores also warned customers they could not process credit cards.

The trouble stems from cybersecurity company CrowdStrike issuing a faulty software update that crashed Windows machines that were running it.

According to Microsoft Azure’s website, “We have been made aware of an issue impacting Virtual Machines running Windows Client and Windows Server, running the CrowdStrike Falcon agent, which may encounter a bug check (BSOD) and get stuck in a restarting state.”

CrowdStrike said the problem is related to its Falcon Sensorproduct, which is intended to detect and block cyber attacks. The product is cloud-based and receives automatic updates. But in this case, a “defect” in just one content update for Windows caused the problems, CrowdStrike said. Mac and Linux were unimpacted, as were machines running Windows 7/2008 R2.

“This is not a security incident or cyber attack,” CrowdStrike wrote.

Cybersecurity journalist Brian Krebsnoted that “like most security software, CrowdStrike requires deep hooks into the Windows operating system to fend off digital intruders, and in that environment a tiny coding error can quickly lead to catastrophic outcomes.”

CrowdStrike also tests its updates before sending them out, but the company found that apparently “something changed or something got corrupted” between this update’s testing and deployment, Fraser said. CrowdStrike has now revoked the bad update, Microsoft reported, and the cybersecurity company said it found the problem and issued a fix.

But the fix cannot be applied automatically, according to cybersecurity company CyberArk’s CIO, Omer Grossman, who spoke to CNBC. Because the glitch crashed endpoints, they cannot be remotely updated, and each individual endpoint needs to be worked on manually. As such, he said he expected the process to take days.

Microsoft is exploring options that Azure customers can take to mitigate the effects. It also recommends customers try to restore from backups predating the software update’s rollout. Microsoft provided additional advice here.

But customers may need to turn to CrowdStrike for further help. CrowdStrike posted advice here, recommending “organizations ensure they’re communicating with CrowdStrike representatives through official channels.”

A version of this story was originally published by Government Technology, Industry Insider — California's sister publication.