Note: This feature was first published on 22 July 2024 and was updated on 23 July 2024 10:15am with the news of Linux systems being affected in May and June and Mac kernel denial.
A familiar sight last week. Image source: Wikipedia.
On July 18, an update was sent out by cybersecurity firm CrowdStrike for its Falcon security platform. Unfortunately, the update began impacting Windows-based systems globally resulting in 8.5 million Windows devices or less than one percent of all Windows machines affected. Airlines, media outlets, banks, retailers, and businesses using Microsoft’s Windows operating system were amongst those affected.
Who is CrowdStrike and what is Falcon?
What is ironic is that Crowdstrike is a cybersecurity company. Based in America, it provides cloud workload protection, endpoint security, threat intelligence, and cyberattack response services.
Falcon is CrowdStrike’s “endpoint detection and response” (EDR) software that organisations install on their computers to keep them safe from cyberattacks and malware. Its job is to monitor what is happening on the computers on which it is installed, looking for signs of malicious activity.
A rushed update is to blame for the outage. Image source: Pixabay.
Why are they important?
CrowdStrike has a 24% market share of the endpoint protection market globally. This means that one in four companies worldwide is its customer and thus affected.
Airports were just one of the many places hurt by the CrowdStrake outage. Image source: Pexels.
Some key points of the incident?
Initial reports began coming out of Australia with broadcasters and banks saying they were affected by the Blue Screen of Death (BSOD).
In the United States, between 2 and 3am (EST), the Federal Aviation Administration announced that all Delta, American Airlines, United and Allegiant Airline flights were grounded.
Late morning US time (EST) saw hospitals reporting delays in services with others cancelling some non-urgent surgeries and medical office visits.
In the U.K., broadcaster Sky News was unable to broadcast its news bulletin, and clinics were unable to book patient appointments.
Locally The Straits Times newsroom was briefly affected, while some carparks had to keep their barrier arms lifted to allow motorists to enter and exit. Changi Airport saw the check-in processes for more than 10 carriers, including Singapore Airlines’ budget arm Scoot as well as Jetstar, AirAsia and Cebu Pacific Air, go down needing manual intervention.
Online reports say CrowdStrike released a statement telling customers that it was “aware of reports of crashes” of its software on Microsoft Windows operating systems later that morning.
What did the CEO say?
CrowdStrike CEO George Kurtz posted on X saying, “We understand the gravity of the situation and are deeply sorry for the inconvenience and disruption,” and that, “Today was not a security or cyber incident. Our customers remain fully protected.”
What are companies around the world doing about this?
CrowdStrike is busy working to help customers recover while reassuring them that their data remains safe.
Although not responsible, Microsoft is working with partners and customers to get them through this hurdle. It‘s released a recovery tool to help impacted Windows endpoints.
According to online reports, some companies are facing issues rolling out the update to fix the issue resulting in a slow recovery from the issue.
More than 1,800 US flights cancelled and more than 9,900 were delayed, according to the tracking website FlightAware. Passengers are trying to get their plans back on track as airlines struggle to return to full capacity.
Jake Moore, Global Security Advisor at ESET said, “The inconvenience caused by the loss of access to services for thousands of people serves as a reminder of our dependence Big Tech such as Microsoft in running our daily lives and businesses. Upgrades and maintenance to systems and networks can unintentionally include small errors, which can have wide-reaching consequences as experienced today by CrowdStrike's customers.”
He cautioned that one key point raised by this incident is that of “diversity” in the use of large-scale IT infrastructure. This applies to critical systems like operating systems (OSes), cybersecurity products and other globally deployed (scaled) applications. “Where diversity is low, a single technical incident, not to mention a security issue, can lead to global-scale outages with subsequent knock-on effects,” he added.
Who is to blame?
Well, all the finger-pointing is aimed at one party - CrowdStrike. The company admitted as such with the statement from the CEO saying that it is “working closely with impacted customers and partners to ensure that all systems are restored”.
Microsoft’s statement laid out whose fault it felt it was with Microsoft’s statement saying, “CrowdStrike, an independent cybersecurity company, released a software update that began impacting IT systems globally.”
The sight many IT helpdesks had to deal with. Image source: Pixabay.
Can this happen with any other EDR provider?
To be fair to CrowdStrike, this could happen to any tech company that pushes updates out to partners and customers.
But as CrowdStrike competitor Trend Micro asserted, while its important software updates are pushed out quickly, the firm takes steps to mitigate the risks of its updates being buggy.
Is this the first tech outage?
This isn’t the first time we’ve seen our access to tech services brought down by an update.
On 21 April 2010, McAfee rolled out a faulty software update that mistakenly identified a critical Windows file as a worm and quarantined it, crashing tens of thousands of computers from Australia and Europe to the US. Ironically, McAfee's CTO at that time, George Kurtz, went on the become the CEO of CrowdStrike!
Google services including Calendar, Gmail, Hangouts, Maps, Meet and YouTube went dark for about an hour on 14 December 2020.
A routine maintenance job gone awry effectively disconnected all Facebook data centres globally causing Instagram, Facebook, and WhatsApp to be inaccessible for all switched off for six hours on 4 October 2021.
Spotify went down due to issues in a cloud-hosted service discovery system used at Spotify on 8 March 2022.
*Update*
In his official statement about the outage, the CEO of CrowdStrike said that Linux-based systems weren't affected. Unfortunately, while that may be true of this recent outage, evidence has arisen about CrowdStrike updates in May and June that affected devices running CrowdStrike Falcon on Rocky Linux and Red Hat Linux. We will update this article if he comments about this.
It turns out that Cuppertino had a good reason for not granting developers kernel-level access since 2020 with the release of macOS Catalina. This is why Macs avoided the CrowdStrike outage.
Updates are still getting pushed out to restore IT services. Image source: Pexels.
Am I safe now?
Chances are that as a consumer, you haven’t really been affected by the faulty update. But that doesn’t mean you're in the clear.
The Cyber Security Agency of Singapore (CSA) is urging Singaporeans to be extra vigilant towards phishing scams related to the outage. Besides phishing attacks related to a fake hotfix, it also warned of individuals or groups leveraging on the tech outage to send phishing e-mails to customers while posing as CrowdStrike support staff or to impersonate CrowdStrike staff in phone calls.
Our articles may contain affiliate links. If you buy through these links, we may earn a small commission.