Home About Blog News Contact

A Bug in Cybersecurity Software CrowdStrike Causes Disruptions

The world is still reeling from disruptions caused by a bug in an automatic update rolled out by cybersecurity company CrowdStrike that has crashed Windows hosts that have implemented it. Airlines, banks, merchants of all kinds, emergency services and more, all over the world, have come to a grinding halt and are unable to operate normally. https://techcrunch.com/2024/07/19/faulty-crowdstrike-update-causes-major-global-it-outage-taking-out-banks-airlines-and-businesses-globally/. CrowdStrike has acknowledged a bug in its update. Affected machines are unable to boot properly.

The workaround or solution to the bug, however, is causing further delays in restoring services, because it requires manual intervention at each workstation affected, and cannot be done via script or network-wide software group policy change or similar. Instead, each affected PC must be manually rebooted into safe mode, in which fewer programs run at startup. That way, the bug, a file which prevents booting into normal mode, can be deleted manually, and the system restarted into normal operation.

I have written about the importance of good DevOps, https://andrewtetzeli.substack.com/p/report-surfaces-thousands-of-potential. DevOps provides a set of practices and tools as methodology in improving, streamlining, and securing software development and output. CrowdStrike's failure with good DevOps in this case has caused significant issues not only for its customers but many millions of people around ther world, and itself.

The big problem and rhetorical question is: How did that update with such a bug that renders computers inoperative make it through DevOps and into the production environment? A bug that severe would and should have been caught and kept from release and rollout to machines around the world had there been adequate DevOps in place.

I mean, how can one roll out an update like that?! Had it been tested by actually loading it into test machines, with the obvious result that they are unable to boot up with it, that bug should and would never have made it into the wild, into machines in production environments. The world now sees the disastrous, rippling damage caused by such a fundamental breakdown in protocol.

Now for the upside: CrowdStrike and other cybersecurity companies will most likely learn and adapt from this incident. They will tighten and lock down software development, DevOps, practices, and redouble their efforts at testing before release.