12mo ago

CrowdStrike Explains Friday Incident Crashing Millions of Windows Devices

9 comments

Wouldn't any internal testing have cought this issue at CrowdStrike?
- A smoke test, aka turn it on and "see if it catches fire," would have caught this.
  
  And a controlled rollout would've limited the damage.
- Yes. Why would anyone trust Crowdstike after this? They’ve ignored foundational deployment steps.
But will you try actually installing the update on a machine or 50 to see if you bork things horrifically?
Crowdstrike: "We are really focused on unit testing right now"
I probably misread it, don't mind my grumbling, rabble rabble rabble
CrowdStrike report of the incident: https://www.crowdstrike.com/falcon-content-update-remediation-and-guidance-hub/
- Local developer testing
  Hmm, didn't think of that one...
  staggered deployment strategy
  Also a novel idea...
  It's like they're catching up to best practices from 10 years ago, good job team!
  
  Listening to literally any sysadmin would have had these practices already in play.
  I wonder if any are in the building, of if it's all devs and "platform engineers."
- Systems in scope include Windows hosts running sensor version 7.11 and above that were online between Friday, July 19, 2024 04:09 UTC and Friday, July 19, 2024 05:27 UTC and received the update.
  Definitely incorrect. My machine was powered off by physical switch at that time. It was powered off at 17:00 the day before and powered up at 08:00 CEST / 06:00 UTC and promptly bluescreened.

9 comments