On Friday July 19, 2024 at 4:09 a.m. IT leaders all over the world woke to a shocking surprise: the blue screen of death had washed away their businesses.
It was an unprecedented event: a massive global outage impacting roughly 8.5 million systems—that experts suggest may be the largest IT disruption in history. Summer travelers were stranded by more than 5,000 canceled flights. Point-of-sale systems stalled, thwarting millions of business transactions. Emergency call centers were disabled, leaving 911 calls unanswered in 14 states in the US. Hospitals lost access to patient records. Some of the screens in New York’s Times Square even turned blue.
All told, the worldwide financial impact of the outage is estimated to exceed US$10 billion.
A fix was quickly released and systems were restored. But we were all reminded of a hard truth: disruptions like this are now a part of doing business.
So we need a way to hit the ‘undo’ button.
Rubrik is in the resilience business; we were “all-hands-on-deck” that day. Our customers have invested in our comprehensive recovery systems to guard against outages of all kinds. We were able to use our expertise, experience, and powerful technology to help our customers recover from the outage–to return to a moment before 4:09 a.m. UTC and quickly restore operations.
Here’s how we worked with customers to keep their businesses online:
Crisis Council
Friday morning, we pulled together our best people with the right collection of skills to respond to the developing crisis.This brain trust of sales engineers, support leaders, experts in systems recovery and business continuity worked together to create Rubrik’s crisis response plan. They did ongoing work to identify the best next steps for customers to take as the situation evolved.
Situation Room (on Zoom)
One of the council’s first actions was to establish a perpetual Zoom call where customers could drop in with Rubrik technical experts as-needed. This gave everyone real-time access to the latest updates on the outage. This virtual situation room allowed our customers’ IT teams to get immediate support from Rubrik.
Rubrik’s Recommendation: In-Place Recovery
That same day, we published a blog post with specific recovery advice: use in-place recovery to restore affected virtual machines. “This enables large groups of VMs to be reverted to a working snapshot in minutes to quickly bring your environment back online,” wrote Rubrik’s Joshua Stenhouse. This document also contains step-by-step demos showing how to execute the restore, plus additional documentation and resources that IT teams may need for reference.
Rubrik Support Resources
Rubrik Support also published this article and this video to help customers with their recovery. In the end, these efforts helped our customers recover from what some call history’s largest IT disruption.
I believe Rubrik has the best cyber resiliency product on the market and the smartest staff in the industry. And our customers agree, as one of them told me in the aftermath of the crisis.
“That is exactly why Rubrik shines above the rest, its ability to quickly do an in-place recovery to rewind to the last working condition,” said Andres Calderon, Director of Cyber Resilience at CommonSpirit Health.
Global disruption happens. But you can hit “undo” to recover.