It started early on Tuesday morning. Robot vacuums ceased sucking, wifi cameras stopped watching and eager Tinder daters were left unable to “swipe right” on their smartphone apps.
An outage at Amazon Web Services, the cloud arm of Amazon, had rippled through the online economy, crippling services used by millions of people.
Among the most distraught were fans of the British singer Adele who had been hoping to snap up the first tickets to her upcoming Las Vegas residency.
“Due to an Amazon Web Services (AWS) outage impacting companies globally,” the ticket seller Ticketmaster explained, “all Adele Verified Fan Presales originally scheduled for today have been moved to tomorrow.”
The disruption highlighted the degree to which many of the internet’s most popular services rely on cloud computing infrastructure handled by a very small number of large companies.
According to Gartner, 80 per cent of the cloud market is handled by just five companies. Amazon, with a 41 per cent share of the cloud computing market, is the very biggest.
"They've had some very large outages," said Servaas Verbiest from Sungard Availability Services, a company that provides "disaster recovery" for multiple cloud platforms. "What makes AWS more exposed is the sheer volume of business they have."
Within Amazon itself on Tuesday, the unthinkable occurred: grounded delivery drivers were unable to load packages and deliver to customers’ doorsteps, just as the peak Christmas season begins to step up.
Drivers at multiple facilities across the country were sent home with pay. With little to do, many of them logged on to social media to enjoy the moment while it lasted - some dreading whatever workload may await once systems were back up and running.
Root cause
An “impairment of several network devices” in one of its server regions – US-EAST-1 – was the “root cause” of the disruption, Amazon said in a message posted to the AWS status page, which monitors the operational health of its global network of interconnected computers.
Amazon did not comment on the disruption to its deliveries.
Business Insider quoted an internal memo detailing a flood of traffic from an “as yet unknown source”.
Publicly, the company logged the first issues at 9.37am US Pacific time (5.37pm) on Tuesday morning, though users of affected services had complained of problems before then. By 3pm, AWS said it had been able to mostly restore service.
Several of the sites first affected appeared to have been able to reroute traffic to alternative servers. Whether or not outages created longer-lasting problems for companies depended on the degree to which executives prioritised diversifying their cloud computing providers, added Verbiest.
“If you’ve embraced the ecosystem, and you’ve got everything in AWS, you’re in a sit-and-wait scenario,” he said.
While high-profile outages can be a boon for competitors such as Google and Microsoft, Verbiest stressed the bar to switching service providers was high.
“It’s difficult to say that one outage is going to sway people to one cloud platform or another, because every cloud provider has outages. It’s just about how long are they and how do they resolve them when they happen?”
In November 2020, the US-EAST-1 region was also at the heart of an AWS outage affecting many of the same websites. In that case, a fault with an Amazon system called Kinesis was said to be the culprit.
Affected companies
This time, according to DownDetector.com, which identifies websites and services that are struggling or failing to load, affected companies included McDonald’s, PayPal-owned payments service Venmo, delivery service DoorDash and video conferencing platform Zoom.
The disruption to Amazon Prime Video and Amazon Music would appear to benefit Netflix and Spotify. However, both rivals also use AWS and were similarly affected.
iRobot, the creators of the autonomous Roomba vacuum, apologised to users who could not log into the device’s app.
One apparent Roomba owner quipped on Twitter: “My wife is going to kill me if the foyers aren’t mopped before she gets home.” – Copyright The Financial Times Limited 2021
