December 26, 2012 – Amazon has resolved a trio of disruptions with its cloud services stemming from an East Coast data center that spanned Christmas Eve and Day for some customers.
Performance problems were first reported Monday afternoon EST for Amazon Web Service’s CloudSearch, EC2 and Elastic Beanstalk offerings, all housed at the same northern Virginia data center, which impacted US-EAST 1 region customers, according to the vendor’s publicly available health dashboard. New domains and indexing operations were unavailable through CloudSearch for some AWS customers, with “severe” delays in domain updates into Christmas Day. During this same time period, certain Elastic Beanstalk customers found inaccurate cloud environment health reports, and were delayed in creating or updating environments. EC2 customers were also experiencing load balancing latency and traffic issues during the holiday.
By noon EST Tuesday, Christmas Day, the dashboard’s-reported issues were resolved for two of the services, which were “operating normally.” The third service, EC2, took three more hours of monitoring before it was cleared as resolved and fully functioning by AWS.
In its dashboard summaries, Amazon did not state if the services issues were directly connected, or the number of customers that were impacted. Along with scant customer chat on Twitter, Netflix publicly acknowledged the outage as an issue with its online movie services. There was no indication that the holiday had a specific role in the outage, like with a Microsoft cloud bug earlier this year.
A post mortem on the disruptions had not been released by Amazon as of Wednesday.
The same AWS cloud center in Virginia was the site of other recent outages, including one in October that was triggered by a hardware replacement that led Amazon to change its approach to memory monitoring and throttling issues. Experts have said that the degree of damage such service disruptions can do to cloud vendors varies, and may face an inordinate amount of scrutiny compared with internal failures because of the maturity level and public nature of cloud computing. Forrester Research analyst Rachel Dines found her Netflix to be down during the cloud disruption, and noted that even though Netflix is among the most mature cloud implementations she's seen there remains a cautionary tale in this incident of enterprise architecture resiliency regarldess of where applications are stored.
The cloud service disruption on Christmas Eve came days after big rollouts by Amazon aimed at cornering more of the cloud market for big data as well as expanding its relational database capabilities.