Did you know that Amazon includes status messages about the health of availability zones in the output of the ec2-describe-availability-zones
command, the associated API call, and the AWS console?
Right now, Amazon is restoring power to a “large number of instances” in one availability zone in the us-east-1
region due to “electrical storms in the area”.
Since the names used for specific availability zones differ between AWS accounts, Amazon can’t just say that the affected zone is us-east-1c
as it might be us-east-1e
in another account.
During this outage, you can find out what the name of the affected availability zone is in your AWS account by running this command (installation instructions):
ec2-describe-availability-zones
Here is the output for one of my accounts showing that the zone is named us-east-1b
.
AVAILABILITYZONE us-east-1a available us-east-1
AVAILABILITYZONE us-east-1b impaired us-east-1 EC2 and EBS APIs are once again operating normally. We are continuing to recover impacted instances and volumes.
AVAILABILITYZONE us-east-1c available us-east-1
AVAILABILITYZONE us-east-1d available us-east-1
AVAILABILITYZONE us-east-1e available us-east-1
and here is the output for another account, showing that the zone is named us-east-1c
.
AVAILABILITYZONE us-east-1a available us-east-1
AVAILABILITYZONE us-east-1b available us-east-1
AVAILABILITYZONE us-east-1c impaired us-east-1 EC2 and EBS APIs are once again operating normally. We are continuing to recover impacted instances and volumes.
AVAILABILITYZONE us-east-1d available us-east-1
AVAILABILITYZONE us-east-1e available us-east-1
If you’re not a command line person, you can also check on the AWS console, which for one of my accounts, shows this right now:
You can generally find more details on the progression of Amazon’s investigation and repair on the AWS Status page. That page also has links for RSS feeds like this one: EC2 us-east-1 Service Status
Since the availability zone status information is available through the command line and API, has anybody written plugins for Nagios or similar monitoring software so that we can send alerts to our teams when Amazon marks availability zones as impaired?
Update 2012-06-30: Jim Browne has taken the challenge and created a Nagios plugin for ec2-describe-availability-zones.
Update 2012-06-30: It looks like the ec2-describe-availability messages are not updated nearly as frequently as the AWS status page. An hour ago the AWS status page changed to say “EC2 instances and EBS volumes are operating normally”, but ec2-describe-availability-zones still says “We are continuing to work to recover the remaining EC2 instances, EBS volumes and ELBs.”