March 2010 Archives

On Amazon EC2, you can create a new EBS volume from an EBS snapshot using a command like

ec2-create-volume --availability-zone us-east-1a --snapshot snap-d53484bc

This returns almost immediately with a volume id which you can attach and mount on an EC2 instance. The EBS documentation describes the magic behind the immediate availability of the data on the volume:

“New volumes created from existing Amazon S3 snapshots load lazily in the background. This means that once a volume is created from a snapshot, there is no need to wait for all of the data to transfer from Amazon S3 to your Amazon EBS volume before your attached instance can start accessing the volume and all of its data. If your instance accesses a piece of data which hasn’t yet been loaded, the volume will immediately download the requested data from Amazon S3, and then will continue loading the rest of the volume’s data in the background.”

This is a cool feature which allows you to start using the volume quickly without waiting for the blocks to be completely populated. In practice, however, there is a period of time where you experience high iowait when your application accesses disk blocks that need to be loaded. This manifests as somewhat to extreme slowness for minutes to hours after the creation of the EBS volume.

For some applications (mine) the initial slowness is acceptable, knowing that it will eventually pick up and perform with the normal EBS access speed once all blocks have been populated. As Clint Popetz pointed out on the ec2ubuntu group, other applications might need to know when the EBS volume has been populated and is ready for high performance usage.

Though there is no API status to poll or trigger to notify you when all the blocks on the EBS volume have been populated, it occurred to me that there was a method which could be used to guarantee that you are waiting until the EBS volume is ready: simply request all of the blocks from the volume and throw them away.

Here’s a simple command which reads all of the blocks on an EBS volume attached to device /dev/xvdX or /dev/sdX (substitute your device name) and does nothing with them:

sudo dd if=/dev/xvdX of=/dev/null bs=10M

OR

sudo dd if=/dev/sdX of=/dev/null bs=10M

By the time this command is done, you know that all of the data blocks have been copied from the EBS snapshot in S3 to the EBS volume. At this point you should be able to start using the EBS volume in your application without the high iowait you would have experienced if you started earlier. Early reports from Clint are that this method works in practice.

[Update 2012-04-02: We now have to support device names like sda1 and xvda1]

Note: I do not recommend that new users start with these AMIs. These AMIs run with older versions of Amazon’s Fedora 8 kernel which have some incompatibilities with Ubuntu and Debian (e.g., XFS is broken). My strong recommendation is for all users to convert to one of the Ubuntu 9.10 Karmic or Ubuntu 8.04 Hardy AMIs which run with a modern, compatible Ubuntu kernel.

I have released updates of the following legacy Ubuntu and Debian S3-based AMIs published in the “alestic” buckets:

  • Ubuntu: 8.04 Hardy, 8.10 Intrepid, 9.04 Jaunty
  • Debian: 4.0 Etch, 5.0 Lenny

I welcome testing and feedback from folks who are already using the older versions of these AMIs.

In addition to upgraded software packages, the following enhancements are in this release:

  • Wait for meta-data before deciding whether to generate ssh host key (Thanks to Dmitry for catching)

  • Patch from Tom White to support compressed user-data scripts

  • Allow user-data script to remove the “been run” file so that it is run on every boot instead of just the first

  • Upgrade EC2 AMI tools to 1.3-34544

  • Debian: Add Alestic PPA to apt sources

  • Debian: Install runurl from Alestic PPA

I would like to reiterate that these AMIs are not recommended for anybody except folks who are already using older versions of these AMIs and I would encourage you to upgrade to an Ubuntu 9.10 Karmic AMI or the upcoming Ubuntu 10.04 Lucid AMI which will be released in April. The base Ubuntu image building responsibilities have been transfered to Canonical and I don’t have any plans to release new AMIs in the older Ubuntu or Debian series beyond what I am announcing here.

I have not yet posted the AMI ids on http://alestic.com pending testing and feedback, but here are the ids for those who need to run them:

us-east-1

ami-1116f978 Ubuntu 9.04 Jaunty   server  32-bit
ami-e116f988 Ubuntu 9.04 Jaunty   server  64-bit
ami-cd16f9a4 Ubuntu 9.04 Jaunty   desktop 32-bit
ami-c316f9aa Ubuntu 9.04 Jaunty   desktop 64-bit

ami-1316f97a Ubuntu 8.10 Intrepid server  32-bit
ami-e316f98a Ubuntu 8.10 Intrepid server  64-bit
ami-cb16f9a2 Ubuntu 8.10 Intrepid desktop 32-bit
ami-c116f9a8 Ubuntu 8.10 Intrepid desktop 64-bit

ami-e916f980 Ubuntu 8.04 Hardy    server  32-bit
ami-e716f98e Ubuntu 8.04 Hardy    server  64-bit
ami-1716f97e Ubuntu 8.04 Hardy    desktop 32-bit
ami-e516f98c Ubuntu 8.04 Hardy    desktop 64-bit

ami-eb16f982 Ubuntu 6.06 Dapper   server  32-bit
ami-f916f990 Ubuntu 6.06 Dapper   server  64-bit

ami-ed16f984 Debian 5.0 Lenny     server  32-bit
ami-fb16f992 Debian 5.0 Lenny     server  64-bit
ami-cf16f9a6 Debian 5.0 Lenny     desktop 32-bit
ami-c516f9ac Debian 5.0 Lenny     desktop 64-bit

ami-ef16f986 Debian 4.0 Etch      server  32-bit
ami-fd16f994 Debian 4.0 Etch      server  64-bit

us-west-1

ami-197a2b5c Ubuntu 9.04 Jaunty   server  32-bit
ami-237a2b66 Ubuntu 9.04 Jaunty   server  64-bit
ami-357a2b70 Ubuntu 9.04 Jaunty   desktop 32-bit
ami-c97a2b8c Ubuntu 9.04 Jaunty   desktop 64-bit

ami-257a2b60 Ubuntu 8.10 Intrepid server  32-bit
ami-2d7a2b68 Ubuntu 8.10 Intrepid server  64-bit
ami-377a2b72 Ubuntu 8.10 Intrepid desktop 32-bit
ami-d57a2b90 Ubuntu 8.10 Intrepid desktop 64-bit

ami-277a2b62 Ubuntu 8.04 Hardy    server  32-bit
ami-2f7a2b6a Ubuntu 8.04 Hardy    server  64-bit
ami-cf7a2b8a Ubuntu 8.04 Hardy    desktop 32-bit
ami-d77a2b92 Ubuntu 8.04 Hardy    desktop 64-bit

ami-217a2b64 Ubuntu 6.06 Dapper   server  32-bit
ami-e77a2ba2 Ubuntu 6.06 Dapper   server  64-bit

ami-d37a2b96 Debian 5.0 Lenny     server  32-bit
ami-df7a2b9a Debian 5.0 Lenny     server  64-bit
ami-e37a2ba6 Debian 5.0 Lenny     desktop 32-bit
ami-db7a2b9e Debian 5.0 Lenny     desktop 64-bit

ami-d17a2b94 Debian 4.0 Etch      server  32-bit
ami-dd7a2b98 Debian 4.0 Etch      server  64-bit

eu-west-1

ami-a798b3d3 Ubuntu 9.04 Jaunty   server  32-bit
ami-af98b3db Ubuntu 9.04 Jaunty   server  64-bit
ami-9798b3e3 Ubuntu 9.04 Jaunty   desktop 32-bit
ami-9d98b3e9 Ubuntu 9.04 Jaunty   desktop 64-bit

ami-a198b3d5 Ubuntu 8.10 Intrepid server  32-bit
ami-a998b3dd Ubuntu 8.10 Intrepid server  64-bit
ami-9198b3e5 Ubuntu 8.10 Intrepid desktop 32-bit
ami-9998b3ed Ubuntu 8.10 Intrepid desktop 64-bit

ami-a398b3d7 Ubuntu 8.04 Hardy    server  32-bit
ami-ab98b3df Ubuntu 8.04 Hardy    server  64-bit
ami-9398b3e7 Ubuntu 8.04 Hardy    desktop 32-bit
ami-8798b3f3 Ubuntu 8.04 Hardy    desktop 64-bit

ami-ad98b3d9 Ubuntu 6.06 Dapper   server  32-bit
ami-9598b3e1 Ubuntu 6.06 Dapper   server  64-bit

ami-8398b3f7 Debian 5.0 Lenny     server  32-bit
ami-8f98b3fb Debian 5.0 Lenny     server  64-bit
ami-8998b3fd Debian 5.0 Lenny     desktop 32-bit
ami-8b98b3ff Debian 5.0 Lenny     desktop 64-bit

ami-8198b3f5 Debian 4.0 Etch      server  32-bit
ami-8d98b3f9 Debian 4.0 Etch      server  64-bit

Ubuntu AMIs

Ubuntu AMIs for EC2:


AWS Jobs

AWS Jobs