Alestic.com - A Personal AWS Blog by Eric Hammond

Canonical has released an updated series of Ubuntu AMIs for EC2. When starting new EC2 instances, you should use the latest AMI ids to pick up kernel security fixes. If you have Ubuntu 10.04 running on a t1.micro instance type, you should at least upgrade the software packages to get the patch for the rebooting issue:

Amazon EC2 just launched the Asia Pacific region with data centers in Singapore.

The standard Ubuntu and Debian AMIs (Amazon Machine Images) from Canonical and Alestic are already available in this new region. I have listed the new AMI ids in the table at the top of Alestic.com.

To see the AMIs, simply click on the ap-southeast-1 tab in the table at the top.

If you’re just getting into using Ubuntu on Amazon EC2, consider joining the EC2 Ubuntu group for community support.

If you’ve been following along, you probably know that I have been recommending that folks using EC2 switch to the official Ubuntu AMIs published by Canonical (Hardy or Karmic). I have been building and publishing Ubuntu AMIs since 2007 (including Dapper, Edgy, Feisty, Gutsy, Hardy, Intrepid, Karmic), but the last year my focus on this project has been to transition these responsibilities to Canonical who have more time and resources to support the initiative.

I’m happy to say that I’ve finally followed my own advice. For my personal Amazon EC2 servers (including for the Alestic.com web site) I am using Ubuntu 9.10 Karmic images published for EC2 by Canonical.

While I was making the transition, I also switched to EBS boot AMIs. However, since it sounds like Canonical is not planning to publish EBS boot AMIs until Lucid, I decided to continue in service to the community and make available EBS boot AMIs for running Ubuntu on EC2.

I have published EBS boot AMIs for Ubuntu 9.10 Karmic and Ubuntu 8.04 Hardy, both 32- and 64-bit architectures, in all current EC2 regions, for a total of a dozen new AMIs.

I chose to use the exact Ubuntu images which Canonical built for running Ubuntu on EC2. This means that these EBS boot AMIs work exactly the same as the official Canonical AMIs including ssh to the ubuntu user. Again, even though I’m publishing the EBS boot AMIs for Karmic and Hardy, the contents of the image were built by Canonical.

The EBS boot AMIs are listed on Alestic.com. I have restructured the table to better feature Canonical AMIs, and now you need to pick an EC2 region to see the IDs.

Give the EBS boot AMIs a spin and let me know if you run into any issues.

In the last article, I described how to use the vmbuilder software to build an EBS boot AMI from scratch for running Ubuntu on EC2 with a persistent root disk.

In the ec2ubuntu Google group, Scott Moser pointed out that users can take advantage of the Ubuntu images for EC2 that Canonical has already built with vmbuilder. This can simplify and speed up the process of building EBS boot AMIs for the rest of us.

Let’s walk through the steps, creating an EBS boot AMI for Ubuntu 9.10 Karmic.

1. Run an instance of the Ubuntu 9.10 Karmic AMI, either 32-bit or 64-bit depending on which architecture AMI you wish to build. Make a note of the resulting instance id:

    # 32-bit
    instanceid=$(ec2-run-instances   \
      --key YOURKEYPAIR              \
      --availability-zone us-east-1a \
      ami-1515f67c |
      egrep ^INSTANCE | cut -f2)
    echo "instanceid=$instanceid"
   
    # 64-bit
    instanceid=$(ec2-run-instances   \
      --key YOURKEYPAIR              \
      --availability-zone us-east-1a \
      --instance-type m1.large       \
      ami-ab15f6c2 |
      egrep ^INSTANCE | cut -f2)
    echo "instanceid=$instanceid"
   

   Wait for the instance to move to the “running” state, then note the public hostname:

    while host=$(ec2-describe-instances "$instanceid" | 
      egrep ^INSTANCE | cut -f4) && test -z $host; do echo -n .; sleep 1; done
    echo host=$host
   

   Copy your X.509 certificate and private key to the instance. Use the correct locations for your credential files:

    rsync                            \
      --rsh="ssh -i YOURKEYPAIR.pem" \
      --rsync-path="sudo rsync"      \
      ~/.ec2/{cert,pk}-*.pem         \
      ubuntu@$host:/mnt/
   

   Connect to the instance:

    ssh -i YOURKEYPAIR.pem ubuntu@$host
   

2. Install EC2 API tools from the Ubuntu on EC2 ec2-tools PPA because they are more up to date than the ones in Karmic, letting us register EBS boot AMIs:

    export DEBIAN_FRONTEND=noninteractive
    echo "deb http://ppa.launchpad.net/ubuntu-on-ec2/ec2-tools/ubuntu karmic main" |
      sudo tee /etc/apt/sources.list.d/ubuntu-on-ec2-ec2-tools.list &&
    sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9EE6D873 &&
    sudo apt-get update &&
    sudo -E apt-get dist-upgrade -y &&
    sudo -E apt-get install -y ec2-api-tools
   

3. Set up some parameters:

    codename=karmic
    release=9.10
    tag=server
    if [ $(uname -m) = 'x86_64' ]; then
      arch=x86_64
      arch2=amd64
      ebsopts="--kernel=aki-fd15f694 --ramdisk=ari-c515f6ac"
      ebsopts="$ebsopts --block-device-mapping /dev/sdb=ephemeral0"
    else
      arch=i386
      arch2=i386
      ebsopts="--kernel=aki-5f15f636 --ramdisk=ari-0915f660"
      ebsopts="$ebsopts --block-device-mapping /dev/sda2=ephemeral0"
    fi
   

4. Download and unpack the latest released Ubuntu server image file. This contains the output of vmbuilder as run by Canonical.

    imagesource=http://uec-images.ubuntu.com/releases/$codename/release/unpacked/ubuntu-$release-$tag-uec-$arch2.img.tar.gz
    image=/mnt/$codename-$tag-uec-$arch2.img
    imagedir=/mnt/$codename-uec-$arch2
    wget -O- $imagesource |
      sudo tar xzf - -C /mnt
    sudo mkdir -p $imagedir
    sudo mount -o loop $image $imagedir
   

5. [OPTIONAL] At this point /mnt/$image contains a mounted filesystem with the complete Ubuntu image as released by Canonical. You can skip this step if you just want an EBS boot AMI which is an exact copy of the released S3 based Ubuntu AMI from Canonical, or you can make any updates, installations, and customizations you’d like to have in your resulting AMI.

   In this example, we’ll perform similar steps as the previous tutorial and update the software packages to the latest releases from Ubuntu. Remember that the released EC2 image could be months old.

    # Allow network access from chroot environment
    sudo cp /etc/resolv.conf $imagedir/etc/
    # Fix what I consider to be a bug in vmbuilder
    sudo rm -f $imagedir/etc/hostname
    # Add multiverse
    sudo perl -pi -e 's%(universe)$%$1 multiverse%' \
      $imagedir/etc/ec2-init/templates/sources.list.tmpl
    # Add Alestic PPA for runurl package (handy in user-data scripts)
    echo "deb http://ppa.launchpad.net/alestic/ppa/ubuntu karmic main" |
      sudo tee $imagedir/etc/apt/sources.list.d/alestic-ppa.list
    sudo chroot $imagedir \
      apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BE09C571
    # Add ubuntu-on-ec2/ec2-tools PPA for updated ec2-ami-tools
    echo "deb http://ppa.launchpad.net/ubuntu-on-ec2/ec2-tools/ubuntu karmic main" |
      sudo tee $imagedir/etc/apt/sources.list.d/ubuntu-on-ec2-ec2-tools.list
    sudo chroot $imagedir \
      apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9EE6D873
    # Upgrade the system and install packages
    sudo chroot $imagedir mount -t proc none /proc
    sudo chroot $imagedir mount -t devpts none /dev/pts
    cat <<EOF > $imagedir/usr/sbin/policy-rc.d
    #!/bin/sh
    exit 101
    EOF
    chmod 755 $imagedir/usr/sbin/policy-rc.d
    DEBIAN_FRONTEND=noninteractive
    sudo chroot $imagedir apt-get update &&
    sudo -E chroot $imagedir apt-get dist-upgrade -y &&
    sudo -E chroot $imagedir apt-get install -y runurl ec2-ami-tools
    sudo chroot $imagedir umount /proc
    sudo chroot $imagedir umount /dev/pts
    rm -f $imagedir/usr/sbin/policy-rc.d
   

   Again, the above step is completely optional and can be skipped to create the EBS boot AMI that Canonical would have published.

6. Copy the image files to a new EBS volume, snapshot it, and register the snapshot as an EBS boot AMI. Make a note of the resulting AMI id:

    size=15 # root disk in GB
    now=$(date +%Y%m%d-%H%M)
    prefix=ubuntu-$release-$codename-$tag-$arch-$now
    description="Ubuntu $release $codename $tag $arch $now"
    export EC2_CERT=$(echo /mnt/cert-*.pem)
    export EC2_PRIVATE_KEY=$(echo /mnt/pk-*.pem)
    volumeid=$(ec2-create-volume --size $size --availability-zone us-east-1a |
      cut -f2)
    instanceid=$(wget -qO- http://instance-data/latest/meta-data/instance-id)
    ec2-attach-volume --device /dev/sdi --instance "$instanceid" "$volumeid"
    while [ ! -e /dev/sdi ]; do echo -n .; sleep 1; done
    sudo mkfs.ext3 -F /dev/sdi
    ebsimage=$imagedir-ebs
    sudo mkdir $ebsimage
    sudo mount /dev/sdi $ebsimage
    sudo tar -cSf - -C $imagedir . | sudo tar xvf - -C $ebsimage
    sudo umount $ebsimage
    ec2-detach-volume "$volumeid"
    snapshotid=$(ec2-create-snapshot "$volumeid" | cut -f2)
    ec2-delete-volume "$volumeid"
    while ec2-describe-snapshots "$snapshotid" | grep -q pending
      do echo -n .; sleep 1; done
    ec2-register                   \
      --architecture $arch         \
      --name "$prefix"             \
      --description "$description" \
      $ebsopts                     \
      --snapshot "$snapshotid"
   

7. Depending on what you want to keep from the above process, there are various things that you might want to clean up.

   If you no longer want to use an EBS boot AMI:

    ec2-deregister $amiid
    ec2-delete-snapshot $snapshotid
   

   When you’re done with the original instance:

    ec2-terminate-instance $instanceid
   

In this example, I set /mnt to the first ephemeral store on the instance even on EBS boot AMIs. This more closely matches the default on the S3 based AMIs, but means that /mnt will not be persistent across a stop/start of an EBS boot instance. If Canonical starts publishing EBS boot AMIs, they may or may not choose to make the same choice.

Community feedback, bug reports, and enhancements for these instructions are welcomed.

[Update 2009-01-14: Wrapped upgrade/installs inside of /usr/sbin/policy-rc.d setting to avoid starting daemons in chroot environment.]

[Update 2010-01-22: New location for downloadable Ubuntu images.]

[Update 2010-03-26: Path tweak, thanks to paul.]

UPDATE-3: As of 2009-06-16 02:35a, Canonical has restored the Ubuntu mirror for EC2 in the US region. It looks like everything is operating normally now.

UPDATE-2: Canonical has restored the Ubuntu mirror for EC2 in the EU region.

UPDATE-1: The DNS names for the Canonical Ubuntu mirrors on EC2 have been temporarily switched to point to the Ubuntu mirrors outside of EC2. This is a great idea that gets things working again until the EC2 mirrors can be brought back up. If you really want to use mirrors inside EC2 for performance or (minor) cost considerations, you could still switch to the RightScale mirrors.

As I write this, the Ubuntu archive mirrors on EC2 run by Canonical are currently unavailable in both the US and European regions. If you are running the Ubuntu images for EC2 published by Canonical, this prevents you from being able to apt-get update or apt-get upgrade

The Canonical IS team is currently on the job investigating and correcting the issue, but if you need a quick fix in the mean time, you can run the following command on the instance to switch to a Canonical Ubuntu mirror outside of EC2 (standard EC2 network charges apply):

oldarchive='(us|eu)\.ec2\.archive\.ubuntu\.com'
newarchive='$1.archive.ubuntu.com'
sudo perl -pi.orig -e "s/$oldarchive/$newarchive/" /etc/apt/sources.list

This command saves a copy of the original file in /etc/apt/sources.list.orig so that you can copy it back when the outage is over.

Alternatively, you could switch to the Ubuntu mirror in EC2 run by RightScale:

oldarchive='[.\w]+\.archive\.ubuntu\.com'
newarchive='ec2-us-east-mirror.rightscale.com'
sudo perl -pi.orig -e "s/$oldarchive/$newarchive/" /etc/apt/sources.list

Note that RightScale does not mirror the source packages, so you might want to comment out the deb-src lines:

sudo perl -pi -e 's/^(deb-src)/#$1/' /etc/apt/sources.list

The Ubuntu images for EC2 that I publish on https://alestic.com use the RightScale Ubuntu mirrors by default and are not affected by the current outage.

Canonical has released official Ubuntu images for EC2 for Ubuntu 9.10 Karmic.

The primary technical benefit brought by Canonical's involvement in building official Ubuntu images is that custom kernels can be built for EC2 through a relationship with Amazon. This means that the Ubuntu images can now run on more modern Ubuntu kernels instead of on Amazon's older, Fedora kernels.

Other differences are listed below:

Items listed are likely to change as images are enhanced. This table may or may not be updated to match. Please leave comments if you notice or question other differences.

Note: There are some older (2009-04) Canonical AMIs floating around for Hardy and Intrepid. These have not been maintained and are not recommended at this point.

Updated 2009-06-15: Alestic.com Jaunty is using an Ubuntu mirror inside EC2. Alestic.com images using load balanced mirror with failover between EC2 availability zones.

Updated 2009-06-25: Alestic.com published Karmic (Alpha) but later withdrew.

Updated 2009-10-29: Canonical released Karmic. None of the image currently have RightScale support built in, but RightScale has their own Ubuntu AMIs.