Alestic.com - A Personal AWS Blog by Eric Hammond

As Scott Moser announced, Canonical has released updated AMIs (Amazon Machine Images) for running Ubuntu 10.04 Lucid, Ubuntu 9.10 Karmic, and Ubuntu 8.04 Hardy on Amazon EC2.

The table at the top of Alestic.com has been updated to reflect the new AMI ids. In fact, I have automated this table so that it will update itself without manual intervention whenever new AMIs are released by Canonical, thanks to their AMI query API.

If you’re interested in automatically retrieving AMI ids for your own uses, here’s a sample URL for the released AMI ids for Ubuntu 10.04 Lucid (server) in an easy to parse format:

http://uec-images.ubuntu.com/query/lucid/server/released.current.txt

Other releases can be retrieved from similar URLs, or you can traverse the available results starting at the root:

http://uec-images.ubuntu.com/query/

You can get pointers to the daily AMI builds through this API, but I recommend sticking to the “released.current” lists for the best tested and most stable AMIs. These are the AMI ids that are listed in the table at the top of Alestic.com for easy browsing, along with the legacy AMIs which I published as “Alestic” for older releases not supported by Canonical.

If you’ve been following along, you probably know that I have been recommending that folks using EC2 switch to the official Ubuntu AMIs published by Canonical (Hardy or Karmic). I have been building and publishing Ubuntu AMIs since 2007 (including Dapper, Edgy, Feisty, Gutsy, Hardy, Intrepid, Karmic), but the last year my focus on this project has been to transition these responsibilities to Canonical who have more time and resources to support the initiative.

I’m happy to say that I’ve finally followed my own advice. For my personal Amazon EC2 servers (including for the Alestic.com web site) I am using Ubuntu 9.10 Karmic images published for EC2 by Canonical.

While I was making the transition, I also switched to EBS boot AMIs. However, since it sounds like Canonical is not planning to publish EBS boot AMIs until Lucid, I decided to continue in service to the community and make available EBS boot AMIs for running Ubuntu on EC2.

I have published EBS boot AMIs for Ubuntu 9.10 Karmic and Ubuntu 8.04 Hardy, both 32- and 64-bit architectures, in all current EC2 regions, for a total of a dozen new AMIs.

I chose to use the exact Ubuntu images which Canonical built for running Ubuntu on EC2. This means that these EBS boot AMIs work exactly the same as the official Canonical AMIs including ssh to the ubuntu user. Again, even though I’m publishing the EBS boot AMIs for Karmic and Hardy, the contents of the image were built by Canonical.

The EBS boot AMIs are listed on Alestic.com. I have restructured the table to better feature Canonical AMIs, and now you need to pick an EC2 region to see the IDs.

Give the EBS boot AMIs a spin and let me know if you run into any issues.

As Thilo Maier pointed out in comments on my request for UDS input, I have been publishing both server and desktop AMIs for running Ubuntu on EC2 up through Jaunty, but the official Karmic AMIs on EC2 only support server installations by default.

Ubuntu makes it pretty easy to install the desktop software on a server, and NX from NoMachine makes it pretty easy to access that desktop remotely, with near real-time interactivity even over slowish connections.

Here’s a quick guide to setting this up, starting with an Ubuntu 9.10 Karmic AMI on Amazon EC2:

1. Create a user-data script which installs runurl (not on Karmic AMIs by default) and then runs a simple desktop and NX server installation script. Examine the desktop script to see what it’s doing to install the software.

    cat <<EOM >install-desktop
    #!/bin/bash -ex
    wget -qO/usr/bin/runurl run.alestic.com/runurl
    chmod 755 /usr/bin/runurl
    runurl run.alestic.com/install/desktop
    EOM
   

2. Start an instance on EC2 telling it to run the above user-data script on first boot. The following example uses the current 32-bit Karmic server AMI. Make sure you’re using the latest AMI id.

    ec2-run-instances                   \
      --key YOURKEY                     \
      --user-data-file install-desktop  \
      ami-1515f67c
   

3. Connect to the new instance and wait for it to complete the desktop software installation (when sshd is restarted). This takes about 30 minutes on an m1.small instance and 10 minutes on a c1.medium instance. Then generate and set a secure password for the ubuntu user using copy/paste from the pwgen output. Save the secure password so you can enter it into the NX client later.

    ssh -i YOURKEY.pem ubuntu@THEHOST
    tail -f /var/log/syslog | egrep --line-buffer user-data:
    pwgen -s 16 1
    sudo passwd ubuntu
   

   If anybody knows how to use ssh keys with NX, I’d love to do this instead of using passwords.

4. Back on your local system, install and run the NX client. For computers not running Ubuntu, download the appropriate software from NoMachine.

    wget http://64.34.161.181/download/3.4.0/Linux/nxclient_3.4.0-5_i386.deb
    sudo dpkg -i nxclient_3.4.0-5_i386.deb
    /usr/NX/bin/nxclient --wizard &
   

   Point the NX Client to the external hostname of your EC2 instance. Enter the Login “ubuntu” and the Password from above. Choose the “Gnome” desktop.

If all goes well, you should have a complete and fresh Ubuntu desktop filling most of your screen, available for you to mess around with and then throw away.

ec2-terminate-instances INSTANCEID

If you want to have a persistent desktop with protection from crashes, you’ll need to learn how to do things like placing critical directories on EBS volumes.

If you’d like to run KDE on EC2, replace the package “ubuntu-desktop” with “kubuntu-desktop” in the installation script.

WARNING!

Though most Ubuntu 9.04 Jaunty systems can upgrade to 9.10 Karmic in place, this is not possible on EC2 and should not be attempted. If you do try this, your system will become unusable on reboot and there will be no recovery and no access to any of the data on the boot disk or ephemeral storage.

Here’s why:

• Ubuntu 9.10 Karmic has a version of udev which requires a newer kernel than you would be running for Ubuntu 9.04 Jaunty (especially on EC2).

• You cannot upgrade the kernel used by a running instance on Amazon EC2 (not even rebooting).

• When an EC2 instance cannot boot (as in the case of the udev/kernel mismatch) your only option is to terminate it, losing the local storage.

How To Upgrade

In order to upgrade to Karmic you will need to start a new EC2 instance running a fresh copy of the appropriate Karmic AMI. I post the latest AMI ids for Karmic in the second table on https://alestic.com/.

Keep your old instance(s) running while you configure and test the new Karmic instances. EC2 makes it easy to have multiple sets of servers running in parallel instead of upgrading in place. When you are confident your new servers are functioning properly, you can discard the old ones.

The Ubuntu 9.10 Karmic AMIs released by Canonical have a number of differences from the community Ubuntu AMIs which have been published on https://alestic.com.

One of the biggest differences is that you will ssh to ubuntu@ instead of to root@ on your instance. You can then sudo to perform commands as the root user. Back in April I wrote a guide about Using sudo, ssh, rsync on the Official Ubuntu Images for EC2.

The Ubuntu server team has put a lot of work into making Ubuntu 9.10 Karmic function beautifully on Amazon EC2 and it’s been a pleasure to have a small part in the process. I’m already using the Karmic AMIs on EC2 for one of my production processes. Please give these AMIs a spin and give feedback.

Ubuntu Karmic Koala Alpha is being developed and will be released as Ubuntu 9.10 in October. If you want to play around with Karmic Alpha on Amazon EC2, I have published new AMIs in the US and EU regions for 32- and 64-bit:

https://alestic.com

A Karmic desktop image for EC2 is also available if you wish to monitor progress in that area.

Warning! Karmic is an unstable alpha developer version and is not intended for use in anything resembling a production environment.

Please note that we are still defaulting to Amazon’s 2.6.21fc8 kernel which, though functional and stable, is getting older and older for each new release of Ubuntu. One effect of this is that AppArmor will not be enabled, though this should not affect the functionality of any software.

Enjoy!

Canonical has released official Ubuntu images for EC2 for Ubuntu 9.10 Karmic.

The primary technical benefit brought by Canonical's involvement in building official Ubuntu images is that custom kernels can be built for EC2 through a relationship with Amazon. This means that the Ubuntu images can now run on more modern Ubuntu kernels instead of on Amazon's older, Fedora kernels.

Other differences are listed below:

Items listed are likely to change as images are enhanced. This table may or may not be updated to match. Please leave comments if you notice or question other differences.

Note: There are some older (2009-04) Canonical AMIs floating around for Hardy and Intrepid. These have not been maintained and are not recommended at this point.

Updated 2009-06-15: Alestic.com Jaunty is using an Ubuntu mirror inside EC2. Alestic.com images using load balanced mirror with failover between EC2 availability zones.

Updated 2009-06-25: Alestic.com published Karmic (Alpha) but later withdrew.

Updated 2009-10-29: Canonical released Karmic. None of the image currently have RightScale support built in, but RightScale has their own Ubuntu AMIs.