Canonical has released updated instance-store AMIs for Ubuntu 8.04 LTS Hardy on Amazon EC2. Read Ben Howard’s announcement on the ec2ubuntu Google group.
Canonical has released updated instance-store AMIs for Ubuntu 8.04 LTS Hardy on Amazon EC2. Read Ben Howard’s announcement on the ec2ubuntu Google group.
For folks still using the old, reliable Ubuntu 8.04 LTS Hardy from 2008, Canonical has released updated AMIs for use on Amazon EC2. Read Scott Moser’s announcement on the ec2ubuntu Google group.
Though Canonical publishes both EBS boot and instance-store for recent Ubuntu releases, they only publish instance-store AMIs for the older Ubuntu 8.04, so…
As steady as clockwork, Ubuntu 11.04 Natty is released on the day scheduled at least eleven months ago; and thanks to Canonical, tested AMIs for Natty are already published for use on Amazon EC2.
Update: Since this article was written, Amazon has released the ability to copy EBS boot AMIs between regions using the web console, command line, and API. You may still find information of use in this article, but Amazon has solved some of the harder parts for you.
Using Amazon EC2, you created an EBS boot AMI and it’s working fine, but now you want to run instances of that AMI in a different EC2 region. Since AMIs are region specific, you need a copy of the image in each region where instances are required.
This article presents one method you can use to copy an EBS boot from one EC2 region to another.
Ubuntu 9.04 Jaunty has reached EOL (End Of Life). It is no longer supported by Ubuntu with security updates and patches. You have known this day was coming for 1.5 years, as all non-LTS Ubuntu releases are supported for only 18 months.
I have no plans to delete the Ubuntu 9.04 Jaunty AMIs for EC2 published under the Alestic name in the foreseeable future, but I request, recommend, and urge you to please stop using them and upgrade to an officially supported, active, kernel-consistent release of Ubuntu on EC2 like 10.04 LTS Lucid or 10.10 Maverick.
Canonical has released an updated series of Ubuntu AMIs for EC2. When starting new EC2 instances, you should use the latest AMI ids to pick up kernel security fixes. If you have Ubuntu 10.04 running on a t1.micro instance type, you should at least upgrade the software packages to get the patch for the rebooting issue:
As Scott Moser announced, Canonical has released updated AMIs (Amazon Machine Images) for running Ubuntu 10.04 Lucid, Ubuntu 9.10 Karmic, and Ubuntu 8.04 Hardy on Amazon EC2.
The table at the top of Alestic.com has been updated to reflect the new AMI ids. In fact, I have automated this table so that it will update itself without manual intervention whenever new AMIs are released by Canonical, thanks to their AMI query API.
If you’re interested in automatically retrieving AMI ids for your own uses, here’s a sample URL for the released AMI ids for Ubuntu 10.04 Lucid (server) in an easy to parse format:
http://uec-images.ubuntu.com/query/lucid/server/released.current.txt
Other releases can be retrieved from similar URLs, or you can traverse the available results starting at the root:
You can get pointers to the daily AMI builds through this API, but I recommend sticking to the “released.current” lists for the best tested and most stable AMIs. These are the AMI ids that are listed in the table at the top of Alestic.com for easy browsing, along with the legacy AMIs which I published as “Alestic” for older releases not supported by Canonical.
Amazon EC2 just launched the Asia Pacific region with data centers in Singapore.
The standard Ubuntu and Debian AMIs (Amazon Machine Images) from Canonical and Alestic are already available in this new region. I have listed the new AMI ids in the table at the top of Alestic.com.
To see the AMIs, simply click on the ap-southeast-1 tab in the table at the top.
If you’re just getting into using Ubuntu on Amazon EC2, consider joining the EC2 Ubuntu group for community support.
Ubuntu 10.04 Lucid was released on Amazon EC2 right on schedule today along with the rest of the normal Ubuntu image download channels.
Congrats to all the Ubuntu folks who were involved in putting this together! Though I’ve enjoyed contributing to the cause over the years, I’m happy to say that I had very little involvement with this release on EC2, leaving me more time to focus on my startup and family. Though many individuals are involved, I’d like to acknowledge the hard work of Scott Moser who has taken the point for publishing official Ubuntu EC2 images these last couple releases.
This is also the first Ubuntu release on EC2 that includes officially supported EBS boot AMIs, taking yet another task off my plate and providing a trusted source for this useful image type.
I’ve listed all of the current Ubuntu and Debian AMI ids in the table at the top of Alestic.com. Simply click on the EC2 region where you want to run your instances to find the AMI ids.
If you’re running an older release of Ubuntu on EC2, see my recommended upgrade paths.
Ubuntu 10.04 Lucid is expected to be released this Thursday (April 29, 2010). This is the first LTS (long term support) release since Ubuntu 8.04 Hardy, which has itself been a stable and reliable server platform for years both inside and outside of Amazon EC2.
If you’re starting a new server project on EC2, Ubuntu 10.04 Lucid would be a great release to start with. It will be supported for five years with security patches and major bug fixes.
If you are already running a different Ubuntu release on EC2, you should consider planning an upgrade to Lucid. Here are my thoughts for each release you might currently be running on EC2:
Ubuntu 8.04 Hardy should continue to be fairly stable for a while, though there does not appear to be a lot of EC2-specific development being applied to the AMIs, so you may want to upgrade to Lucid LTS at your leisure. You will probably want to start new instances, though with Scott’s comment below, it might be possible to upgrade in-place on EBS boot from Hardy to Lucid. If you happen to still be running the Alestic Hardy AMIs, you should at least consider upgrading to the Hardy AMIs published by Canonical and listed at the top of Alestic.com.
Ubuntu 8.10 Intrepid has reached its end of life and should no longer be used on EC2. If you are still running Intrepid, you should immediately start upgrading to Karmic or Intrepid through starting new instances.
Ubuntu 9.04 Jaunty on EC2 is still running on an old Fedora 8 kernel and is not supported by Ubuntu. I’ve been recommending an upgrade from Jaunty for a while. I continue to recommend upgrading to Karmic (if you want something that has been tested in production for a while) or Lucid (once you’re comfortable with the release).
Ubuntu 9.10 Karmic still works great on EC2, though there are a couple bothersome bugs (like hostname reset on reboot) that should be fixed by upgrading to Lucid. If you are running an EBS boot Karmic instance, then it is possible to upgrade in place, but it’s also fine to simply start new instances.
Ubuntu 10.04 Lucid pre-release instances should be upgraded to the released version. If you are running EBS boot Lucid, then it is possible to upgrade in place even if there are a newer kernel and ramdisk.
Scott Moser of Canonical has laid out the steps necessary to upgrade an EC2 instance in place without having to start a new EC2 instance. This only works for instances that are currently running an EBS boot pre-release Lucid AMI or an EBS boot 9.10 Karmic AMI. All other servers will need to be upgraded by starting new Lucid instances to replace the old instances. Fortunately, EC2 makes that easy.
In fact, EC2 makes starting new replacement instances so convenient, I would recommend following that procedure in most cases even if you think you can upgrade a server in place. If you encounter any troubles in getting your service to run on Lucid, running a new instance makes it straight forward to fall back to the old server. And, once you are happy with the new server, make it your production system, terminate the old one and don’t look back.
Welcome to the new world of cloud computing where you can instantly acquire and discard server hardware for mere dollars without worrying about large investments, complicated provisioning, and long term support costs.
I will add the new Lucid AMI ids to the table at the top of Alestic.com as soon as I get word that they are ready.
Note: I do not recommend that new users start with these AMIs. These AMIs run with older versions of Amazon’s Fedora 8 kernel which have some incompatibilities with Ubuntu and Debian (e.g., XFS is broken). My strong recommendation is for all users to convert to one of the Ubuntu 9.10 Karmic or Ubuntu 8.04 Hardy AMIs which run with a modern, compatible Ubuntu kernel.
I have released updates of the following legacy Ubuntu and Debian S3-based AMIs published in the “alestic” buckets:
I welcome testing and feedback from folks who are already using the older versions of these AMIs.
In addition to upgraded software packages, the following enhancements are in this release:
Wait for meta-data before deciding whether to generate ssh host key (Thanks to Dmitry for catching)
Patch from Tom White to support compressed user-data scripts
Allow user-data script to remove the “been run” file so that it is run on every boot instead of just the first
Upgrade EC2 AMI tools to 1.3-34544
Debian: Add Alestic PPA to apt sources
Debian: Install runurl from Alestic PPA
I would like to reiterate that these AMIs are not recommended for anybody except folks who are already using older versions of these AMIs and I would encourage you to upgrade to an Ubuntu 9.10 Karmic AMI or the upcoming Ubuntu 10.04 Lucid AMI which will be released in April. The base Ubuntu image building responsibilities have been transfered to Canonical and I don’t have any plans to release new AMIs in the older Ubuntu or Debian series beyond what I am announcing here.
I have not yet posted the AMI ids on https://alestic.com pending testing and feedback, but here are the ids for those who need to run them:
us-east-1
ami-1116f978 Ubuntu 9.04 Jaunty server 32-bit
ami-e116f988 Ubuntu 9.04 Jaunty server 64-bit
ami-cd16f9a4 Ubuntu 9.04 Jaunty desktop 32-bit
ami-c316f9aa Ubuntu 9.04 Jaunty desktop 64-bit
ami-1316f97a Ubuntu 8.10 Intrepid server 32-bit
ami-e316f98a Ubuntu 8.10 Intrepid server 64-bit
ami-cb16f9a2 Ubuntu 8.10 Intrepid desktop 32-bit
ami-c116f9a8 Ubuntu 8.10 Intrepid desktop 64-bit
ami-e916f980 Ubuntu 8.04 Hardy server 32-bit
ami-e716f98e Ubuntu 8.04 Hardy server 64-bit
ami-1716f97e Ubuntu 8.04 Hardy desktop 32-bit
ami-e516f98c Ubuntu 8.04 Hardy desktop 64-bit
ami-eb16f982 Ubuntu 6.06 Dapper server 32-bit
ami-f916f990 Ubuntu 6.06 Dapper server 64-bit
ami-ed16f984 Debian 5.0 Lenny server 32-bit
ami-fb16f992 Debian 5.0 Lenny server 64-bit
ami-cf16f9a6 Debian 5.0 Lenny desktop 32-bit
ami-c516f9ac Debian 5.0 Lenny desktop 64-bit
ami-ef16f986 Debian 4.0 Etch server 32-bit
ami-fd16f994 Debian 4.0 Etch server 64-bit
us-west-1
ami-197a2b5c Ubuntu 9.04 Jaunty server 32-bit
ami-237a2b66 Ubuntu 9.04 Jaunty server 64-bit
ami-357a2b70 Ubuntu 9.04 Jaunty desktop 32-bit
ami-c97a2b8c Ubuntu 9.04 Jaunty desktop 64-bit
ami-257a2b60 Ubuntu 8.10 Intrepid server 32-bit
ami-2d7a2b68 Ubuntu 8.10 Intrepid server 64-bit
ami-377a2b72 Ubuntu 8.10 Intrepid desktop 32-bit
ami-d57a2b90 Ubuntu 8.10 Intrepid desktop 64-bit
ami-277a2b62 Ubuntu 8.04 Hardy server 32-bit
ami-2f7a2b6a Ubuntu 8.04 Hardy server 64-bit
ami-cf7a2b8a Ubuntu 8.04 Hardy desktop 32-bit
ami-d77a2b92 Ubuntu 8.04 Hardy desktop 64-bit
ami-217a2b64 Ubuntu 6.06 Dapper server 32-bit
ami-e77a2ba2 Ubuntu 6.06 Dapper server 64-bit
ami-d37a2b96 Debian 5.0 Lenny server 32-bit
ami-df7a2b9a Debian 5.0 Lenny server 64-bit
ami-e37a2ba6 Debian 5.0 Lenny desktop 32-bit
ami-db7a2b9e Debian 5.0 Lenny desktop 64-bit
ami-d17a2b94 Debian 4.0 Etch server 32-bit
ami-dd7a2b98 Debian 4.0 Etch server 64-bit
eu-west-1
ami-a798b3d3 Ubuntu 9.04 Jaunty server 32-bit
ami-af98b3db Ubuntu 9.04 Jaunty server 64-bit
ami-9798b3e3 Ubuntu 9.04 Jaunty desktop 32-bit
ami-9d98b3e9 Ubuntu 9.04 Jaunty desktop 64-bit
ami-a198b3d5 Ubuntu 8.10 Intrepid server 32-bit
ami-a998b3dd Ubuntu 8.10 Intrepid server 64-bit
ami-9198b3e5 Ubuntu 8.10 Intrepid desktop 32-bit
ami-9998b3ed Ubuntu 8.10 Intrepid desktop 64-bit
ami-a398b3d7 Ubuntu 8.04 Hardy server 32-bit
ami-ab98b3df Ubuntu 8.04 Hardy server 64-bit
ami-9398b3e7 Ubuntu 8.04 Hardy desktop 32-bit
ami-8798b3f3 Ubuntu 8.04 Hardy desktop 64-bit
ami-ad98b3d9 Ubuntu 6.06 Dapper server 32-bit
ami-9598b3e1 Ubuntu 6.06 Dapper server 64-bit
ami-8398b3f7 Debian 5.0 Lenny server 32-bit
ami-8f98b3fb Debian 5.0 Lenny server 64-bit
ami-8998b3fd Debian 5.0 Lenny desktop 32-bit
ami-8b98b3ff Debian 5.0 Lenny desktop 64-bit
ami-8198b3f5 Debian 4.0 Etch server 32-bit
ami-8d98b3f9 Debian 4.0 Etch server 64-bit
Scott Moser (Canonical) built and released new Ubuntu 8.04.3 LTS Hardy images and AMIs for Amazon EC2. I also published new EBS boot AMIs using the same images. I’ve listed all of the AMI ids on https://alestic.com (pick your region at the top).
These AMIs should work better in the us-west-1 region (apt sources.list) and have updated software packages so upgrades on new instances should be faster.
New updates have been released for the Ubuntu and Debian AMIs (EC2 images) published on:
The following notes apply to this release:
The EC2 AMI tools have been upgraded to version 1.3-34544. Note that an “apt-get upgrade” will downgrade the EC2 AMI tools because the versions of the ec2-ami-tools package currently in the Ubuntu Hardy, Intrepid, Jaunty archives are outdated. If you have an easy solution to this, please let us know.
The ssh host key regeneration has been moved to run after the RNG is seeded. This improves security, especially for folks who are verifying the ssh host key on the first connect by comparing the fingerprint to the value in the instance console output. Thanks to Andrew Becherer for suggesting this improvement.
The Ubuntu Karmic Alpha images were not updated. Due to a new kernel requirement, Karmic will no longer run on Amazon’s 2.6.21 kernel. Once Canonical releases an Ubuntu kernel with the appropriate features, the Karmic series may be resumed. Running “apt-get upgrade” on an existing Karmic AMI will cause it to be inaccessible after rebooting.
The Debian Etch desktop images were not updated because apparently LaTeX refuses to build a format from a source file which is more than five years old. If there is anybody who wants Etch desktops and is willing to investigate, please contact me, otherwise I plan to discontinue support for this series. If you’re using Debian, I’d encourage you to upgrade to Lenny “stable” anyway.
Please give these new images a spin and let us know if you run into any problems.
Enjoy
NOTE: This is an article from 2009, back before EBS boot instances were available on Amazon EC2. I recommend you use EBS boot instances which make it trivial to create new AMIs (single command/API call). Please stop reading this article now and convert to EBS boot AMIs!
When you start up an instance (server) on Amazon EC2, you need to pick the image or AMI (Amazon Machine Image) to run. This determines the Linux distribution and version as well as the initial software installed and how it is configured.
There are a number of public images to choose from with EC2 including the Ubuntu and Debian image published on https://alestic.com but sometimes it is appropriate to create your own private or public images. There are two primary ways to create an image for EC2:
Create an EC2 image from scratch. This process lets you control every detail of what goes into the image and is the easiest way to automate image creation.
Rebundle a running EC2 instance into a new image. This approach is the topic of the rest of this article.
After you rebundle a running instance to create a new image, you can then run new EC2 instances of that image. Each instance starts off looking exactly like the original instance as far as the files on the disk go (with a few exceptions).
This guide is primarily written in the context of running Ubuntu on EC2, but the concepts should apply without too much changing on Debian and other Linux distributions.
To use this rebundling approach, you start by running an instance of an image that (1) is as close as possible to the image you want to create, and (2) is published by a source you trust. You then proceed to install software and configure that instance so that it contains exactly what you want to be available on new instances right down to the startup scripts.
The next step is to bundle the instance’s disk image into a new AMI, but before we get to that, it is important to understand a few things about security.
If you are creating a new EC2 image, you need to be very careful what pieces of information you inadvertently leave on the image, especially if you have the goal of publishing it as a public AMI. Anybody who runs an instance of that AMI will have access to the files you included in the bundle, and there is no way to modify an AMI after it has been created (though you can delete it).
For example, you don’t want to leave your AWS certificate or private key on the disk. You’ll even want to clear out the shell history file in case you had typed secret information in commands or in setting environment variables.
You also want to consider the security concerns from the perspective of the people who run the new image. For example, you don’t want to leave any passwords active on accounts. You should also make sure you don’t include your public ssh key in authorized_keys files. Leaving a back door into other people’s servers is in poor taste even if you have no intention of ever using it.
Here are some sample commands, but only you can decide if this wipes out too much or what other files you need to exclude depending on how you set up and used the instance you are bundling:
sudo rm -f /root/.*hist* $HOME/.*hist*
sudo rm -f /var/log/*.gz
sudo find /var/log -name mysql -prune -o -type f -print |
while read i; do sudo cp /dev/null $i; done
Whole directories can be excluded from the image using the --exclude option of the ec2-bundle-vol command (see below).
Now we’re ready to bundle the actual EC2 image (AMI). To start, you need to copy your certificate and key to the instance ephemeral storage. Adjust the sample command to use the appropriate keypair file for authentication and the appropriate location of your certification and private key files. If you are not running a modern Ubuntu image, then change remoteuser to “root”.
remotehost=<ec2-instance-hostname>
remoteuser=ubuntu
rsync \
--rsh="ssh -i KEYPAIR.pem" \
--rsync-path="sudo rsync" \
PATHTOKEYS/{cert,pk}-*.pem \
$remoteuser@$remotehost:/mnt/
Set up some environment variables for convenience in the following commands. A single S3 bucket can be used for multiple AMIs. The manifest prefix should be descriptive, especially if you plan to publish the AMI publicly, as it is the only piece of documentation many users will see when they look through AMI lists. At a minimum, I recommend including the Linux distribution (e.g, “ubuntu”), the architecture (e.g., “i386” or “32”), and the date (e.g., “20090621”), as well as some tag that indicates the special nature of the image (e.g., “desktop” or “lamp”).
bucket=<your-bucket-name>
prefix=<descriptive-image-title>
On the EC2 instance itself, you also set up some environment variables to help the bundle and upload commands. You can find these values in your EC2 account.
export AWS_USER_ID=<your-value>
export AWS_ACCESS_KEY_ID=<your-value>
export AWS_SECRET_ACCESS_KEY=<your-value>
if [ $(uname -m) = 'x86_64' ]; then
arch=x86_64
else
arch=i386
fi
Bundle the files on the current instance into a copy of the image under /mnt:
sudo -E ec2-bundle-vol \
-r $arch \
-d /mnt \
-p $prefix \
-u $AWS_USER_ID \
-k /mnt/pk-*.pem \
-c /mnt/cert-*.pem \
-s 10240 \
-e /mnt,/root/.ssh,/home/ubuntu/.ssh
Upload the bundle to a bucket on S3:
ec2-upload-bundle \
-b $bucket \
-m /mnt/$prefix.manifest.xml \
-a $AWS_ACCESS_KEY_ID \
-s $AWS_SECRET_ACCESS_KEY
Now that the AMI files have been uploaded to S3, you register the image as a new AMI. This is done back on your local system (with the API tools installed):
ec2-register \
--name "$bucket/$prefix" \
$bucket/$prefix.manifest.xml
The output of this command is the new AMI id which is used to run new instances of that image.
It is important to use the same account access information for the ec2-bundle-vol and ec2-register commands even though they are run on different systems. If you don’t you’ll get an error indicating you don’t have the rights to register the image.
By default, the new EC2 image is private, which means it can only be seen and run by the user who created it. You can share access with another individual account or with the public.
To let another EC2 user run the image without giving access to the world:
ec2-modify-image-attribute -l -a <other-user-id> <ami-id>
To let all other EC2 users run instances of your image:
ec2-modify-image-attribute -l -a all <ami-id>
AWS will charge you standard S3 charges for the stored AMI files which comes out to $0.15 per GB per month. Note, however, that the bundling process uses sparse files and compression, so the final storage size is generally very small and your resulting cost may only be pennies per month.
The AMI owner incurs no charge when users run the image in new instances. The users who run the AMI are responsible for the standard hourly instance charges.
Before removing any public image, please consider the impact this might have on people who depend on that image to run their business. Once you publish an AMI, there is no way to tell how many users are regularly creating instances of that AMI and expecting it to stay available. There is also no way to communicate with these users to let them know that the image is going away.
If you decide you want to remove an image anyway, here are the steps to take.
Deregister the AMI
ec2-deregister ami-XXX
Delete the AMI bundle in S3:
ec2-delete-bundle \
--access-key $AWS_ACCESS_KEY_ID \
--secret-key $AWS_SECRET_ACCESS_KEY \
--bucket $bucket \
--prefix $prefix
[Update 2009-09-12: Security tweak for running under non-root.] [Update 2010-02-01: Update to use latest API/AMI tools and work for Ubuntu 9.10 Karmic.]
Ubuntu Karmic Koala Alpha is being developed and will be released as Ubuntu 9.10 in October. If you want to play around with Karmic Alpha on Amazon EC2, I have published new AMIs in the US and EU regions for 32- and 64-bit:
A Karmic desktop image for EC2 is also available if you wish to monitor progress in that area.
Warning! Karmic is an unstable alpha developer version and is not intended for use in anything resembling a production environment.
Please note that we are still defaulting to Amazon’s 2.6.21fc8 kernel which, though functional and stable, is getting older and older for each new release of Ubuntu. One effect of this is that AppArmor will not be enabled, though this should not affect the functionality of any software.
Enjoy!