Public EBS Boot AMIs for Ubuntu on Amazon EC2

| 27 Comments

If you’ve been following along, you probably know that I have been recommending that folks using EC2 switch to the official Ubuntu AMIs published by Canonical (Hardy or Karmic). I have been building and publishing Ubuntu AMIs since 2007 (including Dapper, Edgy, Feisty, Gutsy, Hardy, Intrepid, Karmic), but the last year my focus on this project has been to transition these responsibilities to Canonical who have more time and resources to support the initiative.

I’m happy to say that I’ve finally followed my own advice. For my personal Amazon EC2 servers (including for the Alestic.com web site) I am using Ubuntu 9.10 Karmic images published for EC2 by Canonical.

While I was making the transition, I also switched to EBS boot AMIs. However, since it sounds like Canonical is not planning to publish EBS boot AMIs until Lucid, I decided to continue in service to the community and make available EBS boot AMIs for running Ubuntu on EC2.

I have published EBS boot AMIs for Ubuntu 9.10 Karmic and Ubuntu 8.04 Hardy, both 32- and 64-bit architectures, in all current EC2 regions, for a total of a dozen new AMIs.

I chose to use the exact Ubuntu images which Canonical built for running Ubuntu on EC2. This means that these EBS boot AMIs work exactly the same as the official Canonical AMIs including ssh to the ubuntu user. Again, even though I’m publishing the EBS boot AMIs for Karmic and Hardy, the contents of the image were built by Canonical.

The EBS boot AMIs are listed on Alestic.com. I have restructured the table to better feature Canonical AMIs, and now you need to pick an EC2 region to see the IDs.

Give the EBS boot AMIs a spin and let me know if you run into any issues.

27 Comments

Hi Eric,

I am trying to get the ami-6743ae0e image up and running.

For some reason I can't get it to find the installable for mysql... haven't changed anything on the AMI - just fired it up...

I am working with:

AMI: ami-6743ae0e
Region: us-east-1b

my sources.list
$ cat sources.list
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ karmic main universe
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ karmic main universe
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ karmic-updates main universe
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ karmic-updates main universe
deb http://security.ubuntu.com/ubuntu karmic-security main universe
deb-src http://security.ubuntu.com/ubuntu karmic-security main universe

Installing mysql-server
$ sudo apt-get install -y mysql-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package mysql-server is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package mysql-server has no installation candidate

Would you be able to tell me what I am doing wrong?

It's my first running EC2 server (after the tutorial), so it's probably me doing something wrong :-)

Do I need to open anything in the security group for example?

I've got SSH, RDP and HTTP open...

Any help would be much appreciated!

And a BIG thanks for providing these images... was just about to get started building one myself, so you saved me a bunch of time!

Cheers,

Anders

Anders: Common mistake. The first thing that needs to be run on a new instance is:

sudo apt-get update

This brings in the latest package index from the repositories so that install and upgrade know which package files to work with.

Oh, yeah, that does the trick ;-)

Again, thanks a bunch for providing these images... a real timesaver!!

I thought I would try attaching the snapshot these EBS based AMI's were registered with so I could check them out, and download a copy without having to boot up an instance (or to check out an ami before booting it). However, it appears the snapshot is not public.

>snap=`ec2-describe-images -H ami-7d43ae14|awk '/sda1/ {print $3}'`
> ec2-describe-snapshots $snap
Client.InvalidSnapshot.NotFound: The snapshot 'snap-886f22e1' does not exist.

Since the data is all public in the AMI, do you think you could make the snapshot public as well?

Michael: You can download the original images[1] from Canonical, but perhaps you're checking up on me to make sure I did what I said? Fair 'nuff :)

According to a presentation at BlackHat 2009[2], knowing the bits on the image may give an attacker an advantage in predicting and guessing things like the ssh host secret key which would let them intercept and control secure connections.

This isn't an issue for most AMIs as normal users can't download the bundles from S3, but now that Canonical is publishing the images they use, it may be a concern.

Because of this, I may change a few bits[3] when copying the Canonical images for future EBS boot AMIs, and I would encourage Canonical to do the same thing for their S3 based AMIs if it turns out this really does make them more secure.

However, since I already stated these were exact copies, I've gone ahead and made the snapshots public for the "20100121" EBS boot AMIs in all regions.

Please let me know if you find any differences. They were file-wise copies and not block-wise copies in order to reduce the size of the resulting snapshot and speed up instance startup.

[1] http://uec-images.ubuntu.com/
[2] http://www.slideshare.net/astamos/cloud-computing-security (page 67)
[3] /var/lib/urandom/random-seed

Hi Eric,

Any reason why these images are not using XFS?

berenddeboer: I left out a number of "improvements" I could have made in these images. The goal here was to have an exact replica of the Ubuntu AMIs published by Canonical, except in EBS boot format.

I am a bit of newbie but also tried to create an EBS image from canonical images but sadly my procedure does not work. :(
I then read your blog further and see you create the EBS image by downloading the canonical image... whereas I was trying to create the image from the instance that was currently running. Which is a technique I have seen vaguely described in a few places.

So question.... Is this possible/feasible? I am thinking now for the future potentially, as I will most like move forward with your EBS image in the short-term.

A summary of my process:
1. Create a new EC2 volume, this needs to be at least as big as the total used size of the EC2 Instance
ec2-create-volume --size 10 --availability-zone us-east-1a

2. Attach the volume to the EC2 instance. Make a note of the device
ec2-attach-volume vol-yyyyyyyy --instance i-xxxxxxxx --device /dev/sdh

3. Format the newly added volume
dd if=/dev/zero of=/dev/sdh
yes | mkfs -t ext3 /dev/sdh

5. Mount the volume
mkdir /mnt/ebsimage
echo '/dev/sdh /mnt/ebsimage ext3 defaults,noatime 0 0' >> /etc/fstab
mount /mnt/ebsimage

6. Copy the required system and data files across to the new volume
NOTE: I tried a number of different version of the following command
rsync -rlpgoD -t -r -S -l -vh \
--exclude /ebs \
--exclude /mnt \
-X /* /mnt/ebsimage > rsync.log 2>rsync.err

# what of
#--exclude /sys \
#--exclude /proc \
#--exclude /dev \
#--exclude /media \

7. Unmount the device and detach the volume
umount /mnt/ebsimage
ec2-detach-volume vol-yyyyyyyy --instance i-xxxxxxxx

8. Take a snapshot of the volume
ec2-create-snapshot vol-yyyyyyyy

9. Register the snapshot as an EC2 AMI
ec2reg -s snap-zzzzzzzz -a {x86_64|i386} -d Description -n imagename

10. Launch and test

freremark: Yes, you can create a new AMI from a running EBS boot instance. You don't have to go through the process of creating a new volume and copying files (your steps 1-7). Simply create an EBS snapshot of your root EBS volume and register it (your steps 8-9). Make sure you specify the correct kernel and ramdisk in the registration step.

I am using public AMI ami-6743ae0e (ebs root).

I have created an instance using this ami, but after I stopped and started the instance, it lost public DNS name (in Amazon console and ssh access). Even restarting the instance did not help.

Do you know what might be the problem?

Thanks,
maks

maks: It can take a bit for the instance to start running and a new IP address to be assigned after it is started. Note that it will likely be different from the old IP address. The AWS console sometimes needs to be refreshed to pick up changes in values.

If you continue having problems, post your instance id to the EC2 forum for help: http://ec2forum.notlong.com

Eric: I'm quite new at using EC2, and probably this is a silly question and more than probably I should read more before posting questions, but here it is:

When I launch a new instance of your EBS Ubuntu Karmic (ami-7d43ae14), a 15 GiB EBS volume is created. Is this some sort of default size? Is it possible to create a new instance with (for example) a 40 GiB EBS volume as the root partiition?

P.S. Of course, I supose I can always manually create a 40 GiB EBS volume, mount both volumes (the original 15 GiB and the newly created 40 GiB), copy the content from one to another, detach the 15 GiB and attach the 40 GiB... would that also work?

Regards,
Xavier

Xavier: I've written two articles on the topic of growing EBS root volumes:
http://alestic.com/2009/12/ec2-ebs-boot-resize
http://alestic.com/2010/02/ec2-resize-running-ebs-root
That said, the most common reason you might want to grow it is to put data on it, and I recommend splitting your data onto a separate EBS volume, if possible.

[newbie here] I need an Ubuntu/Desktop/NX on EBS. I was able to launch ami-593d6c1c (8.10) and NX works great. What do I need to do convert it to EBS?

Thanks,
David

David: I think it would be best to start with an Ubuntu 9.10 Karmic AMI and add Desktop features to it. I wrote an article a while back about how to add NoMachine's NX server to Karmic:
http://alestic.com/2009/11/ec2-karmic-desktop

Or, you can wait a month and Canonical will be publishing Ubuntu 10.04 desktop AMIs with FreeNX installed. I'm not sure if they will be EBS boot.

There are a number of tutorials floating around describing how to convert S3 based AMIs to EBS boot AMIs. Here are a couple by Dave and Mitch:
http://coderslike.us/2009/12/07/amazon-ec2-boot-from-ebs-and-ami-conversion/
http://www.elastician.com/2009/12/creating-ebs-backed-ami-from-s3-backed.html

I created an instance with one of the EBS boot EMI's that ask to SSH in as ubuntu@ and not root@. Problem is that as ubuntu@ i don't have permission to do mkdir..

rsingh: If you need to do things as root, follow the standard Ubuntu practice of using the "sudo" command. On EC2 it is slightly non-standard in that, by default, you don't need to enter a password for sudo.

Eric: Thanks for all the articles. How about reducing an EBS volume? After launching your EBS volume, I would like to create an image, but one much smaller than 15 GB, e.g. 3 or 5 GB. The base OS doesn't need much, particularly if data is mounted from another volume. No point in having (or paying) for space I won't use (though the $1/month isn't much; but I'm now curious on how to accomplish this).

Thanks for helping,
Brady

Brady: I posted a response to this question on the ec2ubuntu group.

I tried creating an image with 100 GB EBS volume, but I keep getting the following error,

Client.InvalidParameterValue: The requested instance type's architecture (i386) does not match the architecture in the manifest for ami-cf8d61a6 (x86_64)

The command I am executing is this,

ec2-run-instances ami-cf8d61a6 -K D:\EC2\aws\pk-HR65D2J54R5BFXJPJGUZW2BJQWJGJJCS.pem -C D:\EC2\aws\cert-HR65D2J54R5BFXJPJGUZW2BJQWJGJJCS.pem -b "/dev/sda1=:100:false"

Thanks for helping in advance,
Rohit

Rohit: I'd recommend asking this question on the ec2ubuntu group and include the command you used to create the AMI.

This might be my own newbie ignorance but why would you want to use an AMI that boots from EBS versus a normal AMI that boots from the instanced hard drive? If anyone could explain the difference I would appreciate it.

thelievense: (1) You can stop/start an EBS boot instance without losing data and without paying run charges for the stopped hours. (2) You can modify attributes of an EBS boot instance like kernel, user-data. (3) If EBS boot instance hardware fails, you do not lose the root disk. (4) You can debug system failures by attaching root disk to another instance and examining/fixing it. (5) You can create snapshots of EBS root disk. (6) You can register those snapshots as a new AMI. (7) EBS boot AMIs may boot faster.

Hi Eric,

I am using the Ubuntu 10.04 Lucid Canonical, ubuntu@
EBS boot image.

1) How do I enable the root account in the instance ?
2) How do I convert the server image into a desktop image ?

Thank you

santhoshsd:
1) http://alestic.com/2009/04/ubuntu-ec2-sudo-ssh-rsync
2) You could simply start with one of the Ubuntu 10.04 Lucid desktop images published by Canonical (daily builds) or follow a procedure like http://alestic.com/2009/11/ec2-karmic-desktop

Hey eric, this looks great, and I'm eager to give it a go. It's my first experience with EBS BOOT. I'm looking forward particularly to:

1 - being able to redeploy without having to make sure no updates/etc were accidentally left out of the deployment image.
2 - being able to do live snapshots of my root fs.

One question. I notice the root fs is ext3, and it doesn't appear to be on an lvm. Does this mean that a live filesystem snapshot may be inconsistent? Is this a big enough risk that if I'm doing nightly backups I should be concerned? Previously I've installed lvm+ext3 to give me the ability to take snapshots. I don't really want to go through the hassle of creating my own EBS BOOT Ubuntu image using lvm+ext3, and not even sure if that'd be possible. Any backup suggestions?

nonent: Yes, there is a risk that a snapshot of your root EBS volume will not be consistent given that it is an ext3 file system which cannot be frozen during the snapshot. I don' t know how big the risk is. In my experience it is small, but there are different levels of "small" depending on the severity of things going bad.

I still recommend creating regular snapshots of root EBS volumes, as each snapshot reduces the risk of the volume itself going bad.

Leave a comment

Ubuntu AMIs

Ubuntu AMIs for EC2:


More Entries

EBS-SSD Boot AMIs For Ubuntu On Amazon EC2
With Amazon’s announcement that SSD is now available for EBS volumes, they have also declared this the recommended EBS volume type. The good folks at Canonical are now building Ubuntu…
EC2 create-image Does Not Fully "Stop" The Instance
The EC2 create-image API/command/console action is a convenient trigger to create an AMI from a running (or stopped) EBS boot instance. It takes a snapshot of the instance’s EBS volume(s)…
Finding the Region for an AWS Resource ID
use concurrent AWS command line requests to search the world for your instance, image, volume, snapshot, … Background Amazon EC2 and many other AWS services are divided up into various…
Changing The Default "ubuntu" Username On New EC2 Instances
configure your own ssh username in user-data The official Ubuntu AMIs create a default user with the username ubuntu which is used for the initial ssh access, i.e.: ssh ubuntu@<HOST>…
Default ssh Usernames For Connecting To EC2 Instances
Each AMI publisher on EC2 decides what user (or users) should have ssh access enabled by default and what ssh credentials should allow you to gain access as that user.…
New c3.* Instance Types on Amazon EC2 - Nice!
Worth switching. Amazon shared that the new c3.* instance types have been in high demand on EC2 since they were released. I finally had a minute to take a look…
Query EC2 Account Limits with AWS API
Here’s a useful tip mentioned in one of the sessions at AWS re:Invent this year. There is a little known API call that lets you query some of the EC2…
Using aws-cli --query Option To Simplify Output
My favorite session at AWS re:Invent was James Saryerwinnie’s clear, concise, and informative tour of the aws-cli (command line interface), which according to GitHub logs he is enhancing like crazy.…
Reset S3 Object Timestamp for Bucket Lifecycle Expiration
use aws-cli to extend expiration and restart the delete or archive countdown on objects in an S3 bucket Background S3 buckets allow you to specify lifecycle rules that tell AWS…
Installing aws-cli, the New AWS Command Line Tool
consistent control over more AWS services with aws-cli, a single, powerful command line tool from Amazon Readers of this tech blog know that I am a fan of the power…
Using An AWS CloudFormation Stack To Allow "-" Instead Of "+" In Gmail Email Addresses
Launch a CloudFormation template to set up a stack of AWS resources to fill a simple need: Supporting Gmail addresses with “-” instead of “+” separating the user name from…
New Options In ec2-expire-snapshots v0.11
The ec2-expire-snapshots program can be used to expire EBS snapshots in Amazon EC2 on a regular schedule that you define. It can be used as a companion to ec2-consistent-snapshot or…
Replacing a CloudFront Distribution to "Invalidate" All Objects
I was chatting with Kevin Boyd (aka Beryllium) on the ##aws Freenode IRC channel about the challenge of invalidating a large number of CloudFront objects (35,000) due to a problem…
Email Alerts for AWS Billing Alarms
using CloudWatch and SNS to send yourself email messages when AWS costs accrue past limits you define The Amazon documentation describes how to use the AWS console to monitor your…
Cost of Transitioning S3 Objects to Glacier
how I was surprised by a large AWS charge and how to calculate the break-even point Glacier Archival of S3 Objects Amazon recently introduced a fantastic new feature where S3…
Running Ubuntu on Amazon EC2 in Sydney, Australia
Amazon has announced a new AWS region in Sydney, Australia with the name ap-southeast-2. The official Ubuntu AMI lookup pages (1, 2) don’t seem to be showing the new location…
Save Money by Giving Away Unused Heavy Utilization Reserved Instances
You may be able to save on future EC2 expenses by selling an unused Reserved Instance for less than its true value or even $0.01, provided it is in the…
Installing AWS Command Line Tools from Amazon Downloads
When you need an AWS command line toolset not provided by Ubuntu packages, you can download the tools directly from Amazon and install them locally. In a previous article I…
Convert Running EC2 Instance to EBS-Optimized Instance with Provisioned IOPS EBS Volumes
Amazon just announced two related features for getting super-fast, consistent performance with EBS volumes: (1) Provisioned IOPS EBS volumes, and (2) EBS-Optimized Instances. Starting new instances and EBS volumes with…
Which EC2 Availability Zone is Affected by an Outage?
Did you know that Amazon includes status messages about the health of availability zones in the output of the ec2-describe-availability-zones command, the associated API call, and the AWS console? Right…
Installing AWS Command Line Tools Using Ubuntu Packages
See also: Installing AWS Command Line Tools from Amazon Downloads Here are the steps for installing the AWS command line tools that are currently available as Ubuntu packages. These include:…
Ubuntu Developer Summit, May 2012 (Oakland)
I will be attending the Ubuntu Developer Summit (UDS) next week in Oakland, CA. ┬áThis event brings people from around the world together in one place every six months to…
Uploading Known ssh Host Key in EC2 user-data Script
The ssh protocol uses two different keys to keep you secure: The user ssh key is the one we normally think of. This authenticates us to the remote host, proving…
Seeding Torrents with Amazon S3 and s3cmd on Ubuntu
Amazon Web Services is such a huge, complex service with so many products and features that sometimes very simple but powerful features fall through the cracks when you’re reading the…
CloudCamp
There are a number of CloudCamp events coming up in cities around the world. These are free events, organized around the various concepts, technologies, and services that fall under the…
Use the Same Architecture (64-bit) on All EC2 Instance Types
A few hours ago, Amazon AWS announced that all EC2 instance types can now run 64-bit AMIs. Though t1.micro, m1.small, and c1.medium will continue to also support 32-bit AMIs, it…
ec2-consistent-snapshot on GitHub and v0.43 Released
The source for ec2-conssitent-snapshot has historically been available here: ec2-consistent-snapshot on Launchpad.net using Bazaar For your convenience, it is now also available here: ec2-consistent-snapshot on GitHub using Git You are…
You Should Use EBS Boot Instances on Amazon EC2
EBS boot vs. instance-store If you are just getting started with Amazon EC2, then use EBS boot instances and stop reading this article. Forget that you ever heard about instance-store…
Retrieve Public ssh Key From EC2
A serverfault poster had a problem that I thought was a cool challenge. I had so much fun coming up with this answer, I figured I’d share it here as…
Running EC2 Instances on a Recurring Schedule with Auto Scaling
Do you want to run short jobs on Amazon EC2 on a recurring schedule, but don’t want to pay for an instance running all the time? Would you like to…