If you’ve been following along, you probably know that I have been recommending that folks using EC2 switch to the official Ubuntu AMIs published by Canonical (Hardy or Karmic). I have been building and publishing Ubuntu AMIs since 2007 (including Dapper, Edgy, Feisty, Gutsy, Hardy, Intrepid, Karmic), but the last year my focus on this project has been to transition these responsibilities to Canonical who have more time and resources to support the initiative.
I’m happy to say that I’ve finally followed my own advice. For my personal Amazon EC2 servers (including for the Alestic.com web site) I am using Ubuntu 9.10 Karmic images published for EC2 by Canonical.
While I was making the transition, I also switched to EBS boot AMIs. However, since it sounds like Canonical is not planning to publish EBS boot AMIs until Lucid, I decided to continue in service to the community and make available EBS boot AMIs for running Ubuntu on EC2.
I have published EBS boot AMIs for Ubuntu 9.10 Karmic and Ubuntu 8.04 Hardy, both 32- and 64-bit architectures, in all current EC2 regions, for a total of a dozen new AMIs.
I chose to use the exact Ubuntu images which Canonical built for running Ubuntu on EC2. This means that these EBS boot AMIs work exactly the same as the official Canonical AMIs including ssh to the ubuntu user. Again, even though I’m publishing the EBS boot AMIs for Karmic and Hardy, the contents of the image were built by Canonical.
The EBS boot AMIs are listed on Alestic.com. I have restructured the table to better feature Canonical AMIs, and now you need to pick an EC2 region to see the IDs.
Give the EBS boot AMIs a spin and let me know if you run into any issues.
Hi Eric,
I am trying to get the ami-6743ae0e image up and running.
For some reason I can't get it to find the installable for mysql... haven't changed anything on the AMI - just fired it up...
I am working with:
AMI: ami-6743ae0e
Region: us-east-1b
my sources.list
$ cat sources.list
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ karmic main universe
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ karmic main universe
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ karmic-updates main universe
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ karmic-updates main universe
deb http://security.ubuntu.com/ubuntu karmic-security main universe
deb-src http://security.ubuntu.com/ubuntu karmic-security main universe
Installing mysql-server
$ sudo apt-get install -y mysql-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package mysql-server is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package mysql-server has no installation candidate
Would you be able to tell me what I am doing wrong?
It's my first running EC2 server (after the tutorial), so it's probably me doing something wrong :-)
Do I need to open anything in the security group for example?
I've got SSH, RDP and HTTP open...
Any help would be much appreciated!
And a BIG thanks for providing these images... was just about to get started building one myself, so you saved me a bunch of time!
Cheers,
Anders
Anders: Common mistake. The first thing that needs to be run on a new instance is:
sudo apt-get update
This brings in the latest package index from the repositories so that install and upgrade know which package files to work with.
Oh, yeah, that does the trick ;-)
Again, thanks a bunch for providing these images... a real timesaver!!
I thought I would try attaching the snapshot these EBS based AMI's were registered with so I could check them out, and download a copy without having to boot up an instance (or to check out an ami before booting it). However, it appears the snapshot is not public.
>snap=`ec2-describe-images -H ami-7d43ae14|awk '/sda1/ {print $3}'`
> ec2-describe-snapshots $snap
Client.InvalidSnapshot.NotFound: The snapshot 'snap-886f22e1' does not exist.
Since the data is all public in the AMI, do you think you could make the snapshot public as well?
Michael: You can download the original images[1] from Canonical, but perhaps you're checking up on me to make sure I did what I said? Fair 'nuff :)
According to a presentation at BlackHat 2009[2], knowing the bits on the image may give an attacker an advantage in predicting and guessing things like the ssh host secret key which would let them intercept and control secure connections.
This isn't an issue for most AMIs as normal users can't download the bundles from S3, but now that Canonical is publishing the images they use, it may be a concern.
Because of this, I may change a few bits[3] when copying the Canonical images for future EBS boot AMIs, and I would encourage Canonical to do the same thing for their S3 based AMIs if it turns out this really does make them more secure.
However, since I already stated these were exact copies, I've gone ahead and made the snapshots public for the "20100121" EBS boot AMIs in all regions.
Please let me know if you find any differences. They were file-wise copies and not block-wise copies in order to reduce the size of the resulting snapshot and speed up instance startup.
[1] http://uec-images.ubuntu.com/
[2] http://www.slideshare.net/astamos/cloud-computing-security (page 67)
[3] /var/lib/urandom/random-seed
Hi Eric,
Any reason why these images are not using XFS?
berenddeboer: I left out a number of "improvements" I could have made in these images. The goal here was to have an exact replica of the Ubuntu AMIs published by Canonical, except in EBS boot format.
I am a bit of newbie but also tried to create an EBS image from canonical images but sadly my procedure does not work. :(
I then read your blog further and see you create the EBS image by downloading the canonical image... whereas I was trying to create the image from the instance that was currently running. Which is a technique I have seen vaguely described in a few places.
So question.... Is this possible/feasible? I am thinking now for the future potentially, as I will most like move forward with your EBS image in the short-term.
A summary of my process:
1. Create a new EC2 volume, this needs to be at least as big as the total used size of the EC2 Instance
ec2-create-volume --size 10 --availability-zone us-east-1a
2. Attach the volume to the EC2 instance. Make a note of the device
ec2-attach-volume vol-yyyyyyyy --instance i-xxxxxxxx --device /dev/sdh
3. Format the newly added volume
dd if=/dev/zero of=/dev/sdh
yes | mkfs -t ext3 /dev/sdh
5. Mount the volume
mkdir /mnt/ebsimage
echo '/dev/sdh /mnt/ebsimage ext3 defaults,noatime 0 0' >> /etc/fstab
mount /mnt/ebsimage
6. Copy the required system and data files across to the new volume
NOTE: I tried a number of different version of the following command
rsync -rlpgoD -t -r -S -l -vh \
--exclude /ebs \
--exclude /mnt \
-X /* /mnt/ebsimage > rsync.log 2>rsync.err
# what of
#--exclude /sys \
#--exclude /proc \
#--exclude /dev \
#--exclude /media \
7. Unmount the device and detach the volume
umount /mnt/ebsimage
ec2-detach-volume vol-yyyyyyyy --instance i-xxxxxxxx
8. Take a snapshot of the volume
ec2-create-snapshot vol-yyyyyyyy
9. Register the snapshot as an EC2 AMI
ec2reg -s snap-zzzzzzzz -a {x86_64|i386} -d Description -n imagename
10. Launch and test
freremark: Yes, you can create a new AMI from a running EBS boot instance. You don't have to go through the process of creating a new volume and copying files (your steps 1-7). Simply create an EBS snapshot of your root EBS volume and register it (your steps 8-9). Make sure you specify the correct kernel and ramdisk in the registration step.
I am using public AMI ami-6743ae0e (ebs root).
I have created an instance using this ami, but after I stopped and started the instance, it lost public DNS name (in Amazon console and ssh access). Even restarting the instance did not help.
Do you know what might be the problem?
Thanks,
maks
maks: It can take a bit for the instance to start running and a new IP address to be assigned after it is started. Note that it will likely be different from the old IP address. The AWS console sometimes needs to be refreshed to pick up changes in values.
If you continue having problems, post your instance id to the EC2 forum for help: http://ec2forum.notlong.com
Eric: I'm quite new at using EC2, and probably this is a silly question and more than probably I should read more before posting questions, but here it is:
When I launch a new instance of your EBS Ubuntu Karmic (ami-7d43ae14), a 15 GiB EBS volume is created. Is this some sort of default size? Is it possible to create a new instance with (for example) a 40 GiB EBS volume as the root partiition?
P.S. Of course, I supose I can always manually create a 40 GiB EBS volume, mount both volumes (the original 15 GiB and the newly created 40 GiB), copy the content from one to another, detach the 15 GiB and attach the 40 GiB... would that also work?
Regards,
Xavier
Xavier: I've written two articles on the topic of growing EBS root volumes:
http://alestic.com/2009/12/ec2-ebs-boot-resize
http://alestic.com/2010/02/ec2-resize-running-ebs-root
That said, the most common reason you might want to grow it is to put data on it, and I recommend splitting your data onto a separate EBS volume, if possible.
[newbie here] I need an Ubuntu/Desktop/NX on EBS. I was able to launch ami-593d6c1c (8.10) and NX works great. What do I need to do convert it to EBS?
Thanks,
David
David: I think it would be best to start with an Ubuntu 9.10 Karmic AMI and add Desktop features to it. I wrote an article a while back about how to add NoMachine's NX server to Karmic:
http://alestic.com/2009/11/ec2-karmic-desktop
Or, you can wait a month and Canonical will be publishing Ubuntu 10.04 desktop AMIs with FreeNX installed. I'm not sure if they will be EBS boot.
There are a number of tutorials floating around describing how to convert S3 based AMIs to EBS boot AMIs. Here are a couple by Dave and Mitch:
http://coderslike.us/2009/12/07/amazon-ec2-boot-from-ebs-and-ami-conversion/
http://www.elastician.com/2009/12/creating-ebs-backed-ami-from-s3-backed.html
I created an instance with one of the EBS boot EMI's that ask to SSH in as ubuntu@ and not root@. Problem is that as ubuntu@ i don't have permission to do mkdir..
rsingh: If you need to do things as root, follow the standard Ubuntu practice of using the "sudo" command. On EC2 it is slightly non-standard in that, by default, you don't need to enter a password for sudo.
Eric: Thanks for all the articles. How about reducing an EBS volume? After launching your EBS volume, I would like to create an image, but one much smaller than 15 GB, e.g. 3 or 5 GB. The base OS doesn't need much, particularly if data is mounted from another volume. No point in having (or paying) for space I won't use (though the $1/month isn't much; but I'm now curious on how to accomplish this).
Thanks for helping,
Brady
Brady: I posted a response to this question on the ec2ubuntu group.
I tried creating an image with 100 GB EBS volume, but I keep getting the following error,
Client.InvalidParameterValue: The requested instance type's architecture (i386) does not match the architecture in the manifest for ami-cf8d61a6 (x86_64)
The command I am executing is this,
ec2-run-instances ami-cf8d61a6 -K D:\EC2\aws\pk-HR65D2J54R5BFXJPJGUZW2BJQWJGJJCS.pem -C D:\EC2\aws\cert-HR65D2J54R5BFXJPJGUZW2BJQWJGJJCS.pem -b "/dev/sda1=:100:false"
Thanks for helping in advance,
Rohit
Rohit: I'd recommend asking this question on the ec2ubuntu group and include the command you used to create the AMI.
This might be my own newbie ignorance but why would you want to use an AMI that boots from EBS versus a normal AMI that boots from the instanced hard drive? If anyone could explain the difference I would appreciate it.
thelievense: (1) You can stop/start an EBS boot instance without losing data and without paying run charges for the stopped hours. (2) You can modify attributes of an EBS boot instance like kernel, user-data. (3) If EBS boot instance hardware fails, you do not lose the root disk. (4) You can debug system failures by attaching root disk to another instance and examining/fixing it. (5) You can create snapshots of EBS root disk. (6) You can register those snapshots as a new AMI. (7) EBS boot AMIs may boot faster.
Hi Eric,
I am using the Ubuntu 10.04 Lucid Canonical, ubuntu@
EBS boot image.
1) How do I enable the root account in the instance ?
2) How do I convert the server image into a desktop image ?
Thank you
santhoshsd:
1) http://alestic.com/2009/04/ubuntu-ec2-sudo-ssh-rsync
2) You could simply start with one of the Ubuntu 10.04 Lucid desktop images published by Canonical (daily builds) or follow a procedure like http://alestic.com/2009/11/ec2-karmic-desktop
Hey eric, this looks great, and I'm eager to give it a go. It's my first experience with EBS BOOT. I'm looking forward particularly to:
1 - being able to redeploy without having to make sure no updates/etc were accidentally left out of the deployment image.
2 - being able to do live snapshots of my root fs.
One question. I notice the root fs is ext3, and it doesn't appear to be on an lvm. Does this mean that a live filesystem snapshot may be inconsistent? Is this a big enough risk that if I'm doing nightly backups I should be concerned? Previously I've installed lvm+ext3 to give me the ability to take snapshots. I don't really want to go through the hassle of creating my own EBS BOOT Ubuntu image using lvm+ext3, and not even sure if that'd be possible. Any backup suggestions?
nonent: Yes, there is a risk that a snapshot of your root EBS volume will not be consistent given that it is an ext3 file system which cannot be frozen during the snapshot. I don' t know how big the risk is. In my experience it is small, but there are different levels of "small" depending on the severity of things going bad.
I still recommend creating regular snapshots of root EBS volumes, as each snapshot reduces the risk of the volume itself going bad.