Alestic.com Sponsorship

For information on sponsoring Alestic.com, please see:

https://alestic.com/sponsors

A quick but heartfelt note of thanks to Ben Cherian at ServiceCloud, a long term supporter and faithful sponsor of Alestic.com (2 years!). Ben started sponsoring this site, supporting the public Ubuntu AWS/EC2 work I performed, back when this was just a one page site listing the Ubuntu AMI ids on EC2. Alestic.com has multiplied many times in content and organic web traffic over these two years.

Alestic.com is now open to new sponsors. If your company is interested in getting brand exposure at the top of every page on this site dedicated to running Ubuntu on Amazon EC2, please drop me an email.

I like to partner with companies like ServiceCloud and RightScale where the service or product matches the highly targeted reader audience of Alestic.com: folks using or planning to use AWS/EC2 cloud technologies.

BTW, if you are looking for a CIO, consider getting in touch with Ben Cherian who is coming available for new opportunities.

Move a Running EBS Boot Instance to New Hardware on Amazon EC2

NOTE: Though this method works and there is useful information in this article about things you can do with EBS boot instances, there is a simpler way to move an instance to new hardware.

Amazon EC2 has been experiencing some power issues in a portion of one of their many data centers. Even though the relative percentage of people affected might be small, when you have as many customers as AWS does, a small fraction can still be a large absolute number of customers who are affected.

Naturally, some customers will be upset about not having access to their systems, but in the time it takes to write a complaint, you might be able to move your server to new hardware within EC2 and go on with your business

First, lets assume that you are running an EBS boot instance. If you didn’t think they were the way to go before reading this article, I expect to convince you with this one example (and there are a number of other benefits).

Setup

For this demo, I’m going to start an instance. In your situation this instance represents the currently running server that you are depending on and which has valuable software, configuration, and perhaps data on its EBS volume(s). I’m also going to drop a note on the EBS root disk on my running instance so that I know it is the one I wanted to preserve. Again, this is just setting things up for the demo:

# SKIP THIS ENTIRE SECTION IF YOU ALREADY HAVE AN INSTANCE RUNNING
keypair=YOURKEYPAIR
sshkey=.ssh/$keypair.pem # or wherever you keep it
region=us-west-1 # pick your region
zone=${region}a # pick your availability zone
type=m1.small # pick your size
amiid=ami-cb97c68e # Ubuntu 10.04 Lucid, 32-bit, EBS boot in that region
oldinstanceid=$(ec2-run-instances \
  --key $keypair \
  --region $region \
  --availability-zone $zone \
  --instance-type $type \
  $amiid |
  egrep ^INSTANCE | cut -f2)
echo "instanceid=$oldinstanceid"

while host=$(ec2-describe-instances --region $region "$oldinstanceid" | 
  egrep ^INSTANCE | cut -f4) && test -z $host; do echo -n .; sleep 1; done
echo host=$host

echo "save the volume" | ssh -i $sshkey ubuntu@$host tee README.txt

Moving to a New Instance

Now, we pretend that the above created instance has failed in some way and we can no longer access it. Here’s how we get our server running on a new instance:

  1. If you can, stop the old instance. This increases your chance of keeping the file system consistent on the EBS volume. If the instance has really failed and this step does not work, then skip it.

     ec2-stop-instances --region $region $oldinstanceid
    
  2. Run a new instance with the same startup parameters as your old instance.

     newinstanceid=$(ec2-run-instances \
       --key $keypair \
       --region $region \
       --availability-zone $zone \
       --instance-type $type \
       $amiid |
       egrep ^INSTANCE | cut -f2)
     echo "newinstanceid=$newinstanceid"
    
  3. Wait until the new instance is running and then stop (not terminate) the new instance and detach the EBS boot volume from it. Delete the volume as it has nothing of importance, having just been created.

     ec2-stop-instances --region $region $newinstanceid
     newebsroot=$(ec2-describe-instances --region $region $newinstanceid |
       grep ^BLOCKDEVICE | grep /dev/sda1 | cut -f3)
     ec2-detach-volume --force --region $region $newebsroot
     ec2-delete-volume --region $region $newebsroot
    
  4. Detach the old (valuable) EBS root volume from the old (broken) instance.

     oldebsroot=$(ec2-describe-instances --region $region $oldinstanceid |
       grep ^BLOCKDEVICE | grep /dev/sda1 | cut -f3)
     ec2-detach-volume --force --region $region $oldebsroot
    
  5. Attach the old (valuable) EBS root volume to the new (stopped) instance.

     ec2-attach-volume \
       --region $region \
       -d /dev/sda1 \
       -i $newinstanceid \
       $oldebsroot
    

    If you had multiple EBS volumes attached to the old instance, you would move each one over in a similar manner.

  6. Restart the new instance which is now going to boot with the original volume.

     ec2-start-instances --region $region $newinstanceid
    

Voila! You have moved your server from an old, perhaps broken instance to new (or at least different) hardware keeping the same file system, and it took only a few minutes! If you’d like, ssh to the new instance and make sure that your valuable information is still there.

If you had an Elastic IP address associated with the old instance, you would move it to the new instance.

Cleanup

You may terminate the old instance if you are comfortable that you won’t need it any more. If you were following this demo as an exercise, you should also terminate the new instance. Since you manually attached the old volume to the new instance yourself, it will not be deleted automatically when the instance is terminated. You can modify the instance attributes to change the delete-on-termination flag for the volume or simply delete it manually.

# BEWARE! Don't copy these blindly, but think about what you should do
ec2-terminate-instances --region $region $oldinstanceid
ec2-terminate-instances --region $region $newinstanceid
ec2-delete-volume --region $region $oldebsroot

Tips

This above process can also be used when your instance is running fine, but you want to move to a different instance type (size) of the same architecture. For example, you could move from m1.small up to c1.medium, or from m2.4xlarge down to c1.xlarge. Update: I wasn’t thinking clearly when I wrote that last sentence. It is possible to change the instance type much more easily: Simply stop the instance, use ec2-modify-instance-attribute, and start it up again. Read more about this method.

You can also resize the root disk of a running EC2 instance using the same basic principle of swapping out an EBS root volume on a running instance.

[Update 2011-02-07: Point out simpler method to move to new hardware.]
[Update 2012-01-08: Link to article on changing instance type.]

One Weekend to Prototype CrowdPhoto.net using AWS Technologies

On Friday evening (2 days ago) I and a number of other high octane LA individuals came to LA StartupWeekend to participate in the process of building the beginning of a startup in 2 days. The group of eighty participants divided up organically by interest into about ten different ideas which had been pitched that very evening.

I ended up in a great team of 9 developers, designers, and product folks who were all interested in building the idea I proposed:

Connect individuals carrying cell phone cameras with people who want timely photographs at a particular location.

Less than 48 hours later, we launched the prototype which you can check out now at:

http://CrowdPhoto.net

And, we had a lot of fun in the process.

I mention this on the Alestic.com blog because our team used a number of cool technologies including:

  • Ubuntu 10.04 Lucid (released one day before we started this project)
  • Amazon EC2
  • Amazon SimpleDB (we decided to jump on the “NoSQL” buzzword bandwagon)
  • Amazon S3 (storage of uploaded photos)
  • Amazon CloudFront (CDN to serve the photos)
  • Perl 5.10
  • Mason (templating)
  • Apache (of course)
  • HTML, CSS3, JavaScript
  • Postfix (automatic processing of incoming photo emails)

Not shown on the site is the fact that two of the developers on the team learned Android development and built an integrated Android application which integrates with the CrowdPhoto.net service. In two days.

We also have prototype integration with Twitter. Feel free to follow us at:

@crowdphoto

Challenges we ran into and resolved included:

  • Our first AWS account took a long time to get approved for SimpleDB. Solution: We used SimpleDB from another AWS account (different than the one running the EC2 instance).

  • We quickly ran into the EC2 email sending limit just with cron job output and emails from our revision control system. Solution: relay email through one of our personal servers on a non-standard port.

  • We found that some SimpleDB requests had too high a latency to run a large number sequentially: Solution: make the requests in parallel (not yet implemented).

Ubuntu 10.04 Lucid worked like a charm.

Take the prototype service for a spin and let us know what you think. Do you think that this type of application has potential? What would you use it for?

Feel free to be harsh on what you see. We are. But, remember that this was built in less than 2 days from concept, to design, to building development infrastructure, to implementation. This is a fun prototyping project.

We don’t know if there is going to be a future for this project as we all go back to our separate real jobs on Monday, but some of us think there might be something really cool here. Give us your ideas.

[Update 2010-05-03: Added images]

Ubuntu AMIs available for Amazon EC2 in Asia Pacific (Singapore)

Amazon EC2 just launched the Asia Pacific region with data centers in Singapore.

The standard Ubuntu and Debian AMIs (Amazon Machine Images) from Canonical and Alestic are already available in this new region. I have listed the new AMI ids in the table at the top of Alestic.com.

To see the AMIs, simply click on the ap-southeast-1 tab in the table at the top.

If you’re just getting into using Ubuntu on Amazon EC2, consider joining the EC2 Ubuntu group for community support.

Ubuntu 10.04 Lucid Released for Amazon EC2

Ubuntu 10.04 Lucid was released on Amazon EC2 right on schedule today along with the rest of the normal Ubuntu image download channels.

Congrats to all the Ubuntu folks who were involved in putting this together! Though I’ve enjoyed contributing to the cause over the years, I’m happy to say that I had very little involvement with this release on EC2, leaving me more time to focus on my startup and family. Though many individuals are involved, I’d like to acknowledge the hard work of Scott Moser who has taken the point for publishing official Ubuntu EC2 images these last couple releases.

This is also the first Ubuntu release on EC2 that includes officially supported EBS boot AMIs, taking yet another task off my plate and providing a trusted source for this useful image type.

I’ve listed all of the current Ubuntu and Debian AMI ids in the table at the top of Alestic.com. Simply click on the EC2 region where you want to run your instances to find the AMI ids.

If you’re running an older release of Ubuntu on EC2, see my recommended upgrade paths.

Upgrading Servers to Ubuntu 10.04 Lucid on Amazon EC2

Ubuntu 10.04 Lucid is expected to be released this Thursday (April 29, 2010). This is the first LTS (long term support) release since Ubuntu 8.04 Hardy, which has itself been a stable and reliable server platform for years both inside and outside of Amazon EC2.

If you’re starting a new server project on EC2, Ubuntu 10.04 Lucid would be a great release to start with. It will be supported for five years with security patches and major bug fixes.

If you are already running a different Ubuntu release on EC2, you should consider planning an upgrade to Lucid. Here are my thoughts for each release you might currently be running on EC2:

  • Ubuntu 8.04 Hardy should continue to be fairly stable for a while, though there does not appear to be a lot of EC2-specific development being applied to the AMIs, so you may want to upgrade to Lucid LTS at your leisure. You will probably want to start new instances, though with Scott’s comment below, it might be possible to upgrade in-place on EBS boot from Hardy to Lucid. If you happen to still be running the Alestic Hardy AMIs, you should at least consider upgrading to the Hardy AMIs published by Canonical and listed at the top of Alestic.com.

  • Ubuntu 8.10 Intrepid has reached its end of life and should no longer be used on EC2. If you are still running Intrepid, you should immediately start upgrading to Karmic or Intrepid through starting new instances.

  • Ubuntu 9.04 Jaunty on EC2 is still running on an old Fedora 8 kernel and is not supported by Ubuntu. I’ve been recommending an upgrade from Jaunty for a while. I continue to recommend upgrading to Karmic (if you want something that has been tested in production for a while) or Lucid (once you’re comfortable with the release).

  • Ubuntu 9.10 Karmic still works great on EC2, though there are a couple bothersome bugs (like hostname reset on reboot) that should be fixed by upgrading to Lucid. If you are running an EBS boot Karmic instance, then it is possible to upgrade in place, but it’s also fine to simply start new instances.

  • Ubuntu 10.04 Lucid pre-release instances should be upgraded to the released version. If you are running EBS boot Lucid, then it is possible to upgrade in place even if there are a newer kernel and ramdisk.

Scott Moser of Canonical has laid out the steps necessary to upgrade an EC2 instance in place without having to start a new EC2 instance. This only works for instances that are currently running an EBS boot pre-release Lucid AMI or an EBS boot 9.10 Karmic AMI. All other servers will need to be upgraded by starting new Lucid instances to replace the old instances. Fortunately, EC2 makes that easy.

In fact, EC2 makes starting new replacement instances so convenient, I would recommend following that procedure in most cases even if you think you can upgrade a server in place. If you encounter any troubles in getting your service to run on Lucid, running a new instance makes it straight forward to fall back to the old server. And, once you are happy with the new server, make it your production system, terminate the old one and don’t look back.

Welcome to the new world of cloud computing where you can instantly acquire and discard server hardware for mere dollars without worrying about large investments, complicated provisioning, and long term support costs.

I will add the new Lucid AMI ids to the table at the top of Alestic.com as soon as I get word that they are ready.

Identifying When a New EBS Volume Has Completed Initialization From an EBS Snapshot

On Amazon EC2, you can create a new EBS volume from an EBS snapshot using a command like

aws ec2 create-volume \
  --availability-zone us-east-1a \
  --snapshot-id snap-d53484bc

This returns almost immediately with a volume id which you can attach and mount on an EC2 instance. The EBS documentation describes the magic behind the immediate availability of the data on the volume:

“New volumes created from existing Amazon S3 snapshots load lazily in the background. This means that once a volume is created from a snapshot, there is no need to wait for all of the data to transfer from Amazon S3 to your Amazon EBS volume before your attached instance can start accessing the volume and all of its data. If your instance accesses a piece of data which hasn’t yet been loaded, the volume will immediately download the requested data from Amazon S3, and then will continue loading the rest of the volume’s data in the background."

This is a cool feature which allows you to start using the volume quickly without waiting for the blocks to be completely populated. In practice, however, there is a period of time where you experience high iowait when your application accesses disk blocks that need to be loaded. This manifests as somewhat to extreme slowness for minutes to hours after the creation of the EBS volume.

For some applications (mine) the initial slowness is acceptable, knowing that it will eventually pick up and perform with the normal EBS access speed once all blocks have been populated. As Clint Popetz pointed out on the ec2ubuntu group, other applications might need to know when the EBS volume has been populated and is ready for high performance usage.

Though there is no API status to poll or trigger to notify you when all the blocks on the EBS volume have been populated, it occurred to me that there was a method which could be used to guarantee that you are waiting until the EBS volume is ready: simply request all of the blocks from the volume and throw them away.

Here’s a simple command which reads all of the blocks on an EBS volume attached to device /dev/xvdX or /dev/sdX (substitute your device name) and does nothing with them:

sudo dd if=/dev/xvdX of=/dev/null bs=10M

OR

sudo dd if=/dev/sdX of=/dev/null bs=10M

By the time this command is done, you know that all of the data blocks have been copied from the EBS snapshot in S3 to the EBS volume. At this point you should be able to start using the EBS volume in your application without the high iowait you would have experienced if you started earlier. Early reports from Clint are that this method works in practice.

[Update 2012-04-02: We now have to support device names like sda1 and xvda1]

[Update 2019-08-07: New aws-cli command format]

New releases of Ubuntu and Debian Images for Amazon EC2 (20100319)

Note: I do not recommend that new users start with these AMIs. These AMIs run with older versions of Amazon’s Fedora 8 kernel which have some incompatibilities with Ubuntu and Debian (e.g., XFS is broken). My strong recommendation is for all users to convert to one of the Ubuntu 9.10 Karmic or Ubuntu 8.04 Hardy AMIs which run with a modern, compatible Ubuntu kernel.

I have released updates of the following legacy Ubuntu and Debian S3-based AMIs published in the “alestic” buckets:

  • Ubuntu: 8.04 Hardy, 8.10 Intrepid, 9.04 Jaunty
  • Debian: 4.0 Etch, 5.0 Lenny

I welcome testing and feedback from folks who are already using the older versions of these AMIs.

In addition to upgraded software packages, the following enhancements are in this release:

  • Wait for meta-data before deciding whether to generate ssh host key (Thanks to Dmitry for catching)

  • Patch from Tom White to support compressed user-data scripts

  • Allow user-data script to remove the “been run” file so that it is run on every boot instead of just the first

  • Upgrade EC2 AMI tools to 1.3-34544

  • Debian: Add Alestic PPA to apt sources

  • Debian: Install runurl from Alestic PPA

I would like to reiterate that these AMIs are not recommended for anybody except folks who are already using older versions of these AMIs and I would encourage you to upgrade to an Ubuntu 9.10 Karmic AMI or the upcoming Ubuntu 10.04 Lucid AMI which will be released in April. The base Ubuntu image building responsibilities have been transfered to Canonical and I don’t have any plans to release new AMIs in the older Ubuntu or Debian series beyond what I am announcing here.

I have not yet posted the AMI ids on https://alestic.com pending testing and feedback, but here are the ids for those who need to run them:

us-east-1

ami-1116f978 Ubuntu 9.04 Jaunty   server  32-bit
ami-e116f988 Ubuntu 9.04 Jaunty   server  64-bit
ami-cd16f9a4 Ubuntu 9.04 Jaunty   desktop 32-bit
ami-c316f9aa Ubuntu 9.04 Jaunty   desktop 64-bit

ami-1316f97a Ubuntu 8.10 Intrepid server  32-bit
ami-e316f98a Ubuntu 8.10 Intrepid server  64-bit
ami-cb16f9a2 Ubuntu 8.10 Intrepid desktop 32-bit
ami-c116f9a8 Ubuntu 8.10 Intrepid desktop 64-bit

ami-e916f980 Ubuntu 8.04 Hardy    server  32-bit
ami-e716f98e Ubuntu 8.04 Hardy    server  64-bit
ami-1716f97e Ubuntu 8.04 Hardy    desktop 32-bit
ami-e516f98c Ubuntu 8.04 Hardy    desktop 64-bit

ami-eb16f982 Ubuntu 6.06 Dapper   server  32-bit
ami-f916f990 Ubuntu 6.06 Dapper   server  64-bit

ami-ed16f984 Debian 5.0 Lenny     server  32-bit
ami-fb16f992 Debian 5.0 Lenny     server  64-bit
ami-cf16f9a6 Debian 5.0 Lenny     desktop 32-bit
ami-c516f9ac Debian 5.0 Lenny     desktop 64-bit

ami-ef16f986 Debian 4.0 Etch      server  32-bit
ami-fd16f994 Debian 4.0 Etch      server  64-bit

us-west-1

ami-197a2b5c Ubuntu 9.04 Jaunty   server  32-bit
ami-237a2b66 Ubuntu 9.04 Jaunty   server  64-bit
ami-357a2b70 Ubuntu 9.04 Jaunty   desktop 32-bit
ami-c97a2b8c Ubuntu 9.04 Jaunty   desktop 64-bit

ami-257a2b60 Ubuntu 8.10 Intrepid server  32-bit
ami-2d7a2b68 Ubuntu 8.10 Intrepid server  64-bit
ami-377a2b72 Ubuntu 8.10 Intrepid desktop 32-bit
ami-d57a2b90 Ubuntu 8.10 Intrepid desktop 64-bit

ami-277a2b62 Ubuntu 8.04 Hardy    server  32-bit
ami-2f7a2b6a Ubuntu 8.04 Hardy    server  64-bit
ami-cf7a2b8a Ubuntu 8.04 Hardy    desktop 32-bit
ami-d77a2b92 Ubuntu 8.04 Hardy    desktop 64-bit

ami-217a2b64 Ubuntu 6.06 Dapper   server  32-bit
ami-e77a2ba2 Ubuntu 6.06 Dapper   server  64-bit

ami-d37a2b96 Debian 5.0 Lenny     server  32-bit
ami-df7a2b9a Debian 5.0 Lenny     server  64-bit
ami-e37a2ba6 Debian 5.0 Lenny     desktop 32-bit
ami-db7a2b9e Debian 5.0 Lenny     desktop 64-bit

ami-d17a2b94 Debian 4.0 Etch      server  32-bit
ami-dd7a2b98 Debian 4.0 Etch      server  64-bit

eu-west-1

ami-a798b3d3 Ubuntu 9.04 Jaunty   server  32-bit
ami-af98b3db Ubuntu 9.04 Jaunty   server  64-bit
ami-9798b3e3 Ubuntu 9.04 Jaunty   desktop 32-bit
ami-9d98b3e9 Ubuntu 9.04 Jaunty   desktop 64-bit

ami-a198b3d5 Ubuntu 8.10 Intrepid server  32-bit
ami-a998b3dd Ubuntu 8.10 Intrepid server  64-bit
ami-9198b3e5 Ubuntu 8.10 Intrepid desktop 32-bit
ami-9998b3ed Ubuntu 8.10 Intrepid desktop 64-bit

ami-a398b3d7 Ubuntu 8.04 Hardy    server  32-bit
ami-ab98b3df Ubuntu 8.04 Hardy    server  64-bit
ami-9398b3e7 Ubuntu 8.04 Hardy    desktop 32-bit
ami-8798b3f3 Ubuntu 8.04 Hardy    desktop 64-bit

ami-ad98b3d9 Ubuntu 6.06 Dapper   server  32-bit
ami-9598b3e1 Ubuntu 6.06 Dapper   server  64-bit

ami-8398b3f7 Debian 5.0 Lenny     server  32-bit
ami-8f98b3fb Debian 5.0 Lenny     server  64-bit
ami-8998b3fd Debian 5.0 Lenny     desktop 32-bit
ami-8b98b3ff Debian 5.0 Lenny     desktop 64-bit

ami-8198b3f5 Debian 4.0 Etch      server  32-bit
ami-8d98b3f9 Debian 4.0 Etch      server  64-bit
SCALE 8x Talk Notes: EC2 Beginners Workshop

At SCALE 8x (Southern California Linux Expo, Feb 2010) I did a walkthrough demonstrating how to use the AWS console to run, connect to, and terminate your first Ubuntu server on Amazon EC2.

Though they do not include my talking points or the Q&A discussion during and following the session, you can download my slides and workshop handout:

http://cdn.anvilon.com/20100220-scale/scale8x-ec2.pdf

http://cdn.anvilon.com/20100220-scale/scale8x-steps.pdf

Notes:

  • These slides were not created for use outside of the workshop, so they are not complete in themselves. I am making them available for the workshop participants who may wish to refer to them to refresh their memory on what was discussed.

  • The instructions talk about removing the private ssh key at the end. This would not be done in a normal secure environment.

  • This demo used a temporary Ubuntu AMI which had wikimedia installed. This AMI is not designed for use in production systems as it includes a known wikimedia admin password.

  • This AMI is subject to being deleted at any time without notice. Did I mention you should not use it in a production system?

The BitSource Interview of Eric Hammond (SCALE, EC2)

Matthew Sacks, of The BitSource, made the mistake of asking me some questions about Amazon EC2, so I rambled on far too long and the results are posted on the SCALE blog:

SCALE: Eric Hammond on Deploying Linux on EC2

In addition to some high level concepts, I point out that Ubuntu is, at the moment, the best choice for a Linux distro on EC2 if you want up-to-date images with modern, consistent kernels.

Resizing the Root Disk on a Running EBS Boot EC2 Instance

In a previous article I described how to run an EBS boot AMI with a larger root disk size than the default. That’s fine if you know the size you want before running the instance, but what if you have an EC2 instance already running and you need to increase the size of its root disk without running a different instance?

As long as you are ok with a little down time on the EC2 instance (few minutes), it is possible to change out the root EBS volume with a larger copy, without needing to start a new instance.

Let’s walk through the steps on a sample Ubuntu 16.04 LTS Xenial HVM instance. I tested this with ami-40d28157 but check for the latest AMI ids.

On the instance we check the initial size of the root file system (8 GB):

$ df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      7.8G  913M  6.5G  13% /
New Ubuntu 8.04.3 Hardy AMIs for Amazon EC2

Scott Moser (Canonical) built and released new Ubuntu 8.04.3 LTS Hardy images and AMIs for Amazon EC2. I also published new EBS boot AMIs using the same images. I’ve listed all of the AMI ids on https://alestic.com (pick your region at the top).

These AMIs should work better in the us-west-1 region (apt sources.list) and have updated software packages so upgrades on new instances should be faster.

Southern California Linux Expo - Februrary 19-21, 2010 at the Westin LAX

SCaLE 8x

The 8th Southern California Linux Expo (aka SCaLE 8x) is a community organized, non-profit event. Those words and the incredibly cheap price might lead you to believe that it is not worth going to, but if this is your first time you’ll be amazed by the size, scope, and professionalism of the event with nearly a hundred exhibits and dozens of informative talks.

Even though you’re not paying hundreds of dollars for the conference fee, it’s still worth traveling to if you’re not in Los Angeles. If you are in LA, then you have no excuse.

Just like last year at SCaLE, I will be leading another “Try-It Lab” where we’ll help folks get started with using Amazon EC2 and Ubuntu Linux. More information about preparation will be posted on the SCaLE blog, so be sure to review it before attending if you’re interested in a hands-on, guided, workshop experience with EC2. The lab seats “sold out” quickly last year, so make sure you get in early.

Deal for readers of Alestic.com: When you register for SCaLE, use the code “ERIC” for 50% off of the listed price. If you sign up today, that gives you a full access pass for a ridiculously low $35. Prices may go up as the weekend gets closer.

[Upate 2010-02-16: Link to preparation instructions on SCALE blog]

Public EBS Boot AMIs for Ubuntu on Amazon EC2

If you’ve been following along, you probably know that I have been recommending that folks using EC2 switch to the official Ubuntu AMIs published by Canonical (Hardy or Karmic). I have been building and publishing Ubuntu AMIs since 2007 (including Dapper, Edgy, Feisty, Gutsy, Hardy, Intrepid, Karmic), but the last year my focus on this project has been to transition these responsibilities to Canonical who have more time and resources to support the initiative.

I’m happy to say that I’ve finally followed my own advice. For my personal Amazon EC2 servers (including for the Alestic.com web site) I am using Ubuntu 9.10 Karmic images published for EC2 by Canonical.

While I was making the transition, I also switched to EBS boot AMIs. However, since it sounds like Canonical is not planning to publish EBS boot AMIs until Lucid, I decided to continue in service to the community and make available EBS boot AMIs for running Ubuntu on EC2.

I have published EBS boot AMIs for Ubuntu 9.10 Karmic and Ubuntu 8.04 Hardy, both 32- and 64-bit architectures, in all current EC2 regions, for a total of a dozen new AMIs.

I chose to use the exact Ubuntu images which Canonical built for running Ubuntu on EC2. This means that these EBS boot AMIs work exactly the same as the official Canonical AMIs including ssh to the ubuntu user. Again, even though I’m publishing the EBS boot AMIs for Karmic and Hardy, the contents of the image were built by Canonical.

The EBS boot AMIs are listed on Alestic.com. I have restructured the table to better feature Canonical AMIs, and now you need to pick an EC2 region to see the IDs.

Give the EBS boot AMIs a spin and let me know if you run into any issues.

How to Report Bugs with Ubuntu on Amazon EC2: ubuntu-bug

The official Ubuntu AMIs published by Canonical for EC2 starting in October have proven to be solid and production worthy. However, you may still on occasion run into an issue which deserves to be brought to the attention of the Ubuntu server team developing these AMIs and the software which enables Ubuntu integration with EC2.

The easiest, most efficient, and most complete way to report problems with Ubuntu on EC2 is to use the ubuntu-bug tool which comes pre-installed on all Ubuntu systems.

The ubuntu-bug command requires a single argument which is one of:

  1. the name of an Ubuntu software package experiencing a problem,

  2. the path to a program related to the problem,

  3. the process id of the program experiencing the problem, or

  4. the path of a crash file.

When reporting EC2 startup issues with an Ubuntu instance, the involved package is generally ec2-init so the command to run would be:

ubuntu-bug ec2-init

This command should be run on the EC2 instance that is experiencing the problem. The ubuntu-bug command will collect relevant information about the instance and file it with the bug report to assist in tracking down and correcting the issue.

If the instance with the problem is no longer running or accessible, try to run another instance of the same AMI to report the bug. This will help submit the correct AMI information with the bug report.

If ubuntu-bug reports “This is not a genuine Ubuntu package” you might have to first run

sudo apt-get update

and then try again.

Unfortunately, ubuntu-bug is an interactive program which does not accept command line options to set choices, so you will need to respond to a couple prompts and then copy and paste a URL it provides to you. First, it asks:

What would you like to do? Your options are:
  S: Send report (1.5 KiB)
  V: View report
  K: Keep report file for sending later or copying to somewhere else
  C: Cancel
Please choose (S/V/K/C): S

Respond by hitting the “S” key because you really do want to report a problem.

ubuntu-bug then displays a URL and asks if you would like to launch a browser.

Choices:
  1: Launch a browser now
  C: Cancel
Please choose (1/C): C

Respond by hitting the “C” key as ubuntu-bug running on the EC2 instance can’t launch the web browser on your local system and you probably don’t want to use a terminal based browser.

Make a note of the URL displayed in:

*** To continue, you must visit the following URL:
  https://bugs.launchpad.net/ubuntu/+source/ec2-init/+filebug/LONGSTRINGHERE?

Copy the URL and paste it into your web browser. You will continue reporting the problem through your browser and the system information will be attached after you submit.

If this is the first time you have used Launchpad.net, you will be prompted to create an account. Use a valid email address as you will need to confirm it.

Launchpad will prompt you to enter a “Summary” which should be a short description of the bug. If it is not a duplicate of one of the bugs already entered, click “No I need to report a new bug” and enter the “Further Information”. Include as much information as possible relevant to the issue. If a developer can reproduce the bug using this description, then it will be addressed more easily.

For general information on submitting bugs in Ubuntu, please see:

https://help.ubuntu.com/community/ReportingBugs

You can see also see a current list of open ec2-images bugs.

If you are reporting Ubuntu on EC2 bugs directly using Launchpad without ubuntu-bug (not recommended) make sure you include the AMI id and tag the bug with “ec2-images”.

Note that ubuntu-bug is not a mechanism to support general support questions. One place to get help with running Ubuntu on EC2 is from the community in the ec2ubuntu Google group and there’s always the general Amazon EC2 forum. You can occasionally get live help with Ubuntu on EC2 on the #ubuntu-server IRC channel on irc.freenode.net