Using Amazon EC2, you created an EBS boot AMI and it’s working fine, but now you want to run instances of that AMI in a different EC2 region. Since AMIs are region specific, you need a copy of the image in each region where instances are required.
This article presents one method you can use to copy an EBS boot from one EC2 region to another.
Setup
Define the region from which we are copying the EBS boot AMI (source) and the region to which we are copying (target). Define the EBS boot AMI that we are copying from the source region.
We also need to determine which ids to use in the target region for the AKI (kernel image) and ARI (ramdisk image). These must correspond to the AKI and ARI in the source region or the new AMI may not work correctly. This is probably the trickiest step of the process and one which is not trivial to automate for the general case.
source_region=us-west-1 # replace with your regions
target_region=eu-west-1
source_ami=[AMI_ID_TO_COPY]
target_aki=[AKI_ID]
target_ari=[ARI_ID]
To make things easier, we’ll upload our own ssh public key to both regions. We could also do this with ssh keys generated by EC2, but that it is slightly more complex as EC2 generates unique keys for each region.
ssh_key_file=$HOME/.ssh/id_rsa
tmp_keypair=copy-ami-keypair-$$
ec2-import-keypair --region $source_region --public-key-file $ssh_key_file.pub $tmp_keypair
ec2-import-keypair --region $target_region --public-key-file $ssh_key_file.pub $tmp_keypair
Find the Ubuntu 10.04 LTS Lucid AMI in each of our regions of interest using the REST API provided by Ubuntu. Pick up some required information about the EBS boot AMI we are going to copy.
instance_type=c1.medium
source_run_ami=$(wget -q -O- http://uec-images.ubuntu.com/query/lucid/server/released.current.txt |
egrep "server.release.*ebs.i386.$source_region" | cut -f8)
target_run_ami=$(wget -q -O- http://uec-images.ubuntu.com/query/lucid/server/released.current.txt |
egrep "server.release.*ebs.i386.$target_region" | cut -f8)
architecture=$(ec2-describe-images --region $source_region $source_ami | egrep ^IMAGE | cut -f8)
ami_name=$(ec2-describe-images --region $source_region $source_ami | egrep ^IMAGE | cut -f3 | cut -f2 -d/)
source_snapshot=$(ec2-describe-images --region $source_region $source_ami | egrep ^BLOCKDEVICEMAPPING | cut -f4)
ami_size=$(ec2-describe-snapshots --region $source_region $source_snapshot | egrep ^SNAPSHOT | cut -f8)
Image Copy
Start an instance in each region. Have EC2 create a new volume from the AMI to copy and attach it to the source instance. Have EC2 create a new, blank volume and attach it to the target instance.
dev=/dev/sdi
xvdev=/dev/sdi # On modern Ubuntu, you will need to use: xvdev=/dev/xvdi
mount=/image
source_instance=$(ec2-run-instances --region $source_region --instance-type $instance_type --key $tmp_keypair --block-device-mapping $dev=$source_snapshot::true $source_run_ami |
egrep ^INSTANCE | cut -f2)
target_instance=$(ec2-run-instances --region $target_region --instance-type $instance_type --key $tmp_keypair --block-device-mapping $dev=:$ami_size:true $target_run_ami |
egrep ^INSTANCE | cut -f2)
while ! ec2-describe-instances --region $source_region $source_instance | grep -q running; do sleep 1; done
while ! ec2-describe-instances --region $target_region $target_instance | grep -q running; do sleep 1; done
source_ip=$(ec2-describe-instances --region $source_region $source_instance | egrep "^INSTANCE" | cut -f17)
target_ip=$(ec2-describe-instances --region $target_region $target_instance | egrep "^INSTANCE" | cut -f17)
target_volume=$(ec2-describe-instances --region $target_region $target_instance | egrep "^BLOCKDEVICE.$dev" | cut -f3)
Copy the file system from the EBS volume in the source region to the to the EBS volume in the target region.
ssh -i $ssh_key_file ubuntu@$source_ip "sudo mkdir -m 000 $mount && sudo mount $xvdev $mount"
ssh -i $ssh_key_file ubuntu@$target_ip "sudo mkfs.ext3 -F -L uec-rootfs $xvdev &&
sudo mkdir -m 000 $mount && sudo mount $xvdev $mount"
ssh -A -i $ssh_key_file ubuntu@$source_ip "sudo -E rsync -PazSHAX --rsh='ssh -o \"StrictHostKeyChecking no\"' --rsync-path 'sudo rsync' $mount/ ubuntu@$target_ip:$mount/"
ssh -i $ssh_key_file ubuntu@$target_ip "sudo umount $mount"
The uec-rootfs file system label is required for Ubuntu 10.10 and above to boot correctly on EC2. It can be left off for other distributions and earlier versions of Ubuntu.
AMI Creation
Snapshot the target EBS volume and register it as a new AMI in the target region. If the source AMI included parameters like block device mappings for ephemeral storage, then add these options to the ec2-register command.
target_snapshot=$(ec2-create-snapshot --region $target_region $target_volume | egrep ^SNAPSHOT | cut -f2)
target_ami=$(ec2-register --region $target_region --snapshot $target_snapshot --architecture $architecture --name "$ami_name" --kernel $target_aki --ramdisk $target_ari |
cut -f2)
echo "Make a note of the new AMI id in $target_region: $target_ami"
Make a note of the new AMI id.
Cleanup
Terminate the EC2 instances that were used to copy the AMI. Since we let EC2 create the EBS volumes on instance run, EC2 will automatically delete those volumes when the instances terminate. Delete the temporary keypairs we used to access the instances. Clean up the temporary files we created on the local system.
ec2-terminate-instances --region $source_region $source_instance
ec2-terminate-instances --region $target_region $target_instance
ec2-delete-keypair --region $source_region $tmp_keypair
ec2-delete-keypair --region $target_region $tmp_keypair
rm $tmp_private_key $user_data
OPTIONAL: If you don’t want to keep the AMI you created, you can remove it with commands like:
ec2-deregister --region $target_region $target_ami
ec2-delete-snapshot --region $target_region $target_snapshot
Other People’s AMIs
In order to follow this procedure you need to have read access to the snapshot associated with the source AMI, which generally means it must be an EBS boot AMI that you created and registered.
If you want to copy a public EBS boot AMI that somebody else created and you don’t have read access to the EBS snapshot for that AMI (the common case) then you can’t create a volume directly from the snapshot.
However, you should be able to play a little trick where you run an instance of that AMI and immediately stop the instance. Detach the EBS root volume from that instance, and attach it to another instance to perform the copy as above.
The new AMI might not be exactly the same if it had a chance to start the actual boot processes, but it should be pretty close.
Cost
The AWS fees for copying an EBS boot AMI following the instructions in this article will include a couple hours of instance time, a small amount for the temporary EBS volumes, and some EBS I/O request charges. You will also be charged a nominal amount each month for the S3 storage of the EBS snapshot for the new AMI in the target region.
[Update 2010-11-01: Added uec-rootfs label for Ubuntu 10.10 thanks to Scott Moser]
[Update 2011-11-30: Updated to use /dev/xvdi for modern Ubuntu.]
Let me point yuo to our cloudyscripts-service that provides the same script (and some others) as a web-service with GUI: https://cloudyscripts.com/tool/show/5
elastic-security.com: Thanks for the link; looks like a useful set of tools. Unfortunately, many folks including me are going to be extremely reluctant or forbidden to give a third party the secret keys to the entire AWS account just to get an AMI copied.
It's not that we don't trust you, but.. we don't trust you :-)
How about publishing the code as open source so that people can use it securely on their own systems? Sure this take some work to package as a tool and maintain, but perhaps the community would pitch in to help.
Note: The web tool currently uses the term "availability zone" when it means "region". There are multiple availability zones inside each EC2 region.
Hi Eric, I completely agree with you and understand your concerns of trust. That's why we made the actually scripting code available as Open Source (http://rubygems.org/gems/CloudyScripts) and also provide an AMI to start the service within your own AWS account. Would be happy if guys like you would be involved into this...
Cheers,
Matthias
Matthias: Sweet! You're way ahead of me :)
Based on discussions at UDS (Ubuntu Developer Summit) this week, I think the Ubuntu project may want to include some tools that make it easier to work with EC2 images. I'll point folks your way to see if there might be some opportunity for collaboration to speed things along.
Hi Eric,
Since rsync might list the file permission under ntfs-3g mount type. Any suggestion for Windows NTFS boot disk?
Cheers,
Ping Wu
Hi Eric,
I have a need to copy my EBS AMI based on Amazon Linux AMI using the kernel id aki-9ba0f1de in the us-west-1 region to all the other 3 regions.
You said "We also need to determine which ids to use in the target region for the AKI (kernel image) and ARI (ramdisk image). These must correspond to the AKI and ARI in the source region or the new AMI may not work correctly. This is probably the trickiest step of the process and one which is not trivial to automate for the general case."
Can you please help me out showing me "the trickiest step" how I can find a compatible one in the other 3 regions? Thanks for the great help.
PSK
PSK: You'll need to find the corresponding AMIs in the other regions and use their associated AKI and ARI (if any).
PSK: I used the ec2dim command to find the corresponding AKIs and ARIs in each region. (The description should be same in each region.)
ec2-describe-images --region us-west-1 -o self -o amazon
Eric, thanks for helpful post!
Ping Wu:
I'm afraid I don't know anything about Windows. You'll want to ask your questions on a different forum.
Hi Eric, thanks for a great article, as always.
I created my own AMI, which has a 8 GB boot ebs volume.
I am having problems after starting the source instance with
ec2-run-instances --region $source_region --instance-type $instance_type --key $tmp_keypair --block-device-mapping $dev=$source_snapshot::true $source_run_ami
The command gives back the instance id, which is running.
I can see
BLOCKDEVICE /dev/sdi vol-52dbbb3b 2011-06-06T08:59:56.000Z
which is created from my AMI-snapshot. $source_snapshot exists and is snap-e05b7889.
But the new instance never fully boots. The log says
[ 6.560020] EXT3 FS on sdi, internal journal
...
The disk drive for /vol1 is not ready yet or not present
Continue to wait; or Press S to skip mounting or M for manual recovery
So it looks like the /dev/sdi can not mount als /vol1.
I already tried adding the availability zone on the above command. But had no luck with that. Would you have any ideas?
TIA
Christian
This seems to only address moving an individual partition. Any thoughts on moving partitioned EBS volumes (such as those sometimes used with pv-grub kernels)
For example;
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 2497 20051968 83 Linux
/dev/sdb2 2497 2611 916481 5 Extended
/dev/sdb5 2497 2611 916480 82 Linux swap / Solaris
Do I just have to manually partition and format, then move over the /dev/sdb1 contents?
Man, this is overcomplicated.. it'd be nice if Amazon could do something to address these shortcomings.
AItOawm0FLL0oS3ypXSfxhvasGJBoKVAopxGtfM:
Since I get to pick the size of my EBS volumes, I no longer use partitions. To me, partitioning was for the old days when we bought hardware disks of a fixed size and then had to customize their configuration to meet our specific needs. In modern times, if I want a couple "partitions" for different purposes, I simply create multiple (unpartitioned) EBS volumes, each exactly the size that I want.
In your case, it appears that your boot EBS volume is partitioned with swap space on it. Besides making it more complex to resize the volume, it seems like an inefficient design as every time you swap it goes over the network and charges you EBS IO transactions. Do you really need swap contents to survive a stop/start or could swap be placed on the ephemeral storage?
In fact, I don't recommend using swap on servers in general. In my experience, if a server starts swapping, it is often a sign that things are not designed well. I'd rather have server software crash quickly running out of memory than bog down in swapping so that it's unusable for minutes or hours before it crashes. I do use swap on my laptop, but just so that I can hibernate to disk.
The Ubuntu AMIs use an unpartitioned EBS root volume even with PV, making it easy to start an instance with a bigger root disk:
http://alestic.com/2009/12/ec2-ebs-boot-resize
In fact, the latest Ubuntu releases on EC2 automatically perform a "resize2fs" so that it detects and adjusts to the new size of the EBS volume without manual intervention.
All that said, if you're stuck with an AMI that uses a partitioned EBS root volume, you should still be able to extend the size. You'll probably need to use low level tools to turn off swap, repartition the drive, extend the file system, and turn swap back on. It looks like you might be using Solaris which I haven't used since the 80's, so I can't offer much more help.
I realize that this article is pretty old.
However I totally do not understand why you do not do:
* Create volume of snapshot (or start / stop)
* Attach Volume to instance in source region
* Create Target Volume
* Attach target volume to instance in target region
And then do something like (insecure way, only for public AMIs):
Target:
nc -l 10000 > /dev/[target] # new nc syntax
Source:
dd if=/dev/[source] | nc 10000
DONE and you have a 1:1 block copy of the volume.
Secure version:
Target:
nc -l 10000 > /dev/[target] # new nc syntax
Source:
ssh -N 10000:localhost:10000 -i &
dd if=/dev/[source] | nc localhost 10000
And again done.
AItOawnGMHi_s2Rs0ZFPvNLJuA4aFfzV01XKbfg:
The article is old, but the technique is still recommended for copying EBS boot AMIs.
The nc approach only replaces the mount/rsync/umount commands, so it's not saving much. Here's what you're losing by using nc on the raw block devices:
1. rsync only copies the data that exists in the file system. This is much less than all of the blocks on the raw device, saving you transfer time.
2. rsync compresses the data significantly, saving you transfer time.
3. rsync will only modify the minimum number of blocks on the target EBS volume required to save the file contents. The nc to raw block device will touch every block on the target EBS volume. This increases the size of the EBS snapshot, increasing storage cost, and possibly increasing instance startup time from the resulting AMI, not to mention EBS snapshot size of EBS volumes on the new instances, and so on.
To the benefit of posterity (at to make sense of half day of swearing):
if, like me you're still on 10.4 LTS EBS, you have to change the label for mkfs.ext3 from uec-rootfs to cloudimg-rootfs.
Thanks Eric
AItOawnNZrf7xuimV9cS453fPC62Whu_tWg83QE:
Good catch, thanks.