You Should Use EBS Boot Instances on Amazon EC2

| 23 Comments

EBS boot vs. instance-store

If you are just getting started with Amazon EC2, then use EBS boot instances and stop reading this article. Forget that you ever heard about instance-store and accept my apology that I just mentioned it. Once you are completely comfortable with using EBS boot instances on EC2, you may (or may not) want to come back here and read why you made a good decision.

EC2 experts may find that there are specific cases, few and far between, where instance-store might make sense, but they don’t attempt to use instance-store without understanding and accounting for all the serious drawbacks and dangers that go with making this choice. For example, experts using instance-store don’t mind losing all of the data on the instance as they have designed the system so that the data is stored elsewhere and so that a new instance can easily and automatically be rebuilt from scratch.

One of the challenges for beginners is that many of the benefits of EBS boot don’t necessarily seem like something you’ll need to use right away. Then they get down the road and into situations where they realize that they would have been much better off if they had gone with EBS boot in the first place and may find it takes some work to make the transition.

Big benefits of EBS boot instances

Here are some of the reasons I use and recommend EBS boot instances. None of these benefits are available with instance-store, so even a single one of these can be an overriding factor for choosing EBS boot.

  1. EBS boot instances store the root file system on an EBS volume which is persistent storage. If the instance hardware fails, the EBS volume remains accessible. It is also possible to request the EBS volume to persist beyond the termination of an EC2 instance. When an instance-store instance fails or is shut down, all of the data on the root disk is lost forever and can never be retrieved. Read more about protecting EC2 instances from accidental termination and loss of data.

  2. EBS boot instances can be stopped and restarted at will. The “stopped” state suspends the hourly instance billing charges, preserving the information on the EBS volumes. The stopped instance can be started again a few minutes later or months later, restoring state just as if the instance was rebooted. An instance-store instance can only be left running with full charges or terminated, which causes you to lose all data on the disk. Read more about how stop/start of an EBS boot instance is similar to and different from a simple reboot.

  3. When something goes wrong with an EBS boot instance so that it can’t be booted, or you lose access through ssh (e.g., lost keys, bad ssh config change), you can still view and modify or fix the EBS root volume by attaching it to another running instance. With an instance-store instance, everything on the root disk is lost and cannot be recovered. Read more about fixing files on the root EBS volume of an EC2 instance.

  4. EBS boot instances can be run with a root EBS volume size from the default specified by the AMI (often 8GB) up to 1,000GB (1TB). The instance-store AMIs have a max root disk size of 10GB with no way to increase it. Read more about increasing the root disk size of an EBS boot AMI.

  5. It is possible to grow the size of the root disk after an EBS boot instance has been started. An instance-store instance has no way to grow the root disk size. Read more about resizing the root disk on a running EBS boot EC2 instance.

  6. It is possible to change the instance type for a running EBS boot instance without needing to start a new instance. For example, you can scale up from an m1.large to an m1.xlarge and then a few hours or days later, scale back down. An instance-store instance is stuck with the type on which it was originally run. Read more about changing the instance type of a running EBS boot instance.

  7. You can easily replace the hardware for your instance if you are running an EBS boot instance. This is extremely valuable if your instance is having problems that you suspect may be related to the underlying hardware. An instance-store instance is bound to the hardware it started on and cannot be moved. Read more about using stop/start to replace EC2 hardware.

  8. EBS boot AMIs are simpler and faster to create than instance-store AMIs. In fact, you can trigger the creation of an EBS boot AMI from a running instance in one command, API call, or console click. You need to copy sensitive AWS credentials to the instance when creating an instance-store AMI.

  9. Amazon has stated that EBS boot AMIs boot up faster than S3 based AMIs (instance-store). In my recent experience, the difference is negligible, especially when testing popular AMIs that are likely to be cached, but we might as well chalk this up as another benefit.

  10. The t1.micro instance type released recently by Amazon only supports EBS boot instances. This move is like a sign from Amazon that you really don’t want to run the legacy instance-store instances.

  11. Some versions of Windows (Server 2008) only run on EBS boot instances. I believe this may be related to the disk size limitations of instance-store, but I don’t use Windows, so am not an expert in that area.

Possible benefits of instance-store instances

EBS volumes and EBS boot instances aren’t perfect. Running an instance-store instance might be preferable in some very specific cases where you don’t care so much about losing the data you are storing on the root disk.

I’m going to list some of the possible benefits of instance-store, but each of these may not be as beneficial as they appear at first glance and you must remember that running instance-store loses all of the above benefits.

  • There is a negigible cost savings with instance-store, as there is no charge for an EBS volume nor the I/O transactions. Note however, that the cost for an 8GB root EBS volume is only around 80 cents per month (depending on the region) and you get about a billion I/O requests for a dollar, billed by the penny increment. The cost savings to run instance-store is generally not worth the increased risk of losing your valuable data, especially when you compare it as a percentage of the cost of running the instance hours in the first place (tens or hundreds of dollars per month).

  • There may be some applications that perform somewhat better when running against ephemeral or instance-store disks than against EBS volumes. Some high end users have also reported inconsistency in the performance they see from EBS volumes at high I/O transaction rates for long periods of time. EBS volumes perform better for some applications and are generally good enough for most applications. If you don’t care about losing your application data, and you have tested to see that instance-store performs better than EBS volumes, then you could either run instance-store instances, or you could run EBS boot instances and drop your data on the ephemeral disks available to all instances above t1.micro. It is also becoming increasingly popular to run multiple EBS volumes in a mdadm RAID-0 or LVM striping configuration to improve performance and smooth out some experiences of performance volatility.

  • There are a couple historical failure modes that have happened with EBS volumes that could not happen for instance-store disks. You may hear people say this is a reason not to use EBS volumes, but EBS volumes are still far more reliable, persistent, and protected against failure than instance-store. The fact that it is possible for EBS to fail is not a reason to use the less persistent instance-store. It is a reason to create regular snapshots of your EBS volumes to improve reliability and act as backups.

  • If you are creating a “paid AMI” you can only do it as an instance-store AMI, not EBS boot. Only two of the thousands of people reading this article are creating paid AMIs and they already know this fact.

Why did I write this article?

I regularly provide consulting services in public communities like serverfault, the ##aws IRC channel on Freenode, the ec2ubuntu Google group, and Amazon’s EC2 forum. It’s a rare week that passes where I am not telling somebody that they would not have the problem they’re having if they had been using EBS boot instances.

The saddest response I can give to a plea for help is that the customer’s valuable data has been lost because they were running instance-store instead of EBS boot and they did not have real-time streaming backups.

Historical background

When Amazon launched EC2 in 2006, only instance-store AMIs were available (though they weren’t called instance-store at the time as they were the only kind). For years, Amazon customers learned to work with the server limitations of risking the loss of all data at any point in time.

In August of 2008, Amazon introduced the concept of EBS volumes, and there was much rejoicing. Data could finally be stored on persistent disks even though the root disk remained on ephemeral storage (another name for instance-store).

In December of 2009, Amazon released the ability to launch instances with EBS volumes as the root disk, or EBS boot instances. Now, the entire server could be persistent and all of the above benefits were realized.

Notes

Just because EBS boot volumes are “persistent” does not mean that they do not ever fail. Amazon has released figures about their failure rates, which are proportional to the number of blocks modified since the last EBS snapshot was created. Regular snapshots improve the intrinsic reliability of your EBS volumes in addition to acting as backups. I also recommend creating regular off-site backups (outside of Amazon) to eliminate your AWS account as a single point of failuire.

Even if you use an EBS boot instance, I still recommend keeping your data on a separate EBS volume. This has a number of benefits that could perhaps form the core of a followup article. Read an example of how to set up a MySQL database on a separate EBS boot volume. I still follow this approach with EBS boot instances, storing data on a second volume.

[Update 2011-01-04: Added benefit #11 (Windows Server 2008). Added note about potential (in)consistency of EBS IO at sustained, high rates.]

23 Comments

It seems that Eric Hammond doesn't use (or know about?) RightScale or a similar service, and so he's focused on benefits that aren't necessary if you have a proper cloud architecture under RightScale (or a similar tool). The best comparison I can give is this: Eric Hammond is telling us about the necessity of backing up our data onto thumb drives instead of just storing files on a single computer--while completely ignoring the fact that Dropbox exists. He's right, but he's missing the bigger picture and the best practices.

A proper cloud architecture has seemless backups off EBS (or instance stores) and extremely quick disaster recovery. The "major advantages" that Eric cites for EBS-backed instances are all related to either (1) having proper backups, (2) scaling, or (3) saving money on running instances. The first is best done much differently than just leaving a bunch of EBS volumes hanging around (RightScale has a great solution for this). The second is best done through a proper scaling system (RightScale has a great for solution for this, too). The third is best done by backing up to *S3* or EBS snapshots, not leaving EBS instances around in a "stopped" status (Guess what--RightScale for this too!)

He also omits/understates three biggest reason for using instance-backed AMIs instead of EBS-backed AMIs: (1) your server doesn't need EBS to launch, which means that the various problems that Amazon has had with EBS over the years won't affect your server launches, (2) EBS-backed AMIs are quite a bit more expensive than using ephemeral storage when you get to any decent amount of space ($100/month for 1TB of EBS, which is less than the ephemeral storge on a c1.xlarge...the 8GB in the piece is definitely a straw man), and (3) EBS volumes are limited to 1TB of space, so you have to stripe them to get bigger volume sizes (huge pain in the butt).

Don't get me wrong--EBS is fabulous. But not for backing instances. It's fabulous for storing massive amounts of data, getting quick snapshot backups, and quick restores for disaster recovery...all of which is best run through RightScale.

joefreshman:

Thanks for your feedback. I've been a fan of RightScale for years and should point out that RightScale is also a long term sponsor of this tech blog: http://alestic.com/sponsors

I am deliberately biased towards EBS boot instances in this article, because I want anybody who chooses instance-store after reading it to know what they are giving up and to have good solutions in place for handling the deficiencies of instance-store. I'm calling these people "experts" because it is not yet a common practice to create the infrastructure necessary to treat production servers and storage as temporary objects and to be able to recreate server configurations and data from external sources.

You can either be an expert because you have developed your own solutions for managing the ephemeral nature of instance-store, or you can depend on the expertise of other people by using third party products, RightScale included.

That said, I'm an expert in working with instance-store (did it for years and invented some of the tools used to automate their configuration) but I still use EBS boot for just about everything. All the EC2 instances run by my company are built through automated processes, have live replication and backups, can be replaced at a moments notice; and all are EBS boot.

Benefits 1-7 of EBS boot instances are only relevant to servers that need to easily persist state. The exceptions in that list are the points about AMI size limits, and the fixed size of the root device. Neither one of those seems particularly important however.

#8 I think is fairly important, and I think that EBS boot makes sense for anyone needing to create their own AMIs.

As you already mentioned, #9 is really stretching it as there's no significant difference in practice between popular instance-store AMIs and EBS volumes. #10 on the other hand seems like pure speculation. Inferring that instance-store is a deprecated option from the fact that t1.micro only supports EBS boot seems like a giant stretch.

A more reasonable assumption would be that t1.micro has no instance storage in order to minimize I/O impact to the instance store hard drive for the other larger instances that are running on the same server. Therefore the set of EBS only instance types is likely to remain t1.micro.

Going back to the point about stateful servers though, instance-store brings the pain of relying on persistent, stateful servers forward. Yes, it's difficult to deal with, but you'll need to deal with this at some point anyway. Instance-store wil also force one to think about places where truly stateless servers would work just fine. For example, most web servers and application servers have no need to persist their state. Doing so only invites one to create unique snowflake machines with an accumulation of small tweaks that is not documented anywhere.

EBS-boot instances are definitely more user friendly, because they closely emulate the physical machines booting of persistent hard drives that people are used to managing in their own data centers. That's not necessarily a good thing when moving to AWS.

AItOawmOhdXs_yFUpqLvqtBA_viuLH54GQNxtQ0:

Thanks for your comments.

The fact that you must use EBS boot instances to run on t1.micro is a factual benefit of EBS boot, not speculation. That this might indicate Amazon deprecates instance-store may be considered speculation.

I wholeheartedly support building infrastructures that can sustain full server loss and have preached this for years: http://alestic.com/2009/06/ec2-non-outage

For most applications it is just fine to build that sort of infrastructure on EBS boot instead of instance-store.

I can appreciate the point that using non-persistent storage forces you to learn a better way of building infrastructure on EC2. I was forced to learn that lesson on EC2 in the years before EBS was released. I'm not sure all EC2 users need to learn that lesson in such a harsh way.

Yes, I should remember to lock my apartment door living in the city. No, I don't think apartment managers should teach their tenants a lesson by stealing things when they forget to lock their doors.

An interesting article, with a few points I was unaware of.

My AWS experience started a few months ago, when I became the sysadmin for a small company with existing instances and a manual process for replacing most of them. We use instance-store on almost everything, and I've made that farily autotomated, but now that I'm familiar with aws, I could see where ebs root might be useful.

One thing I noticed that you left out in this post, though, is a mention of auto scaling. We use the amazon elb/autoscaler for our app and api servers, and those AMI's are all instance-store. I have some magic boot scripts that name and configure them properly, and even create an (or use an existing) ebs volume for storing in-flight data that we'd like to persist. My question: does ebs make sense for auto-scaled applications? To me, it seems like since these systems constantly are brought up and taken down, that instance store might work just as well as ebs, since the autoscaler terminates an unhealthy instance (unless I'm mistaken, and it just stops/reboots an ebs).

I have been using AWS for the past 4+ years and my recommendation to use instance store or EBS backed Instances have always been use case driven. We did some benchmark where we have seen that instance store had better I/O performance than a EBS backed Instance. And off late the Instance store images boot up time is almost equal to the EBS backed ones. So areas in infrastructure architecture which are completely stateless can go with instance store and Instances would automatically get configured on boot.

I don't see a necessity to have larger root volumes - at least for web applications. We design the root volumes to have only the OS and the application which does not require larger volume sizes. Of course if there are any layers in the architecture with persistent data then additional EBS volumes will be attached. System and application log files will be configured to use the ephemeral store since all the Instances come with larger ephemeral stores - logs can grow pretty fast and they enjoy the benefit of better I/O from ephemeral store.

Coming to Windows, we prefer using the EBS backed ones specifically because the image bundling process takes so much time with instance store ones. And of course for any R&D purposes EBS backed ones are preferred because you have the ability to stop and start them.

As I said earlier, it is primarily driven on the use case basis and one should not just go with a specific type with any over sight.

raghurocks:

Thanks for your comments.

I agree that there are some situations where instance-store instances might be preferable to EBS boot. I agree that many of the EBS boot benefits might not be that important to many users.

However, it is important to understand *all* of the EBS boot benefits, to realize that you are giving up *all* of them if you are running instance-store, and to have an architecture design to account for the drawbacks of instance-store.

As mentioned in the article, I agree with the approach of only putting the OS and applications on the root disk and keeping data on separate disks (generally EBS volumes). When you do this, there isn't much benefit to using instance-store for the root disk as very little activity is happening on it, so it might as well be EBS boot so you can get the above benefits.

I don't put application logs on ephemeral store because I can't afford to lose them. They are valuable to the business. I've asked Amazon for a syslogd service where we can stream logs and let them worry about storage and growth, but until then, they go on EBS volumes, get snapshotted, and get backed up offsite.

I don't put system logs on ephemeral store because it's nice to be able to look at the system logs if the instance kernel crashes. If you're running instance-store when that happens, you might have no idea why the instance suddenly became unreachable.

Once you've reached the point where you can read this article and argue for instance-store in a particular case with an explanation of how you are accounting for losing each of the listed benefits of EBS boot... then I name you "expert" as described in the second paragraph and you are free to move to the next level.

AItOawltRDyZBAOK1jRCAxuor0c6_Lyj39l2KF0:

Auto Scaling and spot instances are cases where a lot of the EBS boot benefits don't seem to apply since Amazon may start and terminate them autonomously. On the other hand, are you losing anything if you just always use EBS boot for consistency across your architecture?

Given the nature of Auto Scaling, you probably want the instances to boot up quickly with your desired software and configuration. The easiest way to do this is to create an AMI that boots with very little need to upgrade its configuration after coming up.

It's a lot easier to create EBS boot AMIs for Auto Scaling or spot instances than it is to create instance-store AMIs (point #8 above).

Some people are looking for the equivalent of VPS or dedicated hosting on EC2. They might not ever need to worry about the complexities of building auto scalable infrastructure or dealing with servers that can go away at any minute. Instances that mostly stick around and fail once in a blue moon, backed by EBS, are probably sufficient.

For anyone that wants to take advantage of the E in EC2, the lessons that come from dealing with instance-store instances are not optional. To me it feels that forcing yourself to deal with instance-store, fully aware of the pitfalls, is the best ways to learn those lessons early. Perhaps this way of learning is too harsh and risky. I haven't found a better way though to teach these lessons.

As far as using EBS boot instances even for stateless servers, it might certainly be fine, but I would feel more comfortable without the additional dependency. The EBS outage from not that long ago comes to mind. I would feel much more comfortable from eliminating EBS completely from an architecture. To quote Heroku's post mortem "block storage is not a cloud-friendly technology".

AItOawmOhdXs_yFUpqLvqtBA_viuLH54GQNxtQ0:

Thanks for your additional comments (though without a simple name it's probably hard for folks to tell which user is which).

Remind me not to rent an apartment from you ;-)

Eric-

Thanks for this article; it's going to be useful to give people this link.

One comment - don't you think you're being a little too kind on the matter of EC2 I/O performance with saying only: "There may be some applications that perform somewhat better when running against ephemeral or instance-store disks than against EBS volumes. Some high end users have also reported inconsistency in the performance they see from EBS volumes at high I/O transaction rates for long periods of time. EBS volumes perform better for some applications and are generally good enough for most applications."

My personal experience has been that while instance-store disk performance is relatively poor, EBS is *much* worse - and highly variable, to boot. For me, bringing users to EC2 that are used to our internal vSphere deployment, I find myself having to warn them about how bad disk I/O performance is, especially on EBS, the storage type that maps most closely to what they use on vSphere.

This isn't meant to be a vSphere-is-better-than-EC2 comment - to get okay disk performance with vSphere requires substantial capital and operational investments - we just happen to have made those investments, so it's relevant to my users. My point is more that any discussion of disk on EC2 should probably start with an acknowledgement, in bold letters, that "EC2 disk I/O is poor across the board; choose from the lesser of these evils."

-Andy

Andy:

Thanks for your comments.

Yes, I am being a little too kind to EBS volumes. This is deliberate. I'm recommending that people without experience and understanding of the complexities of EC2 should use EBS first.

  • I believe it is better for a tiny segment of new users to eventually run into performance problems and learn to solve them, than a large segment of users to run into complete data loss and have no way to solve it.
  • I have a hard time believing that EBS boot are unusable or bad for the majority of EC2 users.
  • People have been shouting "EBS performs poorly/inconsistently" from the mountain tops. A few of these are people are running some high end applications that actually do need to eek out the last drop of performance even at the cost of building more complicated systems. Most of these are probably just repeating what they heard because that's what humans do.
  • Some performance issues people are experiencing while running on EBS may not be the fault of EBS, but could be basic application design issues. Disk I/O in general is slow and should be avoided when possible. Sure, EBS (and instance-store/ephemeral) might be slower than physical hardware, but many of the same principles apply.
  • One of the sources of the "EBS is bad" line of thinking is Adrian Cockcroft's blog post Understanding and using Amazon EBS - Elastic Block Store from back in March 2011. It is important to note, however, that Adrian doesn't say "EBS is bad".

    In fact, in the first paragraph Adrian writes: "The conclusion we reached is a rule of thumb for EBS - If you sustain less than 100 iops (input+output per second) long term average it works fine. Short term bursts can be 1000 iops. By short term I mean less than a minute, long term more than 10 minutes. YMMV."

    That post was written 10 months ago and Adrian raises a great question in this tweet related to my article: "do you think EBS has improved in the last 6 months? AWS have been working hard on it."

    Things move fast and keep improving in this market. We should challenge and test our assumptions regularly.

    I use Binami AMI to start building the instance. How do I know it's instance-store instances or EBS boot instances?

    Is it something set in the AMI or some steps while the instance is creating?

    Simon Chen:

    EBS vs. instance-store is a property of the AMI which determines the type of the instance. Wherever you see a list of AMIs, it should indicate "ebs" or "instance-store". Other names for instance-store AMIs might include "ephemeral" or some mention of "S3" since S3 is where the instance-store AMI is stored before it is run on an instance. After an instance-store AMI is run, there is no S3 backing for the instance as the root disk is then ephemeral (disappears when the instance terminates or fails).

    It looks like Bitnami has both EBS boot and instance-store AMIs available: http://bitnami.org/tutorials/amazon_machine_images

    This posting and discussion reminds me of the old saw about how to get replies by posting wrong (or misleading) information... but Eric Hammond is intentionally overstating the case for EBS to counter the historical EC2 bias towards instance-store.

    EBS boot instances are very useful for persistent systems (including those that are only active intermittently) and instance-store/ephemeral instances are for flexible scaling like N machines of the same type.

    paul.baclace:

    I would not argue against any of your points.

    One HUGE negative issue with using EBS backed windows 2008 instances is as follows: Say your instance (I1) is degraded and you want to launch a new Win2008 instance (I2). However, because I1 has an EBS volume, it should be easy to launch I2 with I1's EBS volume, right? Wrong!

    This means that when I1 is degraded, you need to launch I2 with it's own root volume, attach I1's root volume, pull any data off of it that you need, etc. Simply a royal pain - which makes for a lot of manual work.

    Oh - by the way - though Amazon says in the documentation that you can attach an EBS as the root volume for windows, this does NOT work.

    So I have gone back to instance stores. All machine level configuration stuff is baked into an AMI. All my app stuff is stored in S3 - which the instance pulls down and customizes upon launch. Simple.

    Bill

    Bill:

    Thanks for your comments.

    I don't quite follow how losing your data when the instance goes down is better than having to copy it from the old EBS volume you attach to the new instance, but it certainly does sound simpler as you say ;-)

    Eric,

    In my approach, you don't lose any data either. All data is one of two types: a) my application and its data, b) everything else (OS stuff, other apps, etc.).

    I am suggesting that data of type b) are baked into my AMIs. Data of type a) is stored on S3 or in EBS volumes - which get downloaded or attached when we launch a new instance.

    My overriding design principle was that I wanted to be able to automate things so that in the event of failure, I could push a button and automagically a new instance, duly configured would start.

    I just could not figure out how to automate doing the root-volume copy stuff thingie for windows 8000 ebs instances.

    That's why I went down the route I did.

    Bill:

    Thanks for the clarification. It sounds like you have tackled the challenges of running on instance-store, and part of your solution is to use attached EBS volumes. You still risk losing things that the OS and apps store on the boot volume at run time (logs?), but with care your approach makes the best of the situation.

    Automatic instance launch and configuration qualifies you for "expert" status as defined in this article.

    Eric,

    You are correct in identifying what we would lose.

    I am no expert (certainly, not in your league anyway) but our solution works well so far.

    By the way, we do use your ec2-consistent-snapshot as a way of creating backups of our mysql db volumes. Works great!

    Bill

    This is the exact query I have too.

    In fact, I've been using EBS volumes for my EC2 instances and I love them.

    However, when it comes to autoscaling, I'm having a difficult figuring out how to use EBS volumes here. Interestingly, instance-store might be the better solution here since I will have less troubles keeping data consistent among the EC2 instances.

    Would really love to hear the author's opinions on autoscaling and EBS vs. instance store.

    Eric Nguyen

    ericnguyen37:

    By default the EBS root volume will be created when an instance is started with Auto Scaling and it will be deleted when Auto Scaling terminates the instance. This is similar to instance-store with Auto Scaling so there is no additional "trouble keeping data consistent" when you use EBS boot with Auto Scaling.

    With Auto Scaling you lose everything when an instance is terminated either way, so you have to make sure you are saving any important data elsewhere.

    I use EBS boot with Auto Scaling as it's just simpler to use the same AMIs for different purposes and instance-store doesn't offer much additional value. Plus it's much easier to build custom EBS boot AMIs, which you may end up doing more frequently so that your Auto Scaling instances come up faster.

    Leave a comment

    Ubuntu AMIs

    Ubuntu AMIs for EC2:


    AWS Jobs

    AWS Jobs

    More Entries

    Throw Away The Password To Your AWS Account
    reduce the risk of losing control of your AWS account by not knowing the root account password As Amazon states, one of the best practices for using AWS is Don’t…
    AWS Community Heroes Program
    Amazon Web Services recently announced an AWS Community Heroes Program where they are starting to recognize publicly some of the many individuals around the world who contribute in so many…
    EBS-SSD Boot AMIs For Ubuntu On Amazon EC2
    With Amazon’s announcement that SSD is now available for EBS volumes, they have also declared this the recommended EBS volume type. The good folks at Canonical are now building Ubuntu…
    EC2 create-image Does Not Fully "Stop" The Instance
    The EC2 create-image API/command/console action is a convenient trigger to create an AMI from a running (or stopped) EBS boot instance. It takes a snapshot of the instance’s EBS volume(s)…
    Finding the Region for an AWS Resource ID
    use concurrent AWS command line requests to search the world for your instance, image, volume, snapshot, … Background Amazon EC2 and many other AWS services are divided up into various…
    Changing The Default "ubuntu" Username On New EC2 Instances
    configure your own ssh username in user-data The official Ubuntu AMIs create a default user with the username ubuntu which is used for the initial ssh access, i.e.: ssh ubuntu@<HOST>…
    Default ssh Usernames For Connecting To EC2 Instances
    Each AMI publisher on EC2 decides what user (or users) should have ssh access enabled by default and what ssh credentials should allow you to gain access as that user.…
    New c3.* Instance Types on Amazon EC2 - Nice!
    Worth switching. Amazon shared that the new c3.* instance types have been in high demand on EC2 since they were released. I finally had a minute to take a look…
    Query EC2 Account Limits with AWS API
    Here’s a useful tip mentioned in one of the sessions at AWS re:Invent this year. There is a little known API call that lets you query some of the EC2…
    Using aws-cli --query Option To Simplify Output
    My favorite session at AWS re:Invent was James Saryerwinnie’s clear, concise, and informative tour of the aws-cli (command line interface), which according to GitHub logs he is enhancing like crazy.…
    Reset S3 Object Timestamp for Bucket Lifecycle Expiration
    use aws-cli to extend expiration and restart the delete or archive countdown on objects in an S3 bucket Background S3 buckets allow you to specify lifecycle rules that tell AWS…
    Installing aws-cli, the New AWS Command Line Tool
    consistent control over more AWS services with aws-cli, a single, powerful command line tool from Amazon Readers of this tech blog know that I am a fan of the power…
    Using An AWS CloudFormation Stack To Allow "-" Instead Of "+" In Gmail Email Addresses
    Launch a CloudFormation template to set up a stack of AWS resources to fill a simple need: Supporting Gmail addresses with “-” instead of “+” separating the user name from…
    New Options In ec2-expire-snapshots v0.11
    The ec2-expire-snapshots program can be used to expire EBS snapshots in Amazon EC2 on a regular schedule that you define. It can be used as a companion to ec2-consistent-snapshot or…
    Replacing a CloudFront Distribution to "Invalidate" All Objects
    I was chatting with Kevin Boyd (aka Beryllium) on the ##aws Freenode IRC channel about the challenge of invalidating a large number of CloudFront objects (35,000) due to a problem…
    Email Alerts for AWS Billing Alarms
    using CloudWatch and SNS to send yourself email messages when AWS costs accrue past limits you define The Amazon documentation describes how to use the AWS console to monitor your…
    Cost of Transitioning S3 Objects to Glacier
    how I was surprised by a large AWS charge and how to calculate the break-even point Glacier Archival of S3 Objects Amazon recently introduced a fantastic new feature where S3…
    Running Ubuntu on Amazon EC2 in Sydney, Australia
    Amazon has announced a new AWS region in Sydney, Australia with the name ap-southeast-2. The official Ubuntu AMI lookup pages (1, 2) don’t seem to be showing the new location…
    Save Money by Giving Away Unused Heavy Utilization Reserved Instances
    You may be able to save on future EC2 expenses by selling an unused Reserved Instance for less than its true value or even $0.01, provided it is in the…
    Installing AWS Command Line Tools from Amazon Downloads
    This article describes how to install the old generation of AWS command line tools. For the most part, these have been replaced with the new AWS cli that is…
    Convert Running EC2 Instance to EBS-Optimized Instance with Provisioned IOPS EBS Volumes
    Amazon just announced two related features for getting super-fast, consistent performance with EBS volumes: (1) Provisioned IOPS EBS volumes, and (2) EBS-Optimized Instances. Starting new instances and EBS volumes with…
    Which EC2 Availability Zone is Affected by an Outage?
    Did you know that Amazon includes status messages about the health of availability zones in the output of the ec2-describe-availability-zones command, the associated API call, and the AWS console? Right…
    Installing AWS Command Line Tools Using Ubuntu Packages
    See also: Installing AWS Command Line Tools from Amazon Downloads Here are the steps for installing the AWS command line tools that are currently available as Ubuntu packages. These include:…
    Ubuntu Developer Summit, May 2012 (Oakland)
    I will be attending the Ubuntu Developer Summit (UDS) next week in Oakland, CA. ┬áThis event brings people from around the world together in one place every six months to…
    Uploading Known ssh Host Key in EC2 user-data Script
    The ssh protocol uses two different keys to keep you secure: The user ssh key is the one we normally think of. This authenticates us to the remote host, proving…
    Seeding Torrents with Amazon S3 and s3cmd on Ubuntu
    Amazon Web Services is such a huge, complex service with so many products and features that sometimes very simple but powerful features fall through the cracks when you’re reading the…
    CloudCamp
    There are a number of CloudCamp events coming up in cities around the world. These are free events, organized around the various concepts, technologies, and services that fall under the…
    Use the Same Architecture (64-bit) on All EC2 Instance Types
    A few hours ago, Amazon AWS announced that all EC2 instance types can now run 64-bit AMIs. Though t1.micro, m1.small, and c1.medium will continue to also support 32-bit AMIs, it…
    ec2-consistent-snapshot on GitHub and v0.43 Released
    The source for ec2-conssitent-snapshot has historically been available here: ec2-consistent-snapshot on Launchpad.net using Bazaar For your convenience, it is now also available here: ec2-consistent-snapshot on GitHub using Git You are…
    You Should Use EBS Boot Instances on Amazon EC2
    EBS boot vs. instance-store If you are just getting started with Amazon EC2, then use EBS boot instances and stop reading this article. Forget that you ever heard about instance-store…