Alestic.com - A Personal AWS Blog by Eric Hammond

starting with Hugo!

A week ago, I presented a CloudFormation template for an AWS Git-backed Static Website stack. If you are not familiar with it, please click through and review the features of this complete Git

• static website CloudFormation stack.

This weekend, I extended the stack to support a plugin architecture to run the static site generator of your choosing against your CodeCommit Git repository content. You specify the AWS Lambda function at stack launch time using CloudFormation parameters (ZIP location in S3).

The first serious static site generator plugin is for Hugo, but others can be added with or without my involvement and used with the same unmodified CloudFormation template.

The Git-backed static website stack automatically invokes the static site generator whenever the site source is updated in the CodeCommit Git repository. It then syncs the generated static website content to the S3 bucket where the stack serves it over a CDN using https with DNS served by Route 53.

I have written three AWS Lambda static site generator plugins to demonstrate the concept and to serve as templates for new plugins:

with automatic updates on changes in CodeCommit Git repository

A number of CloudFormation templates have been published that generate AWS infrastructure to support a static website. I’ll toss another one into the ring with a feature I haven’t seen yet.

In this stack, changes to the CodeCommit Git repository automatically trigger an update to the content served by the static website. This automatic update is performed using CodePipeline and AWS Lambda.

This stack also includes features like HTTPS (with a free certificate), www redirect, email notification of Git updates, complete DNS support, web site access logs, infinite scaling, zero maintenance, and low cost.

One of the most exciting features is the launch-time ability to specify an AWS Lambda function plugin (ZIP file) that defines a static site generator to run on the Git repository site source before deploying to the static website. A sample plugin is provided for the popular Hugo static site generator.

Here is an architecture diagram outlining the various AWS services used in this stack. The arrows indicate the major direction of data flow. The heavy arrows indicate the flow of website content.

fast, easy, and slightly dangerous recursive deletion of a domain’s DNS

Amazon Route 53 currently charges $0.50/month per hosted zone for your first 25 domains, and $0.10/month for additional hosted zones, even if they are not getting any DNS requests. I recently stopped using Route 53 to serve DNS for 25 domains and wanted to save on the $150/year these were costing.

Amazon’s instructions for using the Route 53 Console to delete Record Sets and a Hosted Zone make it look simple. I started in the Route 53 Console clicking into a hosted zone, selecting each DNS record set (but not the NS or SOA ones), clicking delete, clicking confirm, going back a level, selecting the next domain, and so on. This got old quickly.

Being lazy, I decided to spend a lot more effort figuring out how to automate this process with the aws-cli, and pass the savings on to you.

Steps with aws-cli

Interview conducted in writing July-August 2016.

[Kira] Good morning, Papa! Thank you for inviting me.

CloudStatus allows users to check the service availability of any AWS region. On opening the skill, Alexa says which (if any) regions are experiencing service issues or were recently having problems. Then the user can inquire about the services in specific regions.

This skill was made at my dad’s request. He wanted to quickly see how AWS services were operating, without needing to open his laptop. As well as summarizing service issues for him, my dad thought CloudStatus would be a good opportunity for me to learn about retrieving and parsing web pages in Python.

All the data can be found in more detail at status.aws.amazon.com. But with CloudStatus, developers can hear AWS statuses with their Amazon Echo. Instead of scrolling through dozens of green checkmarks to find errors, users of CloudStatus listen to which services are having problems, as well as how many services are operating satisfactorily.

CloudStatus is intended for anyone who uses Amazon Web Services and wants to know about current (and recent) AWS problems. Eventually it might be expanded to talk about other clouds as well.

[Kira] Just say “Alexa, enable CloudStatus skill”! Ask Alexa to “open CloudStatus” and she will give you a summary of regions with problems. An example of what she might say on the worst of days is:

set up each local CodeCommit repository clone to use a specific cross-account IAM role with git clone --config and aws codecommit credentials-helper

https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-git-remote-codecommit.html

When I started testing AWS CodeCommit, I used the Git ssh protocol with uploaded ssh keys to provide access, because this is the Git access mode I’m most familiar with. However, using ssh keys requires each person to have an IAM user in the same AWS account as the CodeCommit Git repository.

In my personal and work AWS usage, each individual has a single IAM user in a master AWS account, and those users are granted permission to assume cross-account IAM roles to perform operations in other AWS accounts. We cannot use the ssh method to access Git repositories in other AWS accounts, as there are no IAM users in those accounts.

AWS CodeCommit comes to our rescue with an alternative https access method that supports Git Smart HTTP, and the aws-cli offers a credential-helper feature that integrates with the git client to authenticate Git requests to the CodeCommit service.

In my tests, this works perfectly with cross-account IAM roles. After the initial git clone command, there is no difference in how git is used compared to the ssh access method.

Most of the aws codecommit credential-helper examples I’ve seen suggest you set up a git config --global setting before cloning a CodeCommit repository. A couple even show how to restrict the config to AWS CodeCommit repositories only so as to not interfere with GitHub and other repositories. (See “Resoures” below)

I prefer to have the configuration associated with the specific Git repositories that need it, not in the global setting file. This is possible by passing in a couple --config parameters to the git clone command.

Create/Get CodeCommit Repository

Ten commands to launch a minimal, functioning API Gateway

As of this publication date, the Amazon API Gateway is pretty new and the aws-cli interface for it is even newer. The API and aws-cli documentation at the moment is a bit rough, but this article outlines steps to create a functioning API Gateway with the aws-cli. Hopefully, this can help others who are trying to get it to work.

Goals

I regularly have a need to redirect browsers from one domain to another, whether it’s a vanity domain, legacy domain, “www” to base domain, misspelling, or other reasons.

I usually do this with an S3 bucket in website mode with a CloudFront distribution in front to support https. This works, performs well, and costs next to nothing.

Now that the Amazon API Gateway has aws-cli support, I was looking for simple projects to test out so I worked to reproduce the domain redirect. I found I can create an API Gateway that will redirect a hostname to a target URL, without any back end for the API (not even a Lambda function).

I’m not saying the API Gateway method is better than using S3 plus CloudFront for simple hostname redirection. In fact, it costs more (though still cheap), takes more commands to set up, and isn’t quite as flexible in what URL paths get redirected from the source domain. It does, however, work and may be useful as an API Gateway aws-cli example.

Assumptions

use the aws-cli to suspend an AWS Lambda function processing an Amazon Kinesis stream, then resume it again

At Campus Explorer we are using AWS Lambda extensively, with sources including Kinesis, DyanmoDB, S3, SNS, CloudFormation, API Gateway, custom events, and schedules.

This week, Steve Caldwell (CTO and prolific developer) encountered a situation which required pausing an AWS Lambda function with a Kinesis stream source, and later resuming it, preferably from the same point at which it had been reading in each Kinesis shard.

We brainstormed a half dozen different ways to accomplish this with varying levels of difficulty, varying levels of cost, and varying levels of not-quite-what-we-wanted-ness.

A few hours later, Steve shared that he had discovered the answer (and suggested I pass on the answer to you).

retiring “Git with gitolite by Alestic” on AWS Marketplace

Back in 2011 when the AWS Marketplace launched, Amazon was interested in having some examples of open source software listed in the marketplace, so I created and published Git with gitolite by Alestic.

This was a free AWS Marketplace product that endeavored to simplify the process of launching an EC2 instance running Git for private repositories, with ssh access managed through the open source gitolite software.

Though maintaining releases of this product has not been overly burdensome, I am planning to discontinue this work and spend time on other projects that would likely be more beneficial to the community.

Current Plan

stops even all-powerful IAM admin users in their tracks

I run some SNS topics as a public service where anybody can subscribe their own AWS Lambda functions, SQS queues, and email addresses.

If one of these SNS topics were to be accidentally deleted, I could recreate it with the same name and ARN. However, all of the existing user subscriptions would be lost and I would not be able to restore them myself. Each of the hundreds of users would have to figure out what happened and re-subscribe the appropriate targets with the correct permissions.

I don’t want these SNS topics to be deleted. Ever.

taking away some read-only permissions that Amazon allows

Amazon has created an IAM Managed Policy named ReadOnlyAccess, which grants read-only access to active resources on most AWS services. At Campus Explorer, we depend on this convenient managed policy for our read-only roles.

However, our use of “read-only” doesn’t quite match up with the choices that Amazon made when creating this policy. This isn’t to say that Amazon is wrong, just that our concept of “read-only” differs slightly from Amazon’s.

The biggest difference is that we want our read-only roles to be able to see the architecture of our AWS systems and what resources are active, but we would prefer that the role not be able to read sensitive data from DynamoDB, S3, Kinesis, SQS queue messages, CloudFormation template parameters, and the like.

For example, third party services often ask for a ReadOnlyAccess role to allow them to analyze your AWS account and provide helpful tips on how to improve security or cost control. This sounds good, but do you really want them to be reading messages from Kinesis streams and SQS queues or downloading the contents of S3 objects?

adding in some read-only permissions that Amazon missed

Amazon has created a Managed Policy in IAM named ReadOnlyAccess, which grants read-only access to AWS resources and API calls that make no changes to the account. At Campus Explorer, we depend on this convenient managed policy for our read-only roles–though we add a few Deny statements as we don’t believe, for example, that pulling messages off of an SQS queue really belongs in a read-only role.

In theory, and mostly in practice, Amazon manages this managed policy so that we don’t have to keep up with all of the changing API calls from new services and new features in existing services.

Today at AWS re:Invent, Amazon has announced the ability to schedule AWS Lambda function invocations using cron syntax. Yay!

I’m happy to announce that the The Unreliable Town Clock is now using this functionality behind the scenes to send the chime messages to the public SNS topic every quarter hour, in both us-east-1 and us-west-1.

No significant changes should be perceived by the hundreds of subscribers to the Unreliable Town Clock public SNS topic.

If you are already using the Unreliable Town Clock, what should you do?

It took a while, but the Unreliable Town Clock finally lived up to its name. Surprisingly, the fault was not mine, but Amazon’s.

deceptively simple web service with super powers

TimerCheck.io is a fully functional, fully scalable microservice built on the just-released Amazon API Gateway and increasingly popular AWS Lambda platforms.

TimerCheck.io is a public web service that maintains a practically unlimited number of countdown timers with one second resolution and no practical limit to the number of seconds each timer can run.

New timers can be created on a whim and each timer can be reset at any time to any number of seconds desired, whether it is still running or has already expired.

Synopsis

Let’s begin with an example to demonstrate the elegant simplicity of the TimerCheck.io interface.

1. Set timer - Any request of the following URL sets a timer named “YOURTIMERNAME” to start counting down immediately from 60 seconds:

https://timercheck.io/YOURTIMERNAME/60

You may click on that link now, or hit a URL of the same format with your own timer name and your chosen number of seconds. You may use a browser, a command like curl, or your favorite programming language.

2. Poll timer - The following URL requests the status of the above timer. Note that the only difference in the URL is that we have dropped the seconds count.

https://timercheck.io/YOURTIMERNAME

If the named timer is still running, TimerCheck.io will return HTTP Status code 200 OK, along with a JSON structure containing information like how many seconds are left.

If the timer has expired, TimerCheck.io will return an HTTP status code 504 Timeout.

That’s it!

No, really. That’s the entire API.

Interested in adding scheduled job monitoring (dead man’s switch) to the existing monitoring and alerting framework you are already using (Nagios, Sensu, Zenoss, Zabbix, Monit, Pingdom, Montastic, Ruxit, and the like)?

Last month I wrote about how I use Cronitor.io to monitor scheduled events with an example using an SNS Topic and AWS Lambda.

This week I spent a few hours building a simple web service that enables any polling-based monitor software or service to automatically support alerting when a target event has not occurred in a desired timeframe.

The new web service is built on infrastructure technologies that are reliably maintained and scaled by Amazon: