Alestic.com - A Personal AWS Blog by Eric Hammond

retiring “Git with gitolite by Alestic” on AWS Marketplace

Back in 2011 when the AWS Marketplace launched, Amazon was interested in having some examples of open source software listed in the marketplace, so I created and published Git with gitolite by Alestic.

This was a free AWS Marketplace product that endeavored to simplify the process of launching an EC2 instance running Git for private repositories, with ssh access managed through the open source gitolite software.

Though maintaining releases of this product has not been overly burdensome, I am planning to discontinue this work and spend time on other projects that would likely be more beneficial to the community.

Current Plan

stops even all-powerful IAM admin users in their tracks

I run some SNS topics as a public service where anybody can subscribe their own AWS Lambda functions, SQS queues, and email addresses.

If one of these SNS topics were to be accidentally deleted, I could recreate it with the same name and ARN. However, all of the existing user subscriptions would be lost and I would not be able to restore them myself. Each of the hundreds of users would have to figure out what happened and re-subscribe the appropriate targets with the correct permissions.

I don’t want these SNS topics to be deleted. Ever.

taking away some read-only permissions that Amazon allows

Amazon has created an IAM Managed Policy named ReadOnlyAccess, which grants read-only access to active resources on most AWS services. At Campus Explorer, we depend on this convenient managed policy for our read-only roles.

However, our use of “read-only” doesn’t quite match up with the choices that Amazon made when creating this policy. This isn’t to say that Amazon is wrong, just that our concept of “read-only” differs slightly from Amazon’s.

The biggest difference is that we want our read-only roles to be able to see the architecture of our AWS systems and what resources are active, but we would prefer that the role not be able to read sensitive data from DynamoDB, S3, Kinesis, SQS queue messages, CloudFormation template parameters, and the like.

For example, third party services often ask for a ReadOnlyAccess role to allow them to analyze your AWS account and provide helpful tips on how to improve security or cost control. This sounds good, but do you really want them to be reading messages from Kinesis streams and SQS queues or downloading the contents of S3 objects?

adding in some read-only permissions that Amazon missed

Amazon has created a Managed Policy in IAM named ReadOnlyAccess, which grants read-only access to AWS resources and API calls that make no changes to the account. At Campus Explorer, we depend on this convenient managed policy for our read-only roles–though we add a few Deny statements as we don’t believe, for example, that pulling messages off of an SQS queue really belongs in a read-only role.

In theory, and mostly in practice, Amazon manages this managed policy so that we don’t have to keep up with all of the changing API calls from new services and new features in existing services.

Today at AWS re:Invent, Amazon has announced the ability to schedule AWS Lambda function invocations using cron syntax. Yay!

I’m happy to announce that the The Unreliable Town Clock is now using this functionality behind the scenes to send the chime messages to the public SNS topic every quarter hour, in both us-east-1 and us-west-1.

No significant changes should be perceived by the hundreds of subscribers to the Unreliable Town Clock public SNS topic.

If you are already using the Unreliable Town Clock, what should you do?

It took a while, but the Unreliable Town Clock finally lived up to its name. Surprisingly, the fault was not mine, but Amazon’s.

deceptively simple web service with super powers

TimerCheck.io is a fully functional, fully scalable microservice built on the just-released Amazon API Gateway and increasingly popular AWS Lambda platforms.

TimerCheck.io is a public web service that maintains a practically unlimited number of countdown timers with one second resolution and no practical limit to the number of seconds each timer can run.

New timers can be created on a whim and each timer can be reset at any time to any number of seconds desired, whether it is still running or has already expired.

Synopsis

Let’s begin with an example to demonstrate the elegant simplicity of the TimerCheck.io interface.

1. Set timer - Any request of the following URL sets a timer named “YOURTIMERNAME” to start counting down immediately from 60 seconds:

https://timercheck.io/YOURTIMERNAME/60

You may click on that link now, or hit a URL of the same format with your own timer name and your chosen number of seconds. You may use a browser, a command like curl, or your favorite programming language.

2. Poll timer - The following URL requests the status of the above timer. Note that the only difference in the URL is that we have dropped the seconds count.

https://timercheck.io/YOURTIMERNAME

If the named timer is still running, TimerCheck.io will return HTTP Status code 200 OK, along with a JSON structure containing information like how many seconds are left.

If the timer has expired, TimerCheck.io will return an HTTP status code 504 Timeout.

That’s it!

No, really. That’s the entire API.

Interested in adding scheduled job monitoring (dead man’s switch) to the existing monitoring and alerting framework you are already using (Nagios, Sensu, Zenoss, Zabbix, Monit, Pingdom, Montastic, Ruxit, and the like)?

Last month I wrote about how I use Cronitor.io to monitor scheduled events with an example using an SNS Topic and AWS Lambda.

This week I spent a few hours building a simple web service that enables any polling-based monitor software or service to automatically support alerting when a target event has not occurred in a desired timeframe.

The new web service is built on infrastructure technologies that are reliably maintained and scaled by Amazon:

easier, simpler, faster, better

Seven months ago I published the lambdash AWS Lambda Shell Hack that lets you run shell commands to explore the environment in which AWS Lambda functions are executed.

I also posted samples of command output that show fascinating properties of the AWS Lambda runtime environment.

In the last seven months, Amazon has released new features and enhancements that have made a completely new version of lambdash possible, with many benefits including:

get alerted when an expected event does NOT happen

Last week I announced the availability of a public SNS Topic that may be used to run AWS Lambda functions on a recurring schedule. To encourage folks to realize the implications of a free community service maintained by an individual, I named it the “Unreliable Town Clock”.

Even with this understanding, some folks in the AWS community have (again) placed their faith in me and are already starting to depend on the Unreliable Town Clock public SNS Topic to drive their own AWS Lambda functions and SQS queues, and I want to make sure this service is as reliable as I can reasonably make it.

Here are some of the steps I have taken to increase the reliability of the Unreliable Town Clock:

public SNS Topic with a trigger event every quarter hour

Update 2015-10-08: Amazon has released AWS Lambda Scheduled Functions. I recommend using that feature to schedule AWS Lambda functions. In fact, the Unreliable Town Clock switched to use this feature behind the scenes, the day it was announced.

Scheduled executions of AWS Lambda functions on an hourly/daily/etc basis is a frequently requested feature, ever since the day Amazon introduced the service at AWS re:Invent 2014.

Until Amazon releases a reliable, premium cron feature for AWS Lambda, I’m offering a community-built alternative which may be useful for some non-critical applications.

us-east-1:

arn:aws:sns:us-east-1:522480313337:unreliable-town-clock-topic-178F1OQACHTYF

us-west-2:

arn:aws:sns:us-west-2:522480313337:unreliable-town-clock-topic-N4N94CWNOMTH

Background

Beyond its event-driven convenience, the primary attraction of AWS Lambda is eliminating the need to maintain infrastructure to run and scale code. The AWS Lambda function code is simply uploaded to AWS and Amazon takes care of providing systems to run on, keeping it available, scaling to meet demand, recovering from infrastructure failures, monitoring, logging, and more.

The available methods to trigger AWS Lambda functions already include some powerful and convenient events like S3 object creation, DynamoDB changes, Kinesis stream processing, and my favorite: the all-purpose SNS Topic subscription.

Even so, there is a glaring need for code that wants to run at regular intervals: time-triggered, recurring, scheduled event support for AWS Lambda. Attempts to to do this yourself generally ends up with having to maintain your own supporting infrastructure, when your original goal was to eliminate the infrastructure worries.

Unreliable Town Clock (UTC)

The Unreliable Town Clock (UTC) is a new, free, public SNS Topic (Amazon Simple Notification Service) that broadcasts a “chime” message every quarter hour to all subscribers. It can send the chimes to AWS Lambda functions, SQS queues, and email addresses.

As I create architectures that include AWS Lambda functions, I find there are situations where I just want to know that the AWS Lambda function is getting invoked and to review the exact event data structure that is being passed in to it.

I found that a simple “echo” function can be dropped in to copy the AWS Lambda event to the console log (CloudWatch Logs). It’s easy to review this output to make sure the function is getting invoked at the right times and with the right data.

The Alestic.com web site has been redesigned. The old design was going on 8 years old. The new design is:

• Mobile friendly (‘cause Google says so)

• Always secure (‘cause Google says so)

• Fast (‘cause Google says so)

Ok, so I still have a little improvement remaining in the fast dimension, but at least the site is static now and served through a CDN.

Since fellow geeks care, here are the technologies currently employed:

The aws-cli documentation and command line help text have not been updated yet to include the syntax for subscribing an AWS Lambda function to an SNS topic, but it does work!

Here’s the format:

aws sns subscribe \
  --topic-arn arn:aws:sns:REGION:ACCOUNT:SNSTOPIC \
  --protocol lambda \
  --notification-endpoint arn:aws:lambda:REGION:ACCOUNT:function:LAMBDAFUNCTION

where REGION, ACCOUNT, SNSTOPIC, and LAMBDAFUNCTION are substituted with appropriate values for your account.

For example:

Today, Amazon announced that AWS Lambda functions can be subscribed to Amazon SNS topics.

This means that any message posted to an SNS topic can trigger the execution of custom code you have written, but you don’t have to maintain any infrastructure to keep that code available to listen for those events and you don’t have to pay for any infrastructure when the code is not being run.

This is, in my opinion, the first time that Amazon can truly say that AWS Lambda is event-driven, as we now have a central, independent, event management system (SNS) where any authorized entity can trigger the event (post a message to a topic) and any authorized AWS Lambda function can listen for the event, and neither has to know about the other.

Making this instantly useful is the fact that there already are a number of AWS services and events that can post messages to Amazon SNS. This means there are a lot of application ideas that are ready to be implemented with nothing but a few commands to set up the SNS topic, and some snippets of nodejs code to upload as an AWS Lambda function.

Unfortunately…