Getting Started with EC2 Container Service (ECS)

If that is the case you were likely thrilled to hear the announcement of EC2 Container Service (ECS) at Re:Invent in November. Similar to Kubernetes this service helps you run an entire cluster where containers can be deployed, scaled, and managed based on your requirements.

The service was released in preview mode at the end of December, so today I’m bringing you a quick start to using the preview.

Where to Start

1) First you’ll need to sign up for the ECS Preview here
2) Then download and Install the latest AWS CLI here

Terminology

Here’s a quick overview of terminology from the AWS Documentation. Amazon ECS contains the following components:

• Cluster: a logical grouping of container instances that you can place tasks on.
• Container Instance: an Amazon EC2 instance that is running the Amazon ECS agent and has been registered into a cluster.
• Task Definition: a description of an application that contains one or more container definitions.
• Task: an instantiation of a task definition that is running on a container instance.
• Container: a Linux container that was created as part of a task.

Clusters

AWS creates a cluster called ‘default’ when you sign up for the service. If you want to create your own cluster you can, but at the moment you are limited to 2 clusters per account. More on ECS limits here.

The first step is to create your own cluster from the CLI:

aws ecs create-cluster --cluster-name yourCluster

*NOTE* If you chose to use your own cluster you’ll need to specify the --cluster yourCluster option on all the commands shown below, otherwise they will run in the default cluster.

While a cluster is created, they do not add any machines to it. Additionally you’ll want an IAM Role, Security Group and Instance profile for the cluster instances. You can do all this manually, but AWS does provide a nice cloud formation script here.

*NOTE* If you chose to create a new cluster, make sure you use that name in the ClusterName field for this CF script.

Once you’ve run this script you should have at least one machine registered with your cluster. Run this command to verify. Please note that instance start up can take several minutes so you won’t see them appear until the instance is in a running state and passed its’ status checks.

aws ecs list-container-instances

Tasks

Now you’ll need to create a task definition. This tells ECS which images to create instances from and the share of cluster resources for each as well as a few other parameters covered below. Note each definition can have multiple containers defined and even linked together.

The syntax for a task definition is as follows:

[
  {
    "name": "string",
    "image": "string",
    "cpu": integer,
    "memory": integer,
    "links": ["string", ...],
    "portMappings": [
      {
        "containerPort": integer,
        "hostPort": integer
      }
      ...
    ],
    "essential": true|false,
    "entryPoint": ["string", ...],
    "command": ["string", ...],
    "environment": [
      {
        "name": "string",
        "value": "string"
      }
      ...
    ]
  }
  ...
]

This is taken from the AWS manual page here.

You’ll notice that not all Docker parameters are present. One that I’d like to see support for is the –v or volume parameter which allows mounting a host folder as a folder in the container. I can understand why this is not included as EC2 Container Service may decide to run it on any number of machines in the cluster and it’s not a given that they are all the same. Even if they are, keeping the filesystems in sync is a challenge of its own.

That being said, and maybe it’s just me, but I suspect others will find having some capability for volume mounts useful as well. Hopefully, it’s coming in the next release.

I also think there is an opportunity here for AWS to provide some value added capabilities. For example allowing a copy or mount of an S3 path.

So the parameters we do have are pretty standard for Docker users:

• A name for your container and an image to start from.
• cpu and memory for specifying the cluster resources to use.
• links to establish communication between containers and port mapping to expose ports outside the container.
• Standard entryPoint and command you’ll recognize from Dockerfiles.
• environment for environment variables.

One last parameter that is specific to ECS is the essential option. If true this will stop the entire task on failure, otherwise the task and other containers defined will continue to be deployed.

Example Time

Go to Docker Hub and find yourself an image. For this post I am using the Node.js express framework Hello World app, which can be found here.

Now you’ll need to create a task definition and specify the relevant ports and environment variables:

 [ 
  {
    "image": "google/nodejs-hello",
    "name": "nodejs-hello",
    "cpu": 10,
    "memory": 500,
    "essential": true,
    "entryPoint": [
      "/bin/sh"
    ],
    "environment": [],
    "portMappings": [
      {
        "containerPort": 8080,
        "hostPort": 8080
      }
    ]
  }
]

You can either save this as a JSON file and run a command like so:

aws ecs register-task-definition --family hello --container-definitions file://< path_to_json_file >/taskfile.json

Or the uglier version of stuffing the definition into the command line itself:

aws ecs register-task-definition --family hello --container-definitions "[ { \"image\": \"google/nodejs-hello\", \"name\": \"nodejs-hello\", \"cpu\": 10, \"memory\": 500, \"essential\": true, \"entryPoint\": [ \"/bin/sh\", \"-c\" ], \"environment\": [], \"portMappings\": [ { \"containerPort\": 8080, \"hostPort\": 8080 } ] } ]"

Next we run the command. Note the :1 after the task definition name hello. This denotes the revision number. If you make changes and register the task more than once you’ll need to increment that. Use the list-task-definitions command to see what’s registered at any time.
aws ecs run-task --task-definition hello:1 --count 1

Let’s get the task’s UUID so we can do some inspection. The UUID is everything after the final forward slash.

aws ecs list-tasks

Now let’s describe it to verify that it is running (It may be PENDING for a few moments)

aws ecs describe-tasks --task < Task UUID >

Once it’s running the tricky part is checking the results in your browser. If you only have one box running this is no problem, just use that machine’s IP address. However, if you have a cluster of multiple machines you’ll need to first grab the Container Instance UUID from the last command. Look at the value for containerInstanceARN and take everything after the last forward slash.

Then you can use the describe-container-instances command to get the “ec2InstanceId”.

aws ecs describe-container-instances --container-instances < Instance UUID >

Finally you can use that instance id to get the IP address from the CLI or the console.

*NOTE* Don’t forget to open the port in the Security Group ☺

Since my task mapped the port to 8080 I can find the results in my browser at

http://< Container Instance IP >:8080

If all goes well you will now see a “Hello World” screen.

This is another area where I hope AWS will make some improvements in the near future. We really need some abstraction, load balancer or DNS, that can sit in front of the cluster and find the services that are have content available on a certain port. Barring that I’d at least like to see a management interface that would show what is running where in the cluster.

Clean Up

Now to tear everything down when you are done. First we need to stop the task

aws ecs stop-task --task

We can leave the task registered for later use and in fact you’ll have to because the CLI currently does not support deregistering tasks, but there is a command holding it’s place.

You are free to stop your cluster container instances while not in use. If you terminate them or the CF stack completely you’ll need to do some additional clean up in the ECS service.

First get the container instance UUID from the list-container-instances command. Note that the instance still shows even after it’s been terminated

aws ecs list-container-instances

Then run the deregister-container-instance command

aws ecs deregister-container-instance --container-instance

Finally, if you created your own cluster you can delete it with the following
aws ecs delete-cluster --cluster yourCluster

Keep Learning

How to Create a full ECS Cluster

In this post we’ll walk through creating an ECS cluster without the wizard. This should give you a better understanding of how ECS works with EC2 and the rest of the AWS ecosystem.