Amazon ECS: Load Balancing for Containers

Another getting started video for Amazon ECS, this time talking about how to put a load balancer in front of a group of containers

Infamous comment on this video, lmao:

Script:

Hey all, I’m Nathan Peck, and I’m a developer advocate for EC2 Container Service.

In the last segment we covered the basic concepts of ECS, how to get your container images uploaded to ECR, associated with a task, and then launched as a service across your cluster. So now that we have all the ingredients to launch a long running container such as an API or other HTTP serving container the next step is getting network traffic to the container.

Most HTTP servers are going to want to listen for traffic on port 80, but in a traditional environment only one process is going to be able to bind to that port. The old model for solving this problem was to run a web server like Apache or Nginx which accepted traffic on port 80 and forwarded to an appropriate backend process based on some criteria such as the hostname from the request.

Docker works with a different abstraction. Inside your docker container resources such as ports are all namespaced, so a single instance can be running many containers that all bind to the same port. From the perspective of each containerized application it is binding to port 80 and is the only application receiving traffic on that port. In reality none of the containers are listening on port 80. Instead docker provides a networking bridge which maps the container ports to available ports on the host instance.

You can arbitrarily assign a static port for the mapping, but the recommendation for maximum flexibility and convenience is to let docker assign a random port to the container. In an ECS task definition this is accomplished by specifying a container port but leaving the host port as zero. If you launch your containers as a service using EC2 Container Service one of the features that you get for free is integration with Target Groups. Target Groups are basically just a list of instances and ports on those instances.

You can see here that my service has a target group, and if I click through to view the details of the service there are a number of different instances and randomly assigned ports on those instances. Target Groups are designed to integrate seamlessly with the new Application Load Balancer or ALB. The ALB is a smart, level 7 load balancer which can inspect incoming requests and make routing decisions based on things like hostname or path.

Here is an example of an ALB that I’ve set up to define a REST API powered by microservices in ECS. As you can see I’ve specified path based rules so that if requests match /api/users then that request should be directed to a target from the users target group, but if the request matches /api/messages then the request should be directed to a target from the messages target group.

Note that I also could have specified rules by hostname, so I could have had this one ALB serving requests for two different subdomains like users.mycompany.com and messages.mycompany.com

To summarize the ALB will check each incoming request to see what rule it matches, and then round robin it out to one of the targets from the matching rule’s target group. So in this way ALB serves as a service discovery gateway to your backend containers. You can run the ALB either accessible to the public in order to direct public traffic to your containers, as with this example API, or you can also run a private ALB that lives inside the private subnet of your VPC and is only accessible to your other services. This is usually most useful for helping containers from one service communicate with containers from another service. One beautiful thing about the ALB and target groups is how the function together seamlessly when you wish to stop a container, or roll out a new version of a container across your cluster.

Each target on the target group has a status, and this status is set by ECS and used by the ALB. When you ask ECS to stop a container, or you roll out a new task definition version and ECS is automatically replacing old tasks with new tasks it will place each task that it is about to stop into the DRAINING state. This tells the ALB to stop sending any more traffic to that target, and wait for existing requests to complete and connections to close. Once the container is no longer serving any more requests it will be safely stopped, and none of your inflight requests will be dropped.

This mechanism allows you to have safe, zero downtime deploys with no impact to your users. To recap, ECS has networking integrations that make it easy to deploy containers without needing to worry about configuring how traffic gets to the containers.

Instead of needing to keep track of ports and placements, you just ask ECS to place a container, and attach the container’s target group to a rule in an ALB. The specifics of what instances and what ports are used are all handled by ECS, leaving developers with the freedom to focus on building features.