Cost effective container based application management

The modern software industry is rapidly adapting to Serverless Computing to taste its benefits in terms of Scalability and Cost Efficiency. But there are situations that we cannot solely depends upon Serverless. In such situations, a hybrid approach where a collaboration of containers and serverless functions, could come in handy. Without a doubt, container based applications can be used to deliver cost efficiency in a software system. But how can you make the container management process as cost effective as possible?

Recently, my customer wanted to develop several white label products of one of our web applications. By that time, we had already setup our application to run inside a container that is managed by AWS EC2 Container Service(ECS). ECS lets us easily manage docker container across a fleet of EC2 instances. The web application communicates with few other microservices that were also hosted in the same cluster of the ECS. To cater for our customer’s requirement we planned to use the newly introduced Application Load Balancer(ALB) with ECS.

The new ALB makes it easier to run multiple applications on a fleet of EC2 instances by specifying dynamic ports in the ECS task definition. When a new task is added to the fleet, ECS scheduler automatically adds it to the ALB using this port. Since ALB operates at the application layer, it offers high-performance load balancing with different routing options. Moreover, A single ALB can be shared among different ECS services.

Currently, ALB supports three routing options.

1. Path Based Routing
2. Content Based Routing
3. Host Based Routing (Introduced on 5th of April 2017)

Out of these three, we chose Host Based Routing as it removed the need of a proxy server container(earlier we used nginx) that routed traffic to services in our cluster based on virtual hosts. Host Based Routing made our architecture much simpler than it was before. Look at the following diagram.

Overview of ALB and ECS

Application Load Balancer acts as the entry point for clients. It’s DNS name is mapped to the domain names of the white-labeled products via Amazon’s DNS service i.e. Route53. A Listener listens to requests from clients on a specific protocol/port and forward to one or more target groups based on rules. A rule determines how to route a request based on the different routing options. In our case, it uses the host name of the request to route to a target group. A target group comprises of targets i.e. EC2 instances with the ECS agent running(ECS instances). A target can be registered with many target groups. This allows us to use the same ECS instances for many target groups thus leverage the ECS instances. ECS spins up tasks/containers of product1 and product2 at any ECS instances.

Leveraging ECS instances

When it comes to implementation, firstly we created target groups for each white label products(Let’s assume we have 2 white label products). You can find target groups under the EC2 section of the AWS console.

Creating Target Groups

Then we created a new Application Load Balancer and attached a default target group to it. All traffic that reaches ALB’s port 80 is bound to the default target group. (Product1TargetGroup)

Configuring ALB

Then we edited the rules of the default listener that was created along with the ALB. This is the place we define our rules for directing traffic to the related ECS instances in the related target group.

Defining rules for the listener

According to above rules, if the host url of a request is product1.com, it will be sent over to the Product1TargetGroup. If the host url is product2.com it will go to Product2TargetGroup.

Next up was to configure the ECS to work with ALB. We created new task definitions for each product inside the cluster. In the container configuration section of a task definition, we need to add port mapping as follows.

Task Definition Container Port Mapping

When we set the host port to ‘0’ the host will be assigned a dynamic port number in the ephemeral port range. Application Load Balancer uses this dynamic port to route traffic to the container.

Then we created and deployed separate services for each task definition. When creating a service, we need to associate it with the ALB and the respective target group.

Mapping Product1 service to Product1TargetGroup

Similarly we created other services i.e Product2 service, WebSnap service, and Web Sync service by mapping them to the same ALB and selecting the correct target group. (WebSnap and WebSync services are utility microservices used by all the white label products)

In addition to this setup, we added auto scaling at two levels to ensure the high availability of the containers.

1. ECS instance level
2. Service level

Setting up Auto-Scaling

At creation of the ECS cluster, we can setup CloudWatch alarms to scale in and scale out the ECS cluster. We can also define scale in /scale out policies for the services at service creation. For more implementation details, refer this great post.

Now that the configuration part is completed, our ALB accepts traffic from white label product domains and direct them to the corresponding containers. Besides we have setup few other utility microservices for capturing web snapshots, Synching real-time client data and etc… The utility microservices are shared by all the white label products.

The web snapshot process requires high memory. So, we added a memory optimized EC2 instance into the ECS cluster and configured a separate target group for it. WebSnap service was attached to that target group and made sure all the Websnap tasks/containers are created on that Memory optimized instance.

Now our system is complete. It is highly scalable and can easily accommodate any number of future white label products just by adding a new target group and a host based rule on to the ALB.