![\"A](\"/img/picture-1.jpg\")
As someone who works with AWS on a day-to-day basis, It's important to stay up to date with all the changes and new features of the different services on the platform. That's how one recent announcement caught my eye - the new capability of local testing of ECS.
\nI am particularly interested in ECS local testing because it is a feature that bridges a known gap. In recent years, containers have become increasingly popular. Without a doubt Kubernetes has been the leading standard for containers deployment. Whilst Kubernetes is feature-rich and powerful, it attracts users because the same Kubernetes can be run locally on a developer's desktop.
\nWhen developing a feature, I want to be able to test it as quickly and quietly as possible. A tool that allows me to perform rapid and quiet testing while I am trying to figure things out is very helpful. Being able to deploy onto Kubernetes means that I can test the same manifests locally, making sure that all changes can be tested before the deployments are scrutinized by my peers and potentially impact shared resources, it gives me confidence that I am not wasting my teams' resources and time.
\nECS was first announced in 2013 and has been the go-to tool for containers orchestration on AWS (as evidenced by the number of new features on the public containers roadmap). While others can get on the heated -- and even religious -- debate on the virtue of Kubernetes vs ECS, the straightforwardness of ECS, the tight and intuitive integration of ECS with the other AWS features, is very attractive. For someone who is heavily leveraging AWS, and needs some container orchestration for part of workloads, more often than not ECS is sufficient.
\nUntil now, it has been challenging to encourage developers to work closely with ECS, because developers cannot test their work unless they deploy directly onto AWS ECS clusters that are running on AWS. Prior to this release, it is necessary to raise an ECS cluster on AWS and test the task definitions on the cloud to verify ECS workloads. Whilst this makes sense for feature integration and performance testing, it is a significant entry barrier for early development. Not every developer will have direct access to ECS resources, and to follow through with the CI/CD workflow for every minute change tends to be too slow for feedback. To workaround the overhead, developers often choose to develop with docker-compose, and leave the ECS testing for a later state. This delay can make deployments more error-prone and ECS more difficult to adopt. With the new feature, the barrier for ECS adoption seems to be significantly reduced.
\nThe new ECS feature supports testing of network modes, secrets, volumes and credentials without the requirement of launching an ECS Task or an ECS Service on AWS and doing everything locally on the work-station. This is achieved by ecs-cli, which is a wrapper of Docker and Docker Compose that simulates a fully functional ECS environment via a special container. With this wrapper, the user can locally get mock endpoints and resources available to test tasks and verify application functionality.
\nThe usual tools associated with a container-based development environment still apply; namely, Docker and Docker Compose. On top of that; installing ecs-cli command line tool.
\nValid AWS credentials are required too:
\nFor this walkthrough, I have created several ECR Repositories:
\nexport AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxx\nexport AWS_SECRET_ACCESS_KEY=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy\nexport AWS_DEFAULT_REGION=us-west-1
In each repository, there are several images e.g. app2
\n![](\"/img/picture-2.jpg\")
The same images are accessible on the command line with ecs-cli command line tool. For example, the following will list the available images, along with respective metadata.
\necs-cli images![](\"/img/picture-3.jpg\")
In order to pull the images from the ECR to my workstation I need AWS credentials and Docker authenticatication with the remote repository. The following command will initiate and perform login from my local machine to ECR
\n$(aws ecr get-login --no-include-email --region us-west-1)Before local execution of ECS takes place, I need to define the run-time parameters for the containers, such as the memory and CPU requirements and the ports opened. This is done in the ECS task definition. While the task definitions need to be created on AWS initially, I can download the task definition as JSON file to my local machine or use remote task definitions for local testing. For my test application (app-main), I have already created several versions of a task definition on ECS.
\n![](\"/img/picture-4.jpg\")
ecs-cli for local testing have only 4 subcommands for now:
\n![](\"/img/picture-5.jpg\")
The following command will locally launch a Docker container based on a task definition:
\necs-cli local up --task-def-remote app-main:6app-main is the name of the task-definition and 6 is the revision of the task definition - If the version is omitted, the latest version will be used.
\nTrying out with the latest version of my task definition.
\n![](\"/img/picture-6.jpg\")
and voilà!
\necs-cli local updid a few things:
\ndocker-compose.ecs-local.yaml filedocker-compose.ecs-local.override.yml fileecs-local-network for containersamazon/amazon-ecs-local-container-endpointsdocker-compose binary to start the relevant containers as described in our Task Definitionecslocal-lab_app-main_1ECS Local uses the Docker Compose installed on the local machine to create network for containers and provision those containers. The information is captured at the generated docker-compose.ecs-local.yaml file. The ecs-local.yaml file acts as the local counterpart of the task definition that is stored on AWS.
Content of docker-compose.ecs-local.yml
\n![](\"/img/picture-7.jpg\")
Another file docker-compose.ecs-local.override.yml created is used to define a single container that emulates how credentials are exposed to a running container.
Content of docker-compose.ecs-local.override.yml
![](\"/img/picture-8.jpg\")
At examining the running processes with ecs-cli, I can see that my test application is running.
\necs-cli local ps --all![](\"/img/picture-9.jpg\")
docker ps --allHowever, if I repeat the above using native docker command, an additional container running alongside the application container is found:
\n![](\"/img/picture-10.jpg\")
The abstracted amazon-ecs-local-container-endpoints container is essentially a mock service that simulates AWS endpoints with configured ECS Task IAM Role in the task definition.
The above also shows that the application container is running on port 32771 and forwarding traffic to port 80 on the container itself. Port 80 is open because it is defined as such in the task definition.
\nWith this set-up, I can now access the container in two ways,
\nVia Local network IP address by running
\ndocker network inspect ecs-local-network![](\"/img/picture-11.jpg\")
and
\ncurl http://169.254.170.3![](\"/img/picture-12.jpg\")
Via the assigned Port on the local machine by running
\nifconfigto get our local IP address in network ecs-local-network
![](\"/img/picture-13.jpg\")
then calling against the address and assigned port
\ncurl 169.254.170.1:32771![](\"/img/picture-14.jpg\")
The container in concern is a trivial example for a container, therefore testing it is straightforward. But imagine a complex task where multiple containers are running, instead of needing to emulate the task definition locally with a docker-compose (which would be thrown away once the troubleshooting is done) there is now an AWS provided tool that translates the task definition to something that can run locally, therefore bridging some of the difficulty of making ECS usable beyond AWS.
\nOnce I have my local troubleshooting of ECS, I can stop all running containers and delete local network quite easily by another ecs-cli local command:
\necs-cli local down --all![](\"/img/picture-15.jpg\")
When I first heard about ECS local testing, I thought that AWS was stepping into the right direction, to make ECS more accessible to developers. When AWS published ECS CLI a few years ago, it seemed to be following the precedents set by competitors such as Heroku, to make the containers deployment an easy one-step process. The unenviable position for ECS, however, is that while it is not as complex Kubernetes, it is not as straightforward as other developer-friendly services such as Elastic Beanstalk. ECS appears after all, to be targeting infrastructure teams to run containers easily without the complexity, but it lacks the intuitiveness for developers to adopt it readily and enthusiastically. The local testing option of ECS appears to be an attempt to draw developers closer to the service, by emulating the environment locally, and therefore supporting local work and experiment.
\nIt may be too early to give a full verdict to the ECS local testing functionality. As the prerequisites for ECS local are still heavily based on the cloud e.g. task definitions on ECS and containers being uploaded already to ECR, it is still arguably not ready for developers who are completely new to the containers ecosystem. However, it is certainly useful to gain some local understanding of the inner workings of ECS. I look forward to new features being introduced with the local tool, and maybe examples on how to best leverage it.
\nIt is noteworthy that ecs-cli can be used for Fargate development as well. As Fargate is the serverless version of ECS, and is expected to marry the best of containers (in terms of duration, flexibility of resource allocations) and serverless (in terms of simplicity and lack of cluster management), perhaps ecs-local is another way to approach a serverless future.
\nWe have seen that ecs-cli local is a useful wrapper for Docker and Docker Compose, which you can leverage to quickly and efficiently test your ECS Task Definition configurations locally without actually deploying your task to a remote ECS Cluster.
This enables quick and more efficient development & test cycles and is overall a great addition to the evolving AWS ECS ecosystem.
\nInterested in hearing more about our containers expertise?
\nContact us at info@myops.co.il
\n- Written by Denis Astahov
","frontmatter":{"url":"walkthrough-ecs-local","seo":{"title":"Walkthrough - ECS Local: Bringing ECS to your local environment","description":"As someone who works with AWS on a day-to-day basis, It's important to stay up to date with all the changes and new features of the different services on the platform. That's how one recent announcement caught my eye - The new capability of local testing of ECS.","canonical":null,"image":{"childImageSharp":{"fluid":{"aspectRatio":2.1186440677966103,"src":"/static/12224681f2fd40bf0749423e29cf8d0c/724c8/technology-education-information-handover.jpg","srcSet":"/static/12224681f2fd40bf0749423e29cf8d0c/84d81/technology-education-information-handover.jpg 250w,\n/static/12224681f2fd40bf0749423e29cf8d0c/f0719/technology-education-information-handover.jpg 500w,\n/static/12224681f2fd40bf0749423e29cf8d0c/724c8/technology-education-information-handover.jpg 1000w,\n/static/12224681f2fd40bf0749423e29cf8d0c/0ff54/technology-education-information-handover.jpg 1200w","srcWebp":"/static/12224681f2fd40bf0749423e29cf8d0c/36ebb/technology-education-information-handover.webp","srcSetWebp":"/static/12224681f2fd40bf0749423e29cf8d0c/1d872/technology-education-information-handover.webp 250w,\n/static/12224681f2fd40bf0749423e29cf8d0c/4e6d4/technology-education-information-handover.webp 500w,\n/static/12224681f2fd40bf0749423e29cf8d0c/36ebb/technology-education-information-handover.webp 1000w,\n/static/12224681f2fd40bf0749423e29cf8d0c/9000d/technology-education-information-handover.webp 1200w","sizes":"(max-width: 1000px) 100vw, 1000px","maxHeight":471,"maxWidth":1000}}}},"title":"Walkthrough - ECS Local: Bringing ECS to your local environment","date":"2019-09-17T16:00:00.000Z","tags":["Kubernetes","AWS"],"author":{"name":"MyOps","photo":{"extension":"png","publicURL":"/static/3ff870573bc56665ee67e3cf3f5fc163/logo-small.png","childImageSharp":{"fluid":{"aspectRatio":0.8759124087591241,"src":"/static/3ff870573bc56665ee67e3cf3f5fc163/b460a/logo-small.png","srcSet":"/static/3ff870573bc56665ee67e3cf3f5fc163/d966b/logo-small.png 120w,\n/static/3ff870573bc56665ee67e3cf3f5fc163/67196/logo-small.png 240w,\n/static/3ff870573bc56665ee67e3cf3f5fc163/b460a/logo-small.png 480w,\n/static/3ff870573bc56665ee67e3cf3f5fc163/eec14/logo-small.png 596w","srcWebp":"/static/3ff870573bc56665ee67e3cf3f5fc163/35871/logo-small.webp","srcSetWebp":"/static/3ff870573bc56665ee67e3cf3f5fc163/83552/logo-small.webp 120w,\n/static/3ff870573bc56665ee67e3cf3f5fc163/2b5a3/logo-small.webp 240w,\n/static/3ff870573bc56665ee67e3cf3f5fc163/35871/logo-small.webp 480w,\n/static/3ff870573bc56665ee67e3cf3f5fc163/c0cb3/logo-small.webp 596w","sizes":"(max-width: 480px) 100vw, 480px"}}}},"image":{"childImageSharp":{"fluid":{"aspectRatio":2.127659574468085,"src":"/static/12224681f2fd40bf0749423e29cf8d0c/8c3c2/technology-education-information-handover.jpg","srcSet":"/static/12224681f2fd40bf0749423e29cf8d0c/15aed/technology-education-information-handover.jpg 300w,\n/static/12224681f2fd40bf0749423e29cf8d0c/a07a5/technology-education-information-handover.jpg 600w,\n/static/12224681f2fd40bf0749423e29cf8d0c/8c3c2/technology-education-information-handover.jpg 1200w","srcWebp":"/static/12224681f2fd40bf0749423e29cf8d0c/e7405/technology-education-information-handover.webp","srcSetWebp":"/static/12224681f2fd40bf0749423e29cf8d0c/4fec1/technology-education-information-handover.webp 300w,\n/static/12224681f2fd40bf0749423e29cf8d0c/483a3/technology-education-information-handover.webp 600w,\n/static/12224681f2fd40bf0749423e29cf8d0c/e7405/technology-education-information-handover.webp 1200w","sizes":"(max-width: 1200px) 100vw, 1200px"}}}}}},"pageContext":{"id":"380b685c-4d58-57b4-b7e0-5d2bdc1eabc5","categories":["Kubernetes","AWS"]}},"staticQueryHashes":["2022990323","639612397"]}