Deploying NestJS Application on ECS Fargate using AWS Copilot
AWS Fargate
AWS Fargate is a serverless compute engine that enables developers to run containers without the need to manage the underlying instances. With Fargate, developers can focus on building and deploying their applications without worrying about the infrastructure. Fargate automatically provisions and scales the infrastructure required to run the containers, and it also manages tasks such as patching and updates.
AWS Copilot
AWS Copilot is a command-line interface tool that helps developers to build, release, and operate production-ready containerized applications on Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS). With AWS Copilot, developers can easily deploy their containerized applications on Amazon ECS Fargate with just a few commands.
AWS Copilot is a powerful command-line interface (CLI) tool designed to make it easier for developers to build, deploy, and operate containerized applications on AWS. It provides a streamlined and integrated solution for developers who want to focus on building applications rather than managing infrastructure.
One of the key benefits of AWS Copilot is that it simplifies the deployment process by automating the creation of AWS resources and providing a simple CLI interface for developers to create, configure, and deploy their applications. This means that developers can spend less time on infrastructure management and more time on building and improving their applications.
In addition to simplifying deployment, AWS Copilot also helps manage containerized applications by handling tasks such as building and pushing container images to Amazon Elastic Container Registry (ECR), creating and managing Amazon Elastic Container Service (ECS) tasks and services, and enabling application scaling based on load and user demand.
AWS Copilot also integrates with popular CI/CD tools such as AWS CodePipeline and AWS CodeBuild, providing end-to-end CI/CD pipelines for containerized applications. This makes it easy for developers to set up automated builds and deployments, which can save time and reduce the risk of errors.
Let's get started!
Before we dive in, let's make sure you have the right setup. If you're already familiar with setting up NestJS with Docker and Mongoose, you're good to go! If not, don't worry - I've got you covered. Take a look at this previous guide to get up to speed. Once you're ready, we can start exploring together.
and here is the repository.
https://github.com/marksabelita/nestjs-docker-mongodb-mongoose
Once we are done cloning the project we need to set up our aws locally.
Setting up AWS Account and AWS Copilot
Before we begin, do you already have an AWS account with the appropriate permissions to create resources like Amazon Elastic Container Registry, Amazon Elastic Container Service, and AWS Identity and Access Management policies? If not, you'll need to create an account and set up the necessary permissions before proceeding. https://aws.amazon.com/
Install the AWS Command Line Interface (CLI) if you haven't already. You can download and install the AWS CLI from the official AWS documentation: docs.aws.amazon.com/cli/latest/userguide/in..
Once you have the AWS CLI installed, you can install Copilot. There are different ways to install Copilot depending on your operating system. You can find detailed instructions for each operating system in the official AWS documentation: aws.github.io/copilot-cli/docs/getting-star..
After the installation is complete, verify that Copilot is installed by running the following command in your terminal or command prompt:
copilot --version
on this writing the current version of copilot that I am using is v1.25.0
MacBook-Pro:project-name mark$ copilot --version
copilot version: v1.25.0
MacBook-Pro:project-name mark$
- Next, you will need to configure your AWS CLI with your AWS account credentials. You can do this by running the following command and following the prompts:
aws configure
Once AWS CLI and AWS Copilot is properly configured on our local machine we can now start deploying our
Deploying project to AWS Fargate.
Head back to our project and start initializing copilot to our project! Let’s get started!
Initialize copilot in our project by running.
Note: Please note that if you want to attach a domain to your project you need to pass —domain parameter “copilot init —domain {domain_name}” if you failed to initialize your domain on this project and you want to attach your domain after deploying your app, you need to delete your app again and re-initialize it. to know more details about deploying your application with domain check this link aws.github.io/copilot-cli/docs/developing/d..
$ copilot init
Application name: project-app
Which workload type best represents your architecture? [Use arrows to move, type to filter, ? for more help]
Request-Driven Web Service (App Runner)
> Load Balanced Web Service (Internet to ECS on Fargate)
Backend Service (ECS on Fargate)
Worker Service (Events to SQS to ECS on Fargate)
Scheduled Job (Scheduled event to State Machine to Fargate)
Workload type: Load Balanced Web Service
What do you want to name this service? [? for help] project-service
Application name: project-app
Workload type: Load Balanced Web Service
Service name: project-service
Once this is done it will prompt which Dockerfile you want to use for your project, by default it will look into the Dockerfile inside your root folder, if you wish to change your Dockerfile location you can do so later once copilot project manifest.yml file is generated.
Which Dockerfile would you like to use for project-service? [Use arrows to move, type to filter, ? for more help]
> ./Dockerfile
Enter custom path for your Dockerfile
Use an existing image instead
Ok great, we'll set up a Load Balanced Web Service named project-service in application project-app listening on port 3000.
Once it is done it will set up the necessary infrastructure roles for our project.
- Creating the infrastructure for stack project-app-infrastructure-roles [create complete] [47.5s]
- A StackSet admin role assumed by CloudFormation to manage regional stacks [create complete] [21.7s]
- An IAM role assumed by the admin role to create ECR repositories, KMS keys, and S3 buckets [create complete] [19.5s]
✔ The directory copilot will hold service manifests for application project-app.
and here is the created manifest.yml file for our project, you can also check it in your copilot/project-service/manifest.yml.
# The manifest for the "project-service" service.
# Read the full specification for the "Load Balanced Web Service" type at:
# <https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/>
# Your service name will be used in naming your resources like log groups, ECS services, etc.
name: project-service
type: Load Balanced Web Service
# Distribute traffic to your service.
http:
# Requests to this path will be forwarded to your service.
# To match all requests you can use the "/" path.
path: '/'
# You can specify a custom health check path. The default is "/".
# healthcheck: '/'
# Configuration for your containers and service.
image:
# Docker build arguments. For additional overrides: <https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/#image-build>
build: Dockerfile
# Port exposed through your container to route traffic to it.
port: 3000
cpu: 256 # Number of CPU units for the task.
memory: 512 # Amount of memory in MiB used by the task.
platform: linux/x86_64 # See <https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/#platform>
count: 1 # Number of tasks that should be running in your service.
exec: true # Enable running commands in your container.
network:
connect: true # Enable Service Connect for intra-environment traffic between services.
# storage:
# readonly_fs: true # Limit to read-only access to mounted root filesystems.
# Optional fields for more advanced use-cases.
#
#variables: # Pass environment variables as key value pairs.
# LOG_LEVEL: info
#secrets: # Pass secrets from AWS Systems Manager (SSM) Parameter Store.
# GITHUB_TOKEN: GITHUB_TOKEN # The key is the name of the environment variable, the value is the name of the SSM parameter.
# You can override any of the values defined above by environment.
#environments:
# test:
# count: 2 # Number of tasks to run for the "test" environment.
# deployment: # The deployment strategy for the "test" environment.
# rolling: 'recreate' # Stops existing tasks before new ones are started for faster deployments.
Once setup is done it will prompt us to deploy our project to test environment, for now let’s deploy test environment, but we will still create another environment later.
Once setup is done it will prompt us to deploy our project to test environment, for now let’s deploy test environment, but we will still create another environment later.
All right, you're all set for local development.
Would you like to deploy a test environment? [? for help] (y/N) y
Once we deploy our test application it will automatically provision full infrastructure for our project, also since we deploy test environment it will generate a new manifest file inside of copilot/environments/test/manifest.yml
# The manifest for the "test" environment.
# Read the full specification for the "Environment" type at:
# https://aws.github.io/copilot-cli/docs/manifest/environment/
# Your environment name will be used in naming your resources like VPC, cluster, etc.
name: test
type: Environment
# Import your own VPC and subnets or configure how they should be created.
# network:
# vpc:
# id:
# Configure the load balancers in your environment, once created.
# http:
# public:
# private:
# Configure observability for your environment resources.
observability:
container_insights: false
[+] Building 46.5s (14/14) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 74B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/node:18-alpine 1.0s
=> [internal] load build context 1.4s
=> => transferring context: 3.42MB 1.3s
=> [development 1/6] FROM docker.io/library/node:18-alpine@sha256:f8a51c36b0be7434bbf867d4a08decf0100e656203d893b9b0f8b1fe9e40daea 0.0s
=> CACHED [development 2/6] WORKDIR /usr/src/app 0.0s
=> CACHED [development 3/6] COPY package*.json ./ 0.0s
=> CACHED [production 4/6] RUN npm install --only=production 0.0s
=> CACHED [development 4/6] RUN npm install 0.0s
=> [development 5/6] COPY . . 6.3s
=> [production 5/6] COPY . . 6.3s
=> [development 6/6] RUN npm run build 33.9s
=> [production 6/6] COPY --from=development /usr/src/app/dist ./dist 0.0s
=> exporting to image 2.4s
=> => exporting layers 2.4s
=> => writing image sha256:fd0018dc51d10a432acc4cda65edc184bbcf8a765eff6907c68feb9782a9e723 0.0s
=> => naming to 123.dkr.ecr.us-east-1.amazonaws.com/project-app/project-service 0.0s 5.4s
The push refers to repository [123.dkr.ecr.us-east-1.amazonaws.com/project-app/project-service]
a2b6e0ef1a90: Pushed
caeb212b5ca6: Pushed
efb1a8956c32: Pushed
6a806046e50d: Pushed
abcdf971456a: Pushed
dc923cea9549: Pushed
d26cfa2c5d93: Pushed
ec7aaa5c3b9b: Pushed
7cd52847ad77: Pushed
latest: digest: sha256:165b92f1080b2763dbac2db6b912026012b8297b4120d37446574f9d13fc94a5 size: 2207
Once the compiling is done it will upload our container to ECR, to confirm we can check our AWS ECR.
Once this is done it will continue setting up our infrastructure automatically and our ECS.
✔ Proposing infrastructure changes for stack project-app-test-project-service
- Creating the infrastructure for stack project-app-test-project-service [create in progress] [288.1s]
- Service discovery for your services to communicate within the VPC [create complete] [0.0s]
- Update your environment's shared resources [update complete] [159.1s]
- A security group for your load balancer allowing HTTP traffic [create complete] [2.4s]
- An Application Load Balancer to distribute public traffic to your services [create complete] [123.4s]
- A load balancer listener to route HTTP traffic [create complete] [4.6s]
- An IAM role to update your environment stack [create complete] [20.0s]
- An IAM Role for the Fargate agent to make AWS API calls on your behalf [create complete] [18.5s]
- A HTTP listener rule for forwarding HTTP traffic [create complete] [2.7s]
- A custom resource assigning priority for HTTP listener rules [create complete] [0.0s]
- A CloudWatch log group to hold your service logs [create complete] [0.0s]
- An IAM Role to describe load balancer rules for assigning a priority [create complete] [18.5s]
- An ECS service to run and maintain your tasks in the environment cluster [create in progress] [44.3s]
Deployments
Revision Rollout Desired Running Failed Pending
PRIMARY 1 [in progress] 1 0 0 1
- A target group to connect the load balancer to your service [create complete] [16.1s]
- An ECS task definition to group your containers and run them on ECS [create complete] [3.5s]
- An IAM role to control permissions for the containers in your tasks [create complete] [18.5s]
- Updating the infrastructure for stack project-app-test-project-service [update complete] [189.7s]
- An ECS service to run and maintain your tasks in the environment cluster [update complete] [172.3s]
Deployments
Revision Rollout Desired Running Failed Pending
PRIMARY 2 [completed] 1 1 0 0
- An ECS task definition to group your containers and run them on ECS [delete complete] [3.6s]
✔ Deployed service project-service.
Recommended follow-up action:
- You can access your service at http://proje-Publi-16PMDK3FQCRDG-229468250.us-east-1.elb.amazonaws.com over the internet.
Once deployment is done it will give us the service URL, for this deployment the elastic load balancer link is proje-Publi-16PMDK3FQCRDG-229468250.us-east.. over the internet.
Let's test it on our browser!
Hooray, our application is now up and running on ECS Fargate!
Let's create our environments!
As we dive into our project, we're going to need multiple environments to ensure we're developing and testing effectively. Luckily, we can automate the creation of these environments using copilot! With copilot, we can quickly and easily set up multiple environments, freeing up more time for us to focus on developing and refining our project. to do so:
$ copilot env init --name staging
Credential source: Enter temporary credentials
AWS Access Key ID: ****************NWP5
AWS Secret Access Key: ****************49fX
AWS Session Token:
Would you like to use the default configuration for a new environment?
- A new VPC with 2 AZs, 2 public subnets and 2 private subnets
- A new ECS Cluster
- New IAM Roles to manage services and jobs in your environment
[Use arrows to move, type to filter]
> Yes, use default.
Yes, but I'd like configure the default resources (CIDR ranges, AZs).
No, I'd like to import existing resources (VPC, subnets).
Default environment configuration? Yes, use default.
✔ Wrote the manifest for environment staging at copilot/environments/staging/manifest.yml
- Update regional resources with stack set "project-app-infrastructure" [succeeded] [7.0s]
✔ Wrote the manifest for environment staging at copilot/environments/staging/manifest.yml
- Update regional resources with stack set "project-app-infrastructure" [succeeded] [7.0s]
✔ Proposing infrastructure changes for the project-app-staging environment.
- Creating the infrastructure for the project-app-staging environment. [create complete] [49.2s]
- An IAM Role for AWS CloudFormation to manage resources [create complete] [19.8s]
- An IAM Role to describe resources in your environment [create complete] [20.1s]
To deploy our infrastructure we need to run.
$copilot env deploy --name staging
Once infrastructure setup is done we can now start deploying our project
$copilot deploy
Only found one workload, defaulting to: project-service
Select an environment [Use arrows to move, type to filter]
> staging
test
To delete your application run
$ copilot app delete
Sure? Yes
Wow! With just a few simple steps, we've successfully set up multiple environments for our project and deployed them to ECS Fargate - how cool is that?! And the best part? Copilot makes it so easy to manage all of these environments, with the ability to select and deploy them with just a few clicks. Plus, it even provides us with URLs for each deployed environment!
I hope this guide has been helpful in showing you just how simple and efficient it can be to manage multiple environments using copilot. With this powerful tool in our arsenal, we can confidently tackle any project development challenge that comes our way. Thank you for joining me on this exciting journey!
If you are interested in learning how to deploy ECS Fargate with AWS Copilot using Github Actions stay tuned for the next blog post!