Docker on Amazon Web Services: Build, deploy, and manage your container applications at scale

✍ Scribed by Justin Menga

• Publisher: Packt Publishing
• Year: 2018
• Tongue: English
• Leaves: 812
• Category: Library

✦ Synopsis

Run Docker on AWS and build real-world, secure, and scalable container platforms on cloud

Key Features

• Configure Docker for the ECS environment
• Integrate Docker with different AWS tools
• Implement container networking and deployment at scale

Book Description

Over the last few years, Docker has been the gold standard for building and distributing container applications. Amazon Web Services (AWS) is a leader in public cloud computing, and was the first to offer a managed container platform in the form of the Elastic Container Service (ECS).

Docker on Amazon Web Services starts with the basics of containers, Docker, and AWS, before teaching you how to install Docker on your local machine and establish access to your AWS account. You'll then dig deeper into the ECS, a native container management platform provided by AWS that simplifies management and operation of your Docker clusters and applications for no additional cost. Once you have got to grips with the basics, you'll solve key operational challenges, including secrets management and auto-scaling your infrastructure and applications. You'll explore alternative strategies for deploying and running your Docker applications on AWS, including Fargate and ECS Service Discovery, Elastic Beanstalk, Docker Swarm and Elastic Kubernetes Service (EKS). In addition to this, there will be a strong focus on adopting an Infrastructure as Code (IaC) approach using AWS CloudFormation.

By the end of this book, you'll not only understand how to run Docker on AWS, but also be able to build real-world, secure, and scalable container platforms in the cloud.

What you will learn

• Build, deploy, and operate Docker applications using AWS
• Solve key operational challenges, such as secrets management
• Exploit the powerful capabilities and tight integration of other AWS services
• Design and operate Docker applications running on ECS
• Deploy Docker applications quickly, consistently, and reliably using IaC
• Manage and operate Docker clusters and applications for no additional cost

Who this book is for

Docker on Amazon Web Services is for you if you want to build, deploy, and operate applications using the power of containers, Docker, and Amazon Web Services. Basic understanding of containers and Amazon Web Services or any other cloud provider will be helpful, although no previous experience of working with these is required.

Table of Contents

1. Container and Docker Fundamentals
2. Building Applications Using Docker
3. Getting Started with AWS
4. Introduction to ECS
5. Publishing Docker Images using ECR
6. Building Custom ECS Container Instances
7. Creating ECS Clusters
8. Deploying Applications using ECS
9. Managing Secrets
10. Isolating Network Access
11. Managing ECS Infrastucture Lifecycle
12. ECS Auto Scaling
13. Continuously Delivering ECS Applications
14. Fargate and ECS Service Discovery
15. Elastic Beanstalk
16. Docker Swarm in AWS
17. Elastic Kubernetes Service
18. Assessments

✦ Table of Contents

Cover
Title Page
Copyright and Credits
Dedication
Packt Upsell
Contributors
Table of Contents
Preface
Chapter 1: Container and Docker Fundamentals
Technical requirements
Introduction to containers and Docker
Why containers are revolutionary
Docker architecture
Running Docker in AWS
Setting up a local Docker environment
Setting up a macOS environment
Installing other tools
Setting up a Windows 10 environment
Installing the Windows subsystem for Linux
Installing Docker in the Windows subsystem for Linux
Installing other tools in the Windows subsystem for Linux
Setting up a Linux environment
Installing the sample application
Forking the sample application
Running the sample application locally
Installing application dependencies
Running database migrations
Running the local development web server
Testing the sample application locally
Summary
Questions
Further reading
Chapter 2: Building Applications Using Docker
Technical requirements
Testing and building the application using Docker
Creating a test stage
Installing system and build dependencies
Installing application dependencies
Copying application source and running tests
Configuring the release stage
Installing system dependencies
Creating an application user
Copying and installing application source code and dependencies
Building and running the release image
Testing and building the application using Docker Compose
Adding a database service using Docker Compose
Running database migrations
Generating static web content
Creating acceptance tests
Automating the workflow
Automating the test stage
Automating the release stage
Refining the workflow
Cleaning up the Docker environment
Using dynamic port mapping
Adding a version target
Testing the end-to-end workflow
Summary
Questions
Further reading
Chapter 3: Getting Started with AWS
Technical requirements
Setting up an AWS account
Installing Google Authenticator
Logging in as the root account
Creating IAM users, groups, and roles
Creating IAM roles
Creating an Administrators group
Creating a Users group
Creating an IAM user
Logging in as an IAM user
Enabling MFA for an IAM user
Assuming an IAM role
Creating an EC2 Key Pair
Using the AWS CLI
Installing the AWS CLI
Creating an AWS access key
Configuring the AWS CLI
Configuring the AWS CLI to assume a role
Configuring the AWS CLI to use a named profile
Introduction to AWS CloudFormation
Defining a CloudFormation template
Deploying a CloudFormation stack
Updating a CloudFormation Stack
Deleting a CloudFormation stack
Summary
Questions
Further reading
Chapter 4: Introduction to ECS
Technical requirements
ECS architecture
Creating an ECS cluster
Understanding ECS container instances
Joining an ECS cluster
Granting access to join an ECS cluster
Managing ECS container instances
Connecting to ECS container instances
Inspecting the local Docker environment
Inspecting the ECS agent
Verifying the ECS agent
ECS container instance logs
Creating an ECS task definition
Creating an ECS service
Deploying ECS services
Running ECS tasks
Using the ECS CLI
Deleting the Test Cluster
Summary
Questions
Further information
Chapter 5: Publishing Docker Images Using ECR
Technical requirements
Understanding ECR
Creating ECR repositories
Creating ECR repositories using the AWS Console
Creating ECR repositories using the AWS CLI
Creating ECR repositories using AWS CloudFormation
Logging into ECR
Publishing Docker images to ECR
Publishing Docker images using the Docker CLI
Publishing Docker images using Docker Compose
Automating the publish workflow
Automating login and logout
Automating the publishing of Docker images
Pulling Docker images from ECR
ECS container instance access to ECR from the same account
ECS container instance access to ECR from a different account
Configuring ECR resource policies using the AWS Console
Configuring ECR resource policies using the AWS CLI
Configuring ECR resource policies using AWS CloudFormation
Configuring IAM policies in remote accounts
AWS service access to ECR
Configuring lifecycle policies
Configuring lifecycle policies using the AWS Console
Configuring lifecycle policies using the AWS CLI
Configuring lifecycle policies using AWS CloudFormation
Summary
Questions
Further reading
Chapter 6: Building Custom ECS Container Instances
Technical requirements
Designing a custom Amazon Machine Image
Building a custom AMI using Packer
Installing Packer
Creating a Packer template
Packer template structure
Configuring a builder
Configuring variables
Configuring provisioners
Configuring post-processors
Building a machine image
Generating dynamic session credentials
Automating generation of dynamic session credentials
Building the image
Building custom ECS container instance images using Packer
Defining a custom storage configuration
Adding EBS volumes
Formatting and mounting volumes
Installing additional packages and configuring system settings
Installing additional packages
Configuring system settings
Configuring timezone settings
Modifying default cloud-init behavior
Configuring a cleanup script
Creating a first-run script
Configuring ECS cluster membership
Configuring HTTP proxy support
Configuring the CloudWatch logs agent
Starting required services
Performing required health checks
Testing your custom ECS container instance image
Summary
Questions
Further reading
Chapter 7: Creating ECS Clusters
Technical requirements
Deployment overview
Defining an ECS cluster
Configuring an EC2 Auto Scaling group
Creating an EC2 Auto Scaling group
Configuring CloudFormation Input Parameters
Defining an EC2 Auto Scaling launch configuration
Configuring CloudFormation Init Metadata
Configuring Auto Scaling group creation policies
Configuring EC2 instance profiles
Configuring EC2 security groups
Deploying and testing an ECS cluster
Summary
Questions
Further reading
Chapter 8: Deploying Applications Using ECS
Technical requirements
Creating an application database using RDS
Configuring supporting RDS resources
Deploying RDS resources using CloudFormation
Configuring Application Load Balancers
Application Load Balancer architecture
Configuring an Application Load Balancer
Creating an Application Load Balancer
Configuring Application Load Balancer security groups
Creating a listener
Creating a target group
Deploying an Application Load Balancer using CloudFormation
Creating ECS task definitions
Configuring ECS task definition families
Configuring ECS task definition volumes
Configuring ECS task definition containers
Deploying ECS task definitions using CloudFormation
Deploying ECS services
Deploying an ECS service using CloudFormation
ECS rolling deployments
Executing a rolling deployment
Creating a CloudFormation custom resource
Understanding CloudFormation custom resources
Creating a custom resource Lambda function
Understanding the custom resource function code
Understanding the custom resource Lambda function resources
Creating custom resources
Deploying custom resources
Verifying the application
Summary
Questions
Further reading
Chapter 9: Managing Secrets
Technical requirements
Creating KMS keys
Encrypting and decrypting data using KMS
Creating secrets using the AWS Secrets Manager
Creating secrets using the AWS console
Creating secrets using the AWS CLI
Retrieving secrets using the AWS CLI
Updating secrets using the AWS CLI
Deleting and restoring secrets using the AWS CLI
Injecting secrets at container startup
Creating an entrypoint script
Adding an entrypoint script to a Dockerfile
Provisioning secrets using CloudFormation
Configuring ECS task definitions to use secrets
Exposing secrets to other resources
Creating a Secrets Manager Lambda function
Creating a secrets custom resource
Deploying secrets to AWS
Summary
Questions
Further reading
Chapter 10: Isolating Network Access
Technical requirements
Understanding ECS task networking
Docker bridge networking
ECS task networking
Configuring a NAT gateway
Configuring private subnets and route tables
Configuring NAT gateways
Configuring routing for your private subnets
Configuring ECS task networking
Configuring ECS task definitions for task networking
Configuring ECS services for task networking
Configuring supporting resources for task networking
Deploying and testing ECS task networking
Summary
Questions
Further reading
Chapter 11: Managing ECS Infrastructure Life Cycle
Technical requirements
Understanding ECS life cycle management
EC2 Auto Scaling life cycle hooks
ECS container instance draining
ECS life cycle management solution
Building a new ECS container instance AMI
Configuring EC2 Auto Scaling rolling updates
Creating EC2 Auto Scaling life cycle hooks
Creating a Lambda function for consuming life cycle hooks
Configuring permissions for the life cycle hook Lambda function
Deploying and testing Auto Scaling life cycle hooks
Summary
Questions
Further reading
Chapter 12: ECS Auto Scaling
Technical requirements
Understanding ECS cluster resources
CPU resources
Memory resources
Network resources
Calculating the ECS cluster capacity
Calculating the container capacity
Deciding when to scale out
Calculating the idle host capacity
Idle host capacity example
Implementing an ECS Auto Scaling solution
Configuring CloudWatch events for ECS
Programming the Lambda function that calculates the cluster capacity
Adding IAM permissions for calculating the cluster capacity
Testing cluster-capacity calculations
Publishing custom CloudWatch metrics
Creating CloudWatch alarms for cluster-capacity management
Creating EC2 Auto Scaling policies
Testing ECS cluster-capacity management
Testing scale out
Testing scale in
Configuring the AWS application Auto Scaling service
Configuring CloudWatch alarms
Defining an Auto Scaling target
Creating an Auto Scaling IAM role
Configuring scale-out and scale-in policies
Deploying application Auto Scaling
Summary
Questions
Further reading
Chapter 13: Continuously Delivering ECS Applications
Technical requirements
Introducing CodePipeline and CodeBuild
Creating a custom CodeBuild container
Defining a custom CodeBuild container
Creating a repository for the custom CodeBuild container
Adding CodeBuild support to your application repository
Creating a continuous integration pipeline using CodePipeline
Creating a CodePipeline pipeline using the AWS console
Creating a continuous delivery pipeline using CodePipeline
Publishing version information in your source repository
Adding CodePipeline support to the deployment repository
Creating an IAM role for CloudFormation deployments
Adding a deployment repository to CodePipeline
Adding an output artifact to the build stage
Adding a deployment stage to the pipeline
Continuously delivering to production using CodePipeline
Adding a new environment configuration file to your deployment repository
Adding a create change set action to the pipeline
Adding a manual approval action to the pipeline
Adding a deploy change set action to the pipeline
Deploying to production
Summary
Questions
Further reading
Chapter 14: Fargate and ECS Service Discovery
Technical requirements
When to use Fargate?
Adding support for AWS X-Ray to applications
Creating an X-Ray daemon Docker image
Configuring ECS service discovery resources
Configuring a service discovery namespace
Configuring a service discovery service
Configuring an ECS task definition for Fargate
Configuring IAM roles for Fargate
Configuring an ECS service for Fargate
Deploying and testing the X-Ray daemon
Configuring the todobackend stack for X-Ray support
Testing the X-Ray service
Summary
Questions
Further reading
Chapter 15: Elastic Beanstalk
Technical requirements
Introduction to Elastic Beanstalk
Elastic Beanstalk concepts
Creating an Elastic Beanstalk application
Creating a Dockerrun.aws.json file
Creating an Elastic Beanstalk application using the AWS console
Configuring the EC2 instance profile
Configuring Elastic Beanstalk applications using the CLI
Managing Elastic Beanstalk EC2 instances
Customizing Elastic Beanstalk applications
Resolving Docker volume permissions issues
Configuring database settings
Running database migrations
Summary
Questions
Further reading
Chapter 16: Docker Swarm in AWS
Technical requirements
Docker Swarm introduction
Docker Swarm versus Kubernetes
Installing Docker for AWS
Resources created by the Docker for AWS CloudFormation stack
Accessing the Swarm cluster
Setting up local access to Docker Swarm
Configuring SSH agent forwarding
Configuring SSH tunneling
Deploying applications to Docker Swarm
Docker services
Docker stacks
Deploying the sample application to Docker Swarm
Integrating Docker Swarm with the Elastic Container Registry
Defining a stack
Creating shared storage for hosting static content
Creating a collectstatic service
Creating persistent storage for storing the application database
Relocating an EBS volume
Secrets management using Docker secrets
Configuring applications to consume secrets
Running database migrations
Summary
Questions
Further reading
Chapter 17: Elastic Kubernetes Service
Technical requirements
Introduction to Kubernetes
Kubernetes versus Docker Swarm
Kubernetes architecture
Getting started with Kubernetes
Creating a pod
Creating a deployment
Creating a service
Exposing a service
Adding volumes to your pods
Adding init containers to your pods
Adding a database service
Creating persistent storage
Creating a database service
Creating and consuming secrets
Consuming secrets for the database service
Consuming secrets for the application
Running jobs
Creating an EKS cluster
Installing client components
Creating cluster resources
Configuring kubectl for EKS
Creating worker nodes
Joining worker nodes to your EKS cluster
Deploying the Kubernetes dashboard
Deploying the sample application to EKS
Configuring support for persistent volumes using AWS EBS
Configuring support for AWS Elastic Load Balancers
Deploying the sample application
Creating secrets
Deploying the database service
Deploying the application service
Tearing down down the sample application
Summary
Questions
Further reading
Assessments
Other Books You May Enjoy
Index