Practical GitOps: Infrastructure Management Using Terraform, AWS, and GitHub Actions

✍ Scribed by Rohit Salecha

• Publisher: Apress
• Year: 2022
• Tongue: English
• Leaves: 538
• Edition: 1st ed.
• Category: Library

✦ Synopsis

Infrastructure as Code (IaC) is gaining popularity and developers today are deploying their application environments through IaC tools to the cloud. However, it can become extremely difficult and time-consuming to manage the state of the infrastructure that has been deployed. This book will provide a complete walkthrough of deploying a SpringBoot application on AWS with multiple environments like production, staging and development. Everything is orchestrated through GitHub Actions and executed through Terraform Cloud to monitor changes in your infrastructure and manage its state.
You'll start by reviewing how your infrastructure can be stored in code by spinning up an EC2 server first through the console, then AWS CLI and then using Terraform. You'll then be presented with a practical scenario of setting up a simple EC2 server in a multi-environment (production, staging and development) using GitHub Actions and Terraform Cloud. In the advanced section that follows, this simple EC2 server is expanded into an application that is deployed on an AWS EKS (Elastic Kubernetes Service) using AWS RDS (Relational Database Service) exposed through an AWS ALB (Application Load Balancer) protected using AWS ACM (AWS Certificate Manager), and accessible by setting the AWS Route53.
The book then builds up on this infrastructure and demonstrates how it can be deployed in a multi-environment scenario by implementing accounts through AWS organizations. You'll see how to put in restrictions through Service Control Policies, how to protect secrets using AWS Secrets Manager, and how to work with least privileges using IRSA (IAM Roles for Service Accounts). Finally, you'll make the infrastructure more observable using Graphana, Prometheus, and AWS OpenSearch, run security tools, host Route53 zones dynamically based on environments, and implement CloudWatch Alarms for various use cases.

✦ Table of Contents

Table of Contents
About the Author
About the Technical Reviewers
Preface
Chapter 1: What Is GitOps?
1.1 The Era of DevOps
1.1.1 What Problems Is DevOps Addressing?
1.2 Introduction to DevOps
1.2.1 Continuous Integration
1.2.2 Continuous Delivery/Deployment
1.2.3 Continuous Monitoring
1.3 Infrastructure As Code (IaC)
1.3.1 Evolution of Server Infrastructure
1.3.2 Tools of Trade – IaC
1.3.3 Mutable vs. Immutable Infrastructure
1.3.4 State in IaC
1.4 Problems with Continuous Delivery
1.5 Introduction to GitOps
1.6 Conclusion
Chapter 2: Introduction to AWS
2.1 Prerequisites
2.2 Introduction to AWS
2.2.1 Create an AWS Account
2.2.2 Log in into the AWS Account
2.3 Creating an EC2 Machine – GUI
2.3.1 Default Region Selection
2.3.2 Instance Type Selection
2.3.3 Instance Configuration
2.3.4 Instance Storage
2.3.5 Tags
2.3.6 Security Groups
2.3.7 SSH Key-Pair Generation
2.3.8 Launching EC2 Instance
2.3.9 Terminating EC2 Instance
2.4 Creating an EC2 Machine – CLI
2.4.1 Configuring CLI Environment
2.4.2 Fetching Security Group ID
2.4.3 Fetching Subnet ID
2.4.4 Launching EC2 Instance
2.4.5 Accessing EC2 Instance
2.4.6 Terminating EC2 Instance
2.5 Clean-Up
2.6 Conclusion
Chapter 3: Introduction to Terraform
3.1 Prerequisites
3.2 Introduction to Terraform
3.3 Getting Started with Terraform
3.4 Iteration #1
3.4.1 Terraform Working Directory
3.4.2 Terraform init
3.4.3 Terraform Plan
3.4.4 Terraform Apply
3.4.5 Terraform State File
3.4.6 Terraform Destroy
3.5 Iteration #2
3.5.1 Terraform Variables
3.5.2 Terraform Data Source
3.5.3 Terraform Resource
3.5.4 Terraform Output
3.6 Iteration #3
3.6.1 Terraform Modules
3.7 Selective Destroy
3.7.1 Terraform Destroy Protection
3.8 Terraform Drift
3.9 Clean-Up
3.10 Terraform Commands Reference
3.11 Conclusion
Chapter 4: Introduction to Terraform Cloud and Workspaces
4.1 Prerequisites
4.2 Terraform State Management
4.3 Introduction to Terraform Cloud
4.4 Terraform Cloud – Getting Started
4.4.1 Creating a Workspace
4.4.2 Configure Workspace
4.4.3 Workspace Variables
4.4.4 Terraform Login
4.4.5 Running Terraform in Terraform Cloud
4.4.6 Terraform Cloud Run and States
4.5 Multi-environment with DRY
4.5.1 Adding Production Environment
4.5.2 TF Variables Overriding
4.6 Clean-Up
4.6.1 Destroying Terraform Cloud Workspaces
4.7 Terraform Cloud Security Best Practices
4.8 Conclusion
Chapter 5: Terraform Automation with Git
5.1 Prerequisites
5.2 Terraform Automation with GitHub
5.2.1 Setting Up the GitHub Repository
5.2.2 Connecting GitHub with Terraform Cloud
5.2.3 Creating VCS-Driven Workspace
5.2.4 Configuring Environment Variables
5.2.5 Executing Plan in Cloud
5.2.6 Triggering the VCS
5.2.7 Destroying in Terraform Cloud
5.2.8 Drawbacks of Terraform Cloud with VCS
5.3 Introduction to GitHub Actions
5.3.1 CI/CD General Workflow
5.3.2 Introduction to GitHub Actions
5.3.3 Sample GitHub Actions YAML
5.3.4 Docker Build Automation with GitHub Actions
5.3.5 Triggering GitHub Actions
5.4 Terraform Automation with GitHub Actions
5.4.1 Code Walkthrough
5.4.2 Terraform Login Token
5.4.3 Creating an API-Driven Terraform Workspace
5.4.4 Executing API-Driven Workflow
5.5 Clean-Up
5.6 Conclusion
Chapter 6: Practical GitOps
6.1 Prerequisites
6.2 Staging Environment Workflow
6.2.1 High-Level Plan
6.2.2 Code Walkthrough
6.2.3 Set Up Terraform Staging Workspace
6.2.4 Set Up a GitHub Repository
6.2.5 Executing Staging Workflow
6.3 Prod Environment Workflow
6.3.1 SSH Key-Pair Generation
6.4 Complete Workflow
6.4.1 New Feature Request
6.4.2 Developer Testing
6.4.3 Create Pull Request – Dev
6.4.4 Create Pull Request – Prod
6.5 Clean-Up
6.6 Conclusion
Chapter 7: Spring Boot App on AWS EKS
7.1 Prerequisites
7.1.1 GitHub Repository
Terraform Cloud Workspace
7.1.2 DNS Configuration
Top-Level Domain
Subdomain
Delete Route53 Records
7.2 Solution Architecture
7.3 Networking
7.3.1 Network Module
7.3.2 Network Tags
7.3.3 Security Groups
7.4 Database
7.4.1 Database Module
7.4.2 Database Configuration and Secrets
7.5 Kubernetes
7.5.1 EKS
7.5.2 EKS Module
7.5.3 EKS Worker Nodes
7.5.4 EKS Security Groups
7.5.5 EKS Ingress Controller
7.6 DNS
7.6.1 Route53 Record Creation
7.6.2 Multi-environment DNS
7.6.3 ACM
7.7 Application
7.7.1 Namespace
7.7.2 Deployment
7.7.3 NodePort Service
7.7.4 Ingress
7.7.5 Kubernetes Provider
7.7.6 Variables
7.7.7 Default Environment
7.8 Execution
7.8.1 Setting Up the EKS Cluster
7.8.2 Accessing the EKS Cluster
7.9 Clean-Up
7.10 Conclusion
Chapter 8: Authentication and  Authorization
8.1 Prerequisites
8.1.1 Code Update
8.2 AWS Organizations
8.2.1 Root Account
8.2.2 Member Accounts and OUs
8.2.3 AWS Provider
8.3 AWS IAM
8.3.1 IAM Users
8.3.2 IAM Groups
8.3.3 IAM Roles
8.4 AWS Route53
8.5 Executing Global
8.5.1 Generate GNU PG Keys
8.5.2 Configure Variables
8.5.3 Terraform Cloud Workspace
8.5.4 Terraform Apply
8.5.5 Terraform Output
Account IDs
Links
Name Servers
Users
8.5.6 Possible Issues
Email Already Exists
AWS Subscription Not Found
8.6 EKS Authz
8.6.1 EKS Client Authentication
8.6.2 EKS Node Authentication
8.6.3 Provider Configuration
8.7 Executing Infra
8.7.1 AWS Profiles
8.7.2 Development
8.7.3 Staging
8.7.4 Prod
8.8 EKS Access Control
8.9 Clean-Up
8.9.1 Dev
8.9.2 Staging/Prod
8.10 Conclusion
Chapter 9: Security and Secrets Management
9.1 Code Update
9.1.1 Committing only Global
9.1.2 Committing Infra
9.2 Enhancing EKS Security
9.2.1 Encrypting K8s Secrets
9.2.2 Encrypting EKS EBS
9.2.3 Enhancing EC2 Metadata Security
9.3 Enhancing AWS ALB Security
9.4 Restricting RDS Exposure
9.5 Secrets Exposure
9.5.1 Terraform State
9.5.2 Kubernetes Deployment Descriptions
9.6 AWS Secrets Manager
9.6.1 Secrets Management Design
Current Solution
New Solution
9.6.2 Secrets Store CSI Driver
9.7 IRSA
9.7.1 Background
9.7.2 Internal Working
9.7.3 IAM Permissions Policy
9.8 Checkov Scanning
9.8.1 Checkov in Action
9.9 Service Control Policies
9.9.1 SCP in Action
9.10 Clean-Up
9.11 Conclusion
Chapter 10: Observability
10.1 Code Update
10.1.1 Committing Only Global
10.1.2 Committing Infra
10.2 Executing Infra
10.3 Organization Trail
10.4 CloudWatch Alarm
10.4.1 Configuring AWS SNS
10.4.2 Configuring CloudWatch Alarm
10.4.3 Root Login Alarm in Action
10.5 Master Account Login
10.6 ALB Access Logs
10.7 Logging
10.7.1 Exploring OpenSearch
10.8 Monitoring
10.8.1 Exploring Grafana
10.9 Karpenter Autoscaler
10.9.1 Karpenter in Action
10.10 Upgrading Kubernetes
10.11 Clean-Up
10.12 Conclusion
Annexure A: Manually Delete Resources
Annexure B: Terraform Cloud Destroy Problem
Annexure C: Code Compatibility on OS
C.1 Mac M1
C.2 Windows Native (cmd.exe)
C.3 WSL2 (Ubuntu)
Index