Go for DevOps: Learn how to use the Go language to automate servers

✍ Scribed by John Doak; David Justice

• Publisher: Packt Publishing
• Year: 2022
• Tongue: English
• Leaves: 878
• Category: Library

✦ Synopsis

Achieve reliable release automation and get zero troublesome notifications on your release day

Key Features
Develop the skills to build command-line tools to control thousands of nodes
Use Go to create Terraform providers and GitHub actions and extend Kubernetes
Gain the knowledge to build DevOps workflows that are understandable, scalable, and safe

Book Description
Go is the go-to language for DevOps libraries and services, and without it, achieving fast and safe automation is a challenge. With the help of Go for DevOps, you'll learn how to deliver services with ease and safety, becoming a better DevOps engineer in the process.

Some of the key things this book will teach you are how to write Go software to automate configuration management, update remote machines, author custom automation in GitHub Actions, and interact with Kubernetes. As you advance through the chapters, you'll explore how to automate the cloud using software development kits (SDKs), extend HashiCorp's Terraform and Packer using Go, develop your own DevOps services with gRPC and REST, design system agents, and build robust workflow systems.

By the end of this Go for DevOps book, you'll understand how to apply development principles to automate operations and provide operational insights using Go, which will allow you to react quickly to resolve system failures before your customers realize something has gone wrong.

What you will learn
Understand the basic structure of the Go language to begin your DevOps journey
Interact with filesystems to read or stream data
Communicate with remote services via REST and gRPC
Explore writing tools that can be used in the DevOps environment
Develop command-line operational software in Go
Work with popular frameworks to deploy production software
Create GitHub actions that streamline your CI/CD process
Write a ChatOps application with Slack to simplify production visibility

Who this book is for
This book is for Ops and DevOps engineers who would like to use Go to develop their own DevOps tooling or integrate custom features with DevOps tools such as Kubernetes, GitHub Actions, HashiCorp Packer, and Terraform. Experience with some type of programming language, but not necessarily Go, is necessary to get started with this book.

✦ Table of Contents

Go for DevOps
Contributors
About the authors
About the reviewers
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share your thoughts
Section 1: Getting Up and Running with Go
Chapter 1: Go Language Basics
Technical requirements
Using the Go Playground
Utilizing Go packages
Declaring a package
Importing a package
Using a package
Package name conflicts
Packages must be used
A Go Hello World
Using Go's variable types
Go's types
Declaring variables
Variable scopes and shadowing
Function/statement variable must be used
Looping in Go
C style
Removing the init statement
Remove the post statement too and you have a while loop
Creating an infinite loop
Using conditionals
if statements
else
Learning about functions
Returning multiple values and named results
Variadic arguments
Anonymous functions
Defining public and private
Using arrays and slices
Arrays
Slices
Extracting all values
Understanding maps
Declaring a map
Accessing values
Adding new values
Extracting all values
Understanding Go pointers
Memory addresses
Function arguments are copies
Pointers to the rescue
Getting to know about structs
Declaring a struct
Declaring a custom type
Custom struct types
Adding methods to a type
Changing a field's value
Changing a field's value in a method
Constructors
Comprehending Go interfaces
Defining an interface type
Important things about interfaces
The blank interface – Go's universal value
Type assertion
Summary
Chapter 2: Go Language Essentials
Handling errors in Go
Creating an error
Using an error
Creating named errors
Custom errors
Wrapping errors
Utilizing Go constants
Declaring a constant
Enumeration via constants
Printing enumerators
Using defer, panic, and recover
defer
panic
recover
Utilizing goroutines for concurrency
Starting a goroutine
Synchronization
WaitGroups
Channels
Sending/receiving
select statements
Channels as an event signal
Mutexes
RWMutex
Understanding Go's Context type
Using a Context to signal a timeout
Honoring a context when receiving
Context in the standard library
Context to pass values
Best practices
Utilizing Go's testing framework
Creating a basic test file
Creating a simple test
Table Driven Tests (TDT)
Creating fakes with interfaces
Third-party testing packages
Generics – the new kid on the block
Type parameters
Using type constraints
We could do better with constraints
Current built-in constraints
Type constraints with methods
Adding type parameters to struct types
Specifying the type when calling a generic function
Gotchas to watch for
When to use generics
Summary
Chapter 3: Setting Up Your Environment
Technical requirements
Installing Go on your machine
macOS installation using the package installer
macOS installation via Homebrew
Windows installation using MSI
Linux
Other platforms
A note on Go compiler version compatibility
Building code locally
Creating a module directory and go.mod file
Updating a module when adding dependencies
Adding a hello world
Running our first program
Summary
Chapter 4: Filesystem Interactions
All I/O in Go are files
I/O interfaces
Reading and writing to files
Reading local files
Writing local files
Reading remote files
Streaming file content
Stdin/Stdout/Stderr are just files
Reading data out of a stream
Writing data into a stream
OS-agnostic pathing
What OS/platform am I running?
Using filepath
Relative and absolute pathing
OS-agnostic filesystems
io.fs filesystems
embed
Walking our filesystem
The io/fs future
Summary
Chapter 5: Using Common Data Formats
Technical requirements
CSV files
Basic value separation using the strings package
Using the encoding/csv package
Using excelize when dealing with Excel
Popular encoding formats
The Go field tags
JSON
YAML encoding
Summary
Chapter 6: Interacting with Remote Data Sources
Technical requirements
Accessing SQL databases
Connecting to a Postgres database
Querying a Postgres database
Null values
Writing data to Postgres
Transactions
Postgres-specific types
Other options
Storage abstractions
Case study – data migration of an orchestration system – Google
Developing REST services and clients
REST for RPCs
Developing gRPC services and clients
Protocol buffers
Stating the prerequisites
Generating your packages
Writing a gRPC client
Writing a gRPC server
Creating a server binary
Creating a client binary
Company-standard RPC clients and servers
Summary
Chapter 7: Writing Command-Line Tooling
Technical requirements
Implementing application I/O
The flag package
Custom flags
Basic flag error handling
Shorthand flags
Accessing non-flag arguments
Retrieving input from STDIN
Using Cobra for advanced CLI applications
Code organization
The optional Cobra generator
The command package
Handling OS signals
Capturing an OS signal
Using Context to cancel
Summary
Chapter 8: Automating Command-Line Tasks
Technical requirements
Using os/exec to automate local changes
Determining the availability of essential tools
Using SSH in Go to automate remote changes
Connecting to another system
Designing safe, concurrent change automations
Components of a change
Writing a concurrent job
Case study – Network rollouts
Writing a system agent
Designing a system agent
Implementing Install
Implementing SystemPerf
Summary
Section 2: Instrumenting, Observing, and Responding
Chapter 9: Observability with OpenTelemetry
Technical requirements
An introduction to OpenTelemetry
Reference architecture for OpenTelemetry
OpenTelemetry components
Logging with context
Our first log statement
Structured and leveled logs with Zap
Ingesting, transforming, and exporting logs using OpenTelemetry
Instrumenting for distributed tracing
The life cycle of a distributed trace
Correlating traces and logs
Adding log entries to spans
Instrumenting for metrics
The life cycle of a metric
Client/server metrics with OpenTelemetry
Alerting on metrics abnormalities
Adding and configuring Alertmanager
Summary
Chapter 10: Automating Workflows with GitHub Actions
Technical requirements
Understanding the basics of GitHub Actions
Exploring the components of a GitHub Action
How to build and trigger your first GitHub Action
Building a continuous integration workflow
Introducing the tweeter command-line tool
Goals of the tweeter continuous integration workflow
Continuous integration workflow for tweeter
Building a release workflow
GitHub releases
Release automation for tweeter
Creating a custom GitHub Action using Go
Basics of custom actions
Goals for the tweeter custom GitHub Action
Creating the tweeter action
Publishing a custom Go GitHub Action
The basics of publishing actions
Goals for publishing the tweeter custom action
Managing action semantic versioning
Publishing the tweeter action to the GitHub Marketplace
Summary
Chapter 11: Using ChatOps to Increase Efficiency
Technical requirements
Environment architecture
Using an Ops service
Building a basic chatbot
Creating event handlers
Case Study – Regexes versus Lexer and Parser
Creating our Slack application
Running the applications
Summary
Section 3: Cloud ready Go
Chapter 12: Creating Immutable Infrastructure Using Packer
Technical requirements
Building an Amazon Machine Image
Setting up an AWS source
Defining a build block and adding some provisioners
Executing a Packer build
Validating images with Goss
Creating a spec file
Adding a Packer provisioner
Customizing Packer with plugins
Writing your own plugin
Releasing a plugin
Using our plugin in a build
Debugging a Packer plugin
Summary
Chapter 13: Infrastructure as Code with Terraform
Technical requirements
An introduction to IaC
Understanding the basics of Terraform
Initializing and applying infrastructure specs using Terraform
Understanding the basics of Terraform providers
Defining and provisioning cloud resources
Building a pet store Terraform provider
Resources for building custom providers
The pet store provider
Publishing custom providers
Summary
Chapter 14: Deploying and Building Applications in Kubernetes
Technical requirements
Interacting with the Kubernetes API
Creating a KinD cluster
Using kubectl to interact with the API
Deploying a load-balanced HTTP application using Go
It all starts with main
Creating a ClientSet
Creating a namespace
Deploying the application into the namespace
Creating the NGINX deployment
Waiting for ready replicas to match desired replicas
Creating a Service to load-balance
Creating an ingress to expose our application on a local host port
Streaming pod logs for the NGINX application
Extending Kubernetes with custom resources and operators
Custom Resource Definitions
Controllers
Standing on the shoulders of giants
Building a pet store operator
Initializing the new operator
Summary
Chapter 15: Programming the Cloud
Technical requirements
What is the cloud?
Learning the basics of the Azure APIs
A background on cloud APIs and SDKs
Microsoft Azure identity, RBAC, and resource hierarchy
Creating an Azure account and accessing the API
Building infrastructure using Azure Resource Manager
Azure SDK for Go
Setting up your local environment
Building an Azure virtual machine
Using provisioned Azure infrastructure
Building an Azure Storage account
Summary
Chapter 16: Designing for Chaos
Technical requirements
Using overload prevention mechanisms
Case study – AWS client requests overwhelm the network
Using circuit breakers
Using backoff implementations
Combining circuit breakers with backoff
Using rate limiters to prevent runaway workflows
Case study – Google satellite disk erase
Channel-based rate limiter
Token-bucket rate limiter
Building workflows that are repeatable and never lost
Building idempotent workflows
Using three-way handshakes to prevent workflow loss
Using policies to restrict tools
Defining a gRPC workflow service
Creating a policy engine
Writing a policy
Cautions on policy engines
Building systems with an emergency stop
Understanding emergency stops
Building an emergency-stop package
Using the emergency-stop package
Case study – Google's network backbone emergency stop
Summary
Why subscribe?
Other Books You May Enjoy
Packt is searching for authors like you
Share your thoughts