System Programming Essentials with Go: System calls, networking, efficiency, and security practices

✍ Scribed by Alex Rios

• Publisher: Packt Publishing Pvt. Ltd.
• Year: 2024
• Tongue: English
• Leaves: 408
• Edition: 1
• Category: Library

✦ Synopsis

Go beyond the web, learn system programming with Go, and build efficient, secure applications

Key Features
Get to grips with system programming concepts in Go with application examples
Gain expert guidance on essential topics like file operations, process management, and network programming
Learn how to develop modern, functional applications from scratch
Purchase of the print or Kindle book includes a free PDF eBook
Book Description
Alex Rios, a seasoned Go developer and active community builder, shares his 15 years of expertise in designing large-scale systems through this book. It masterfully cuts through complexity, enabling you to build efficient and secure applications with Go's streamlined syntax and powerful concurrency features.

In this book, you’ll learn how Go, unlike traditional system programming languages (C/C++), lets you focus on the problem by prioritizing readability and elevating developer experience with features like automatic garbage collection and built-in concurrency primitives, which remove the burden of low-level memory management and intricate synchronization.

Through hands-on projects, you'll master core concepts like file I/O, process management, and inter-process communication to automate tasks and interact with your system efficiently. You'll delve into network programming in Go, equipping yourself with the skills to build robust, distributed applications. This book goes beyond the basics by exploring modern practices like logging and tracing for comprehensive application monitoring, and advance to distributed system design using Go to prepare you to tackle complex architectures.

By the end of this book, you'll emerge as a confident Go system programmer, ready to craft high-performance, secure applications for the modern world.

What you will learn
Understand the fundamentals of system programming using Go
Grasp the concepts of goroutines, channels, data races, and managing concurrency in Go
Manage file operations and inter-process communication (IPC)
Handle USB drives and Bluetooth devices and monitor peripheral events for hardware automation
Familiarize yourself with the basics of network programming and its application in Go
Implement logging, tracing, and other telemetry practices
Construct distributed cache and approach distributed systems using Go

Who this book is for
This book is for software engineers looking to expand their understanding of system programming concepts. Professionals with a coding foundation seeking profound knowledge of system-level operations will also greatly benefit. Additionally, individuals interested in advancing their system programming skills, whether experienced developers or those transitioning to the field, will find this book indispensable.

✦ Table of Contents

System Programming Essentials with Go
Contributors
About the author
About the reviewer
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Conventions used
Get in touch
Share Your Thoughts
Download a free PDF copy of this book
Part 1: Introduction
1
Why Go?
Choosing Go
Concurrency and goroutines
Concurrency
Goroutines
CSP-inspired model
Share by communication
Interacting with the OS
Tooling
go build
go test
go run
go vet
go fmt
Cross-platform development with Go
Summary
2
Refreshing Concurrency and Parallelism
Technical requirements
Understanding goroutines
WaitGroup
Changing shared state
Managing data races
Atomic operations
Mutexes
Making sense of channels
How to use channels
An unbuffered channel
Buffered channels
The guarantee of delivery
Latency
State and signaling
State
Signaling
Choosing your synchronization mechanism
Summary
Part 2: Interaction with the OS
3
Understanding System Calls
Technical requirements
Introduction to system calls
The catalog of services and identification
Information exchange
The syscall package
A closer look at the os and x/sys packages
x/sys package – low-level system calls
Operating system functionality
Portability
Everyday system calls
Tracing system calls
Tracing specific system calls
Developing and testing a CLI program
Standard streams
File descriptors
Creating a CLI application
Redirections and standard streams
Making it testable
Summary
4
File and Directory Operations
Technical requirements
Identifying unsafe file and directory permissions
Files and permissions
Scanning directories in Go
Understanding file paths
Using the path/filepath package
Traversing directories
Symbolic links and unlinking files
Symbolic links – the shortcut of the file world
Unlinking files – the great escape act
Calculating directory size
Finding duplicate files
Optimizing filesystem operations
Summary
5
Working with System Events
Managing system events
What are signals?
The os/signal package
Task scheduling in Go
Why schedule?
Basic scheduling
Handling timer signals
File monitoring
Inotify
fsnotify
File rotation
Process management
Execution and timeouts
Execute and control process execution time
Building a distributed lock manager in Go
Summary
6
Understanding Pipes in Inter-Process Communication
Technical requirements
What are pipes in IPC?
Why are pipes important?
Pipes in Golang
The mechanics of anonymous pipes
Navigating named pipes (Mkfifo())
Best practices – guidelines for using pipes
Efficient data handling
Error handling and resource management
Security considerations
Performance optimization
Developing a log processing tool
Summary
7
Unix Sockets
Introduction to Unix sockets
Creating a Unix socket
Going a little deeper into socket creation
Creating the client
Inspecting the socket with lsof
Building a chat server
The complete chat client
Serving HTTP under UNIX domain sockets
Client
HTTP request line
HTTP request header
Empty line signifying end of headers
The textproto package
Performance
Other common use cases
Summary
Part 3: Performance
8
Memory Management
Technical requirements
Garbage collection
Stack and heap allocation
The GC algorithm
GOGC
GC pacer
GODEBUG
Memory ballast
GOMEMLIMIT
Memory arenas
Using memory arenas
Summary
9
Analyzing Performance
Escape analysis
Stack and pointers
Pointers
Stack
Heap
How can we analyze?
Benchmarking your code
Writing your first benchmark
Memory allocations
Common pitfalls
CPU profiling
Memory profiling
Profiling memory over time
Preparing to explore the trade-offs
Summary
Part 4: Connected Apps
10
Networking
The net package
TCP sockets
HTTP servers and clients
HTTP verbs
HTTP status codes
Putting it all together
Securing the connection
Certificates
Advanced networking
UDP versus TCP
Summary
11
Telemetry
Technical requirements
Logs
Zap versus slog
Logging for debugging or monitoring?
What to log?
What not to log?
Traces
Effective tracing
Distributed tracing
Metrics
What metric should we use?
The OTel project
OTel
Summary
12
Distributing Your Apps
Technical requirements
Go Modules
The routine using modules
CI
Caching
Static analysis
Releasing your application
Summary
Part 5: Going Beyond
13
Capstone Project – Distributed Cache
Technical requirements
Understanding distributed caching
System requirements
Requirements
Design and trade-offs
Creating the project
Thread safety
Choosing the right approach
Adding thread safety
The interface
TCP
HTTP
Others
Eviction policies
Sharding
Summary
14
Effective Coding Practices
Technical requirements
Reusing resources
Using sync.Pool in a network server
Using sync.Pool for JSON marshaling
Executing tasks once
singleflight
Effective memory mapping
API usage
Advanced usage with protection and mapping flags
Avoiding common performance pitfalls
Leaking with time.After
Defer in for loops
Maps management
Resource management
Handling HTTP bodies
Channel mismanagement
Summary
15
Stay Sharp with System Programming
Real-world applications
Dropbox’s leap of faith
HashiCorp – Go from day one
Grafana Labs – visualizing success with Go
Docker – building a container revolution with Go
SoundCloud – from Ruby to Go
Navigating the system programming landscape
Go release notes and blog
Community
Contribution
Experimentation
Resources for continued learning
Advanced Programming in the UNIX Environment by W. Richard Stevens
Learn C Programming - Second Edition: A beginner’s guide to learning the most powerful and general-purpose programming language with ease
Linux Kernel Programming - Second Edition: A comprehensive and practical guide to kernel internals, writing modules, and kernel synchronization
Linux System Programming Techniques: Become a proficient Linux system programmer using expert recipes and techniques
Operating Systems: Design and Implementation by Andrew S. Tanenbaum
Unix Network Programming by W. Richard Stevens
Linux System Programming Techniques: Become a proficient Linux system programmer using expert recipes and techniques
Mastering Embedded Linux Programming - Third Edition: Create fast and reliable embedded solutions with Linux 5.4 and the Yocto Project 3.1 (Dunfell)
Modern Operating Systems by Andrew S. Tanenbaum
The Art of UNIX Programming by Eric S. Raymond
Your system programming journey
Appendix
Hardware Automation
Automation in system programming
USB
Application
The goal
The /proc/mounts file
Reading the files on the flash drive
Partitions versus blocks versus devices versus disks
Open source to the rescue!
Interacting with USB events
Bluetooth
Detecting the smartwatch
Locking the screen
XDG dilemma
The Wayland conundrum
Summary
Index
Why subscribe?
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts
Download a free PDF copy of this book