Hypermodern Python Tooling: Building Reliable Workflows for an Evolving Python Ecosystem

✍ Scribed by Claudio Jolowicz

• Publisher: O'Reilly Media
• Year: 2024
• Tongue: English
• Leaves: 308
• Category: Library

✦ Synopsis

Keeping up with the Python ecosystem can be daunting. Its developer tooling doesn't provide the out-of-the-box experience native to languages like Rust and Go. When it comes to long-term project maintenance or collaborating with others, every Python project faces the same problem: how to build reliable workflows beyond local development while staying in sync with the evolving ecosystem.

With this hands-on guide, Python developers will learn how to forge the moving parts of a Python project into an easy-to-use toolchain, using state-of-the-art tools including Poetry, Nox, pytest, mypy, pre-commit, Black, Ruff, uv, Rye, Hatch, and more. Author Claudio Jolowicz shows you how to create robust Python project structures complete with unit tests, static analysis, code formatting, and type checking.

You'll learn how to:

Create open source projects with state-of-the-art infrastructure

Build a custom infrastructure for all Python projects in a company or...

✦ Table of Contents

Preface
Who Should Read This Book?
How This Book Is Organized
References and Further Reading
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
I. Working with Python
1. Installing Python
Supporting Multiple Versions of Python
Locating Python Interpreters
Installing Python on Windows
The Python Launcher for Windows
Installing Python on macOS
Homebrew Python
The python.org Installers
Installing Python on Linux
Fedora Linux
Ubuntu Linux
Other Linux Distributions
The Python Launcher for Unix
Installing Python with pyenv
Installing Python from Anaconda
A Brave New World: Installing with Hatch and Rye
An Overview of Installers
Summary
2. Python Environments
A Tour of Python Environments
Python Installations
The interpreter
Python modules
Entry-point scripts
Other components
The Per-User Environment
Virtual Environments
Installing packages
Activation scripts
A look under the hood
Installing Applications with pipx
pipx in a Nutshell
Installing pipx
Managing Applications with pipx
Running Applications with pipx
Configuring pipx
Managing Environments with uv
Finding Python Modules
Module Objects
The Module Cache
Module Specs
Finders and Loaders
The Module Path
The current directory or the directory containing the script
The PYTHONPATH variable
The standard library
Site Packages
Back to the Basics
Summary
II. Python Projects
3. Python Packages
The Package Lifecycle
An Example Application
Why Packaging?
The pyproject.toml File
Building Packages with build
Uploading Packages with Twine
Installing Projects from Source
Project Layout
Managing Packages with Rye
Wheels and Sdists
Project Metadata
Naming Projects
Versioning Projects
Dynamic Fields
Entry-Point Scripts
Entry Points
Authors and Maintainers
The Description and README
Keywords and Classifiers
The Project URLs
The License
The Required Python Version
Dependencies and Optional Dependencies
Summary
4. Dependency Management
Adding Dependencies to the Example Application
Consuming an API with HTTPX
Console Output with Rich
Specifying Dependencies for a Project
Version Specifiers
Extras
Optional dependencies
Environment Markers
Development Dependencies
An Example: Testing with pytest
Optional Dependencies
Requirements Files
Locking Dependencies
Freezing Requirements with pip and uv
Compiling Requirements with pip-tools and uv
Summary
5. Managing Projects with Poetry
Installing Poetry
Creating a Project
The Project Metadata
The Package Contents
The Source Code
Managing Dependencies
Caret Constraints
Extras and Environment Markers
The Lock File
Updating Dependencies
Managing Environments
Dependency Groups
Package Repositories
Publishing Packages to Package Repositories
Fetching Packages from Package Sources
Extending Poetry with Plugins
Generating Requirements Files with the Export Plugin
Deploying Environments with the Bundle Plugin
The Dynamic Versioning Plugin
Summary
III. Testing and Static Analysis
6. Testing with pytest
Writing a Test
Managing Test Dependencies
Designing for Testability
Fixtures and Parameterization
Advanced Techniques for Fixtures
Extending pytest with Plugins
The pytest-httpserver Plugin
The pytest-xdist Plugin
The factory-boy and faker Libraries
Other Plugins
Summary
7. Measuring Coverage with Coverage.py
Using Coverage.py
Branch Coverage
Testing in Multiple Environments
Parallel Coverage
Measuring in Subprocesses
What Coverage to Aim For
Summary
8. Automation with Nox
First Steps with Nox
Working with Sessions
Working with Multiple Python Interpreters
Session Arguments
Automating Coverage
Session Notification
Automating Coverage in Subprocesses
Parameterizing Sessions
Session Dependencies
Using Nox with Poetry Projects
Locking Dependencies with nox-poetry
Summary
9. Linting with Ruff and pre-commit
Linting Basics
The Ruff Linter
Pyflakes and pycodestyle
Fantastic Linters and Where to Find Them
Disabling Rules and Warnings
Automation with Nox
The pre-commit Framework
First Steps with pre-commit
A Hook Up Close
Automatic Fixes
Running pre-commit from Nox
Running pre-commit from Git
The Ruff Formatter
Approaches to Code Formatting: autopep8
Approaches to Code Formatting: YAPF
An Uncompromising Code Formatter
The Black Code Style
Formatting Code with Ruff
Summary
10. Using Types for Safety and Inspection
Benefits and Costs of Type Annotations
A Brief Tour of Python’s Typing Language
Variable Annotations
The Subtype Relation
Union Types
Gradual Typing
Function Annotations
Annotating Classes
Type Aliases
Generics
Protocols
Compatibility with Older Python Versions
Static Type Checking with mypy
First Steps with mypy
Revisiting the Wikipedia Example
Strict Mode
Automating mypy with Nox
Distributing Types with Python Packages
Type Checking the Tests
Inspecting Type Annotations at Runtime
Writing a @dataclass Decorator
Runtime Type Checking
Serialization and Deserialization with cattrs
Runtime Type Checking with Typeguard
Summary
Index