Deep Learning in Production

✍ Scribed by Sergios Karagiannakos

• Year: 2022
• Tongue: English
• Leaves: 223
• Category: Library

✦ Table of Contents

Preface
Acknowledgements
About this Book
Welcome to Deep Learning in Production
Is this book for me?
What is the book’s goal?
Will this be difficult to learn?
Why should you read this book?
How to use this book?
How is the book structured?
Do I need to know anything else before I get started?
Designing a Machine Learning System
Machine learning: phase zero
Data engineering
Model engineering
DevOps engineering
Putting it all together
Tackling a real-life problem
Setting up a Deep Learning Workstation
Laptop setup
Laptop requirements
Operating system
Frameworks and libraries
Development tools
Terminal
Version control
Python package and environment management
IDE / code editor
Other tools
Writing and Structuring Deep Learning Code
Best practices
Project structure
Object-oriented programming
Configuration
Type checking
Documentation
Unit testing
Basics of unit testing
Unit tests in Python
Tests in Tensorflow
Mocking
Test coverage
Test example cases
Integration / acceptance tests
Debugging
How to a debug deep learning project?
Python’s debugger
Debugging data with schema validation
Logging
Python’s Logging module
Useful Tensorflow debugging and logging functions
Data Processing
ETL: Extract, Transform, Load
Data reading
Loading from multiple sources
Parallel data extraction
Processing
Loading
Iterators
Optimizing a data pipeline
Batching
Prefetching
Caching
Streaming
Training
Building a trainer
Creating a custom training loop
Training checkpoints
Saving the trained model
Visualizing the training with Tensorboard
Model validation
Training in the cloud
Getting started with cloud computing
Creating a VM instance
Connecting to the VM instance
Transferring files to the VM instance
Running the training remotely
Accessing training data from a remote environment
Distributed training
Data vs model parallelism
Training in a single machine
Synchronous training
Asynchronous training
Model parallelism
Serving
Preparing the model
Building the model’s inference function
Creating a web application using Flask
Basics of modern web applications
Exposing the deep learning model using Flask
Creating a client
Serving with uWSGI and Nginx
Basic Terminology
Designing a serving system
Setting up a uWSGI server with Flask
Setting up Nginx as a reverse proxy
Serving with model servers
Tensorflow Serving vs Flask
Export a Tensorflow model
Install Tensorflow Serving
Load a model
Multiple versions support
Multiple models support
Batching inferences
Deploying
Containerizing using Docker and Docker Compose
What is a container?
What is Docker
Setting up Docker
Building a deep learning Docker image
Running a deep learning Docker container
Creating an Nginx container
Defining multi-container Docker apps using Docker Compose
Deploying in a production environment
Using containers in Google Cloud
Allowing network traffic to the instance
Deploying in Google Cloud
Continuous Integration and Delivery (CI / CD)
Scaling
A journey from 1 to millions of users
First iterations of the machine learning app
Vertical vs horizontal scaling
Autoscaling
Cache mechanisms
Monitoring alerts
Retraining machine learning models
Model A/B testing
Offline inference
Growing with Kubernetes
What is Kubernetes?
Getting started with Kubernetes
Deploying with Google Kubernetes Engine
Scaling with Kubernetes
Updating the application
Monitoring the application
Running a (re)training job
Using Kubernetes with GPUs
Model A/B testing
Building an End-to-End Pipeline
MLOps
Basic principles
MLOps levels
Building a pipeline using TFX
TFX glossary
Data ingestion
Data validation
Feature engineering
Train the model
Validate model
Push model
Build a TFX pipeline
Run a TFX pipeline
MLOps with Vertex AI and Google Cloud
Hands on Vertex AI
Experimenting with notebooks
Loading data
Training the model
Deploying to Vertex AI
Creating a pipeline
More end-to-end solutions
Where to Go from Here
Appendix
List of Figures
Index
About the Author