Engineering MLOps: Rapidly build, test, and manage production-ready machine learning life cycles at scale

✍ Scribed by Emmanuel Raj

• Publisher: Packt Publishing
• Year: 2021
• Tongue: English
• Leaves: 370
• Category: Library

✦ Synopsis

Get up and running with machine learning life cycle management and implement MLOps in your organization

Key Features

• Become well-versed with MLOps techniques to monitor the quality of machine learning models in production
• Explore a monitoring framework for ML models in production and learn about end-to-end traceability for deployed models
• Perform CI/CD to automate new implementations in ML pipelines

Book Description

MLOps is a systematic approach to building, deploying, and monitoring machine learning (ML) solutions. It is an engineering discipline that can be applied to various industries and use cases. This book presents comprehensive insights into MLOps coupled with real-world examples to help you to write programs, train robust and scalable ML models, and build ML pipelines to train and deploy models securely in production.

The book begins by familiarizing you with the MLOps workflow so you can start writing programs to train ML models. Then you'll then move on to explore options for serializing and packaging ML models post-training to deploy them to facilitate machine learning inference, model interoperability, and end-to-end model traceability. You'll understand how to build ML pipelines, continuous integration and continuous delivery (CI/CD) pipelines, and monitoring pipelines to systematically build, deploy, monitor, and govern ML solutions for businesses and industries. Finally, you'll apply the knowledge you've gained to build real-world projects.

By the end of this ML book, you'll have a 360-degree view of MLOps and be ready to implement MLOps in your organization.

What you will learn

• Formulate data governance strategies and pipelines for ML training and deployment
• Get to grips with implementing ML pipelines, CI/CD pipelines, and ML monitoring pipelines
• Design a robust and scalable microservice and API for test and production environments
• Curate your custom CD processes for related use cases and organizations
• Monitor ML models, including monitoring data drift, model drift, and application performance
• Build and maintain automated ML systems

Who this book is for

This MLOps book is for data scientists, software engineers, DevOps engineers, machine learning engineers, and business and technology leaders who want to build, deploy, and maintain ML systems in production using MLOps principles and techniques. Basic knowledge of machine learning is necessary to get started with this book.

Table of Contents

1. Fundamentals of MLOps Workflow
2. Characterizing your Machine learning problem
3. Code Meets Data
4. Machine Learning Pipelines
5. Model evaluation and packaging
6. Key principles for deploying your ML system
7. Building robust CI and CD pipelines
8. APIs and microservice Management
9. Testing and Securing Your ML Solution
10. Essentials of Production Release
11. Key principles for monitoring your ML system
12. Model Serving and Monitoring
13. Governing the ML system for Continual Learning

✦ Table of Contents

Cover
Title Page
Copyright and Credits
Contributors
Table of Contents
Preface
Section 1: Framework for Building Machine Learning Models
Chapter 1: Fundamentals of an MLOps Workflow
The evolution of infrastructure and software development
The rise of machine learning and deep learning
The end of Moore's law
AI-centric applications
Software development evolution
Traditional software development challenges
Trends of ML adoption in software development
Understanding MLOps
Concepts and workflow of MLOps
Discussing a use case
Summary
Chapter 2: Characterizing Your Machine Learning Problem
The ML solution development process
Types of ML models
Learning models
Hybrid models
Statistical models
HITL models
Structuring your MLOps
Small data ops
Big data ops
Hybrid MLOps
Large-scale MLOps
An implementation roadmap for your solution
Phase 1 – ML development
Phase 2 – Transition to operations
Phase 3 – Operations
Procuring data, requirements, and tools
Data
Requirements
Tools and infrastructure
Discussing a real-life business problem
Summary
Chapter 3: Code Meets Data
Business problem analysis and categorizing the problem
Setting up the resources and tools
Installing MLflow
Azure Machine Learning
Azure DevOps
JupyterHub
10 principles of source code management for ML
What is good data for ML?
Data preprocessing
Data quality assessment
Calibrating missing data
Label encoding 
New feature – Future_weather_condition
Data correlations and filtering
Time series analysis
Data registration and versioning
Toward the ML Pipeline
Feature Store
Summary
Chapter 4: Machine Learning Pipelines
Going through the basics of ML pipelines
Data ingestion and feature engineering
Data ingestion (training dataset)
Machine learning training and hyperparameter optimization
Support Vector Machine
Random Forest classifier
Model testing and defining metrics
Testing the SVM classifier
Testing the Random Forest classifier
Model packaging
Registering models and production artifacts
Registering production artifacts
Summary
Chapter 5: Model Evaluation and Packaging
Model evaluation and interpretability metrics
Learning models' metrics
Hybrid models' metrics
Statistical models' metrics
HITL model metrics
Production testing methods
Batch testing
A/B testing
Stage test or shadow test
Testing in CI/CD
Why package ML models?
Portability
Inference
Interoperability
Deployment agnosticity
How to package ML models
Serialized files
Packetizing or containerizing
Microservice generation and deployment
Inference ready models
Connecting to the workspace and importing model artifacts
Loading model artifacts for inference
Summary
Section 2: Deploying Machine Learning Models at Scale
Chapter 6: Key Principles for Deploying Your ML System
ML in research versus production
Data
Fairness
Interpretability
Performance
Priority
Understanding the types of ML inference in production
Deployment targets
Mapping the infrastructure for our solution
Hands-on deployment (for the business problem)
Deploying the model on ACI
Deploying the model on Azure Kubernetes Service (AKS)
Deploying the service using MLflow
Understanding the need for continuous integration and continuous deployment
Summary
Chapter 7: Building Robust CI-CD Pipelines
Continuous integration, delivery, and deployment in MLOps
Continuous integration
Continuous delivery
Continuous deployment
Setting up a CI-CD pipeline and the test environment (using Azure DevOps)
Creating a service principal
Installing the extension to connect to the Azure ML workspace
Setting up a continuous integration and deployment pipeline for the test environment
Connecting artifacts to the pipeline
Setting up a test environment
Pipeline execution and testing
Pipeline execution triggers
Summary
Chapter 8: APIs and Microservice Management
Introduction to APIs and microservices
What is an Application Programming Interface (API)?
Microservices
The need for microservices for ML
Hypothetical use case
Stage 1 – Proof of concept (a monolith)
Stage 2 – Production (microservices)
Old is gold – REST API-based microservices
Hands-on implementation of serving an ML model as an API
API design and development
Developing a microservice using Docker
Testing the API
Summary
Chapter 9: Testing and Securing Your ML Solution
Understanding the need for testing and securing your ML application
Testing your ML solution by design
Data testing
Model testing
Pre-training tests
Post-training tests
Hands-on deployment and inference testing (a business use case)
Securing your ML solution by design
Types of attacks
Summary
Chapter 10: Essentials of Production Release
Setting up the production infrastructure
Azure Machine Learning workspace
Azure Machine Learning SDK
Setting up our production environment in the CI/CD pipeline
Testing our production-ready pipeline
Configuring pipeline triggers for automation
Setting up a Git trigger
Setting up an Artifactory trigger
Setting up a Schedule trigger
Pipeline release management
Toward continuous monitoring
Summary
Section 3: Monitoring Machine Learning Models in Production
Chapter 11: Key Principles for Monitoring Your ML System
Understanding the key principles of monitoring an ML system
Model drift
Model bias
Model transparency
Model compliance
Explainable AI
Monitoring in the MLOps workflow
Understanding the Explainable Monitoring Framework
Monitor
Analyze
Govern
Enabling continuous monitoring for the service
Summary
Chapter 12: Model Serving and Monitoring
Serving, monitoring, and maintaining models in production
Exploring different modes of serving ML models
Serving the model as a batch service
Serving the model to a human user
Serving the model to a machine
Implementing the Explainable Monitoring framework
Monitoring your ML system
Analyzing your ML system
Governing your ML system
Summary
Chapter 13: Governing the ML System for Continual Learning
Understanding the need for continual learning
Continual learning
The need for continual learning
Explainable monitoring – governance
Alerts and actions
Model QA and control
Model auditing and reports
Enabling model retraining
Manual model retraining
Automated model retraining
Maintaining the CI/CD pipeline
Summary
Why subscribe?
About Packt
Other Books You May Enjoy
Index