Designing Machine Learning Systems

✍ Scribed by Chip Huyen

• Publisher: O'Reilly Media, Inc.
• Year: 2022
• Tongue: English
• Leaves: 143
• Category: Library

✦ Synopsis

Many tutorials show you how to develop ML systems from ideation to deployed models. But with constant changes in tooling, those systems can quickly become outdated. Without an intentional design to hold the components together, these systems will become a technical liability, prone to errors and be quick to fall apart.

In this book, Chip Huyen provides a framework for designing real-world ML systems that are quick to deploy, reliable, scalable, and iterative. These systems have the capacity to learn from new data, improve on past mistakes, and adapt to changing requirements and environments. Youâ??ll learn everything from project scoping, data management, model development, deployment, and infrastructure to team structure and business analysis.

Learn the challenges and requirements of an ML system in production
Build training data with different sampling and labeling methods
Leverage best techniques to engineer features for your ML models to avoid data leakage
Select, develop, debug, and evaluate ML models that are best suit for your tasks
Deploy different types of ML systems for different hardware
Explore major infrastructural choices and hardware designs
Understand the human side of ML, including integrating ML into business, user experience, and team structure

✦ Table of Contents

1. Machine Learning Systems in Production
   When and When not to Use Machine Learning
   When To Use Machine Learning
   When not to Use Machine Learning
   Machine Learning Use Cases
   Understanding Machine Learning Systems
   Machine learning in research vs. in production
   Machine learning systems vs. traditional software
   Designing ML Systems in Production
   Requirements for ML Systems
   Iterative Process
   Summary
2. Data Engineering Fundamentals
   Mind vs. Data
   Data Sources
   Data Formats
   JSON
   Row-major vs. Column-major Format
   Text vs. Binary Format
   Data Processing and Storage
   Transactional and Analytical Databases
   ETL: Extract, Transform, Load
   Summary
3. Training Data
   Sampling
   Non-Probability Sampling
   Simple Random Sampling
   Stratified Sampling
   Weighted Sampling
   Importance Sampling
   Reservoir Sampling
   Labeling
   Hand Labels
   Handling the Lack of Hand Labels
   Class Imbalance
   Challenges of Class Imbalance
   Handling Class Imbalance
   Data Augmentation
   Simple Label-Preserving Transformations
   Perturbation
   Data Synthesis
   Summary
4. Feature Engineering
   Learned Features vs. Engineered Features
   Common Feature Engineering Operations
   Handling Missing Values
   Scaling
   Discretization
   Encoding Categorical Features
   Feature Crossing
   Discrete and Continuous Positional Embeddings
   Data Leakage
   Common Causes for Data Leakage
   Detecting Data Leakage
   Engineering Good Features
   Feature Importance
   Feature Generalization
   Summary
5. Model Development
   Model Selection
   Basic ML Review
   Choosing ML Models
   Ensembles
   AutoML
   Model Training
   Distributed Training
   Experiment Tracking and Versioning
   Model Offline Evaluation
   Baselines
   Evaluation Methods
   Summary