Machine Learning for High-Risk Applications

✍ Scribed by Patrick Hall

• Publisher: O'Reilly Media, Inc.
• Year: 2021
• Tongue: English
• Leaves: 112
• Category: Library

✦ Table of Contents

Preface
Who Should Read This Book
What Readers Will Learn
Preliminary Book Outline
Bringing it All Together
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
1. Contemporary Model Governance
Basic Legal Obligations
AI Incidents
Organizational and Cultural Competencies for Responsible AI
Accountability
Drinking Your Own Champagne
Diverse and Experienced Teams
“Going Fast and Breaking Things”
Organizational Processes for Responsible AI
Forecasting Failure Modes
Model Risk Management
Beyond Model Risk Management
Case Study: Death by Autonomous Vehicle
Fallout
An Unprepared Legal System
Lessons Learned
2. How to Red-Team AI Systems
Security Basics
The Adversarial Mindset
CIA Triad
Best Practices for Data Scientists
Machine Learning Attacks
Integrity Attacks: Manipulated Machine Learning Outputs
Confidentiality Attacks: Extracted Information
General AI Security Concerns
Counter-measures
Model Debugging for Security
Model Monitoring For Security
Privacy-enhancing Technologies
Robust Machine Learning
General Countermeasures
Case Study: Real-world Evasion Attacks
Lessons Learned
Resources
3. Debugging AI Systems for Safety and Performance
Training
Reproducibility
Data Quality and Feature Engineering
Model Specification
Model Debugging
Software Testing
Traditional Model Assessment
Residual Analysis for Machine Learning
Sensitivity Analysis
Benchmark Models
Machine Learning Bugs
Remediation: Fixing Bugs
Deployment
Domain Safety
Model Monitoring
Case Study: Remediating the Strawman
Resources