A Course in Machine Learning

✍ Scribed by Hal Daumé III

• Publisher: Self-published
• Year: 2017
• Tongue: English
• Leaves: 227
• Edition: 2017
• Category: Library

✦ Synopsis

CIML is a set of introductory materials that covers most major aspects of modern machine learning (supervised learning, unsupervised learning, large margin methods, probabilistic modeling, learning theory, etc.). It's focus is on broad applications with a rigorous backbone. A subset can be used for an undergraduate course; a graduate course could probably cover the entire material and then some.

✦ Table of Contents

About this Book
How to Use this Book
Why Another Textbook?
Organization and Auxilary Material
Acknowledgements
Decision Trees
What Does it Mean to Learn?
Some Canonical Learning Problems
The Decision Tree Model of Learning
Formalizing the Learning Problem
Chapter Summary and Outlook
Further Reading
Limits of Learning
Data Generating Distributions
Inductive Bias: What We Know Before the Data Arrives
Not Everything is Learnable
Underfitting and Overfitting
Separation of Training and Test Data
Models, Parameters and Hyperparameters
Real World Applications of Machine Learning
Further Reading
Geometry and Nearest Neighbors
From Data to Feature Vectors
K-Nearest Neighbors
Decision Boundaries
K-Means Clustering
Warning: High Dimensions are Scary
Further Reading
The Perceptron
Bio-inspired Learning
Error-Driven Updating: The Perceptron Algorithm
Geometric Intrepretation
Interpreting Perceptron Weights
Perceptron Convergence and Linear Separability
Improved Generalization: Voting and Averaging
Limitations of the Perceptron
Further Reading
Practical Issues
The Importance of Good Features
Irrelevant and Redundant Features
Feature Pruning and Normalization
Combinatorial Feature Explosion
Evaluating Model Performance
Cross Validation
Hypothesis Testing and Statistical Significance
Debugging Learning Algorithms
Bias/Variance Trade-off
Further Reading
Beyond Binary Classification
Learning with Imbalanced Data
Multiclass Classification
Ranking
Further Reading
Linear Models
The Optimization Framework for Linear Models
Convex Surrogate Loss Functions
Weight Regularization
Optimization with Gradient Descent
From Gradients to Subgradients
Closed-form Optimization for Squared Loss
Support Vector Machines
Further Reading
Bias and Fairness
Train/Test Mismatch
Unsupervised Adaptation
Supervised Adaptation
Fairness and Data Bias
How Badly can it Go?
Further Reading
Probabilistic Modeling
Classification by Density Estimation
Statistical Estimation
Naive Bayes Models
Prediction
Generative Stories
Conditional Models
Regularization via Priors
Further Reading
Neural Networks
Bio-inspired Multi-Layer Networks
The Back-propagation Algorithm
Initialization and Convergence of Neural Networks
Beyond Two Layers
Breadth versus Depth
Basis Functions
Further Reading
Kernel Methods
From Feature Combinations to Kernels
Kernelized Perceptron
Kernelized K-means
What Makes a Kernel
Support Vector Machines
Understanding Support Vector Machines
Further Reading
Learning Theory
The Role of Theory
Induction is Impossible
Probably Approximately Correct Learning
PAC Learning of Conjunctions
Occam's Razor: Simple Solutions Generalize
Complexity of Infinite Hypothesis Spaces
Further Reading
Ensemble Methods
Voting Multiple Classifiers
Boosting Weak Learners
Random Ensembles
Further Reading
Efficient Learning
What Does it Mean to be Fast?
Stochastic Optimization
Sparse Regularization
Feature Hashing
Further Reading
Unsupervised Learning
K-Means Clustering, Revisited
Linear Dimensionality Reduction
Autoencoders
Further Reading
Expectation Maximization
Grading an Exam without an Answer Key
Clustering with a Mixture of Gaussians
The Expectation Maximization Framework
Further Reading
Structured Prediction
Multiclass Perceptron
Structured Perceptron
Argmax for Sequences
Structured Support Vector Machines
Loss-Augmented Argmax
Argmax in General
Dynamic Programming for Sequences
Further Reading
Imitation Learning
Imitation Learning by Classification
Failure Analysis
Dataset Aggregation
Expensive Algorithms as Experts
Structured Prediction via Imitation Learning
Further Reading
Code and Datasets
Bibliography
Index

✦ Subjects

Machine Learning