Hands-On Machine Learning with C++: Build, train, and deploy end-to-end machine learning and deep learning pipelines

✍ Scribed by Kirill Kolodiazhnyi

• Publisher: Packt Publishing
• Year: 2020
• Tongue: English
• Leaves: 515
• Category: Library

✦ Synopsis

Implement supervised and unsupervised machine learning algorithms using C++ libraries such as PyTorch C++ API, Caffe2, Shogun, Shark-ML, mlpack, and dlib with the help of real-world examples and datasets

Key Features

• Become familiar with data processing, performance measuring, and model selection using various C++ libraries
• Implement practical machine learning and deep learning techniques to build smart models
• Deploy machine learning models to work on mobile and embedded devices

Book Description

C++ can make your machine learning models run faster and more efficiently. This handy guide will help you learn the fundamentals of machine learning (ML), showing you how to use C++ libraries to get the most out of your data. This book makes machine learning with C++ for beginners easy with its example-based approach, demonstrating how to implement supervised and unsupervised ML algorithms through real-world examples.

This book will get you hands-on with tuning and optimizing a model for different use cases, assisting you with model selection and the measurement of performance. You'll cover techniques such as product recommendations, ensemble learning, and anomaly detection using modern C++ libraries such as PyTorch C++ API, Caffe2, Shogun, Shark-ML, mlpack, and dlib. Next, you'll explore neural networks and deep learning using examples such as image classification and sentiment analysis, which will help you solve various problems. Later, you'll learn how to handle production and deployment challenges on mobile and cloud platforms, before discovering how to export and import models using the ONNX format.

By the end of this C++ book, you will have real-world machine learning and C++ knowledge, as well as the skills to use C++ to build powerful ML systems.

What you will learn

• Explore how to load and preprocess various data types to suitable C++ data structures
• Employ key machine learning algorithms with various C++ libraries
• Understand the grid-search approach to find the best parameters for a machine learning model
• Implement an algorithm for filtering anomalies in user data using Gaussian distribution
• Improve collaborative filtering to deal with dynamic user preferences
• Use C++ libraries and APIs to manage model structures and parameters
• Implement a C++ program to solve image classification tasks with LeNet architecture

Who this book is for

You will find this C++ machine learning book useful if you want to get started with machine learning algorithms and techniques using the popular C++ language. As well as being a useful first course in machine learning with C++, this book will also appeal to data analysts, data scientists, and machine learning developers who are looking to implement different machine learning models in production using varied datasets and examples. Working knowledge of the C++ programming language is mandatory to get started with this book.

Table of Contents

1. Introduction to Machine Learning with C++
2. Data Processing
3. Measuring Performance and Selecting Models
4. Clustering
5. Anomaly Detection
6. Dimensionality Reduction
7. Classification
8. Recommender Systems
9. Ensemble Learning
10. Neural Networks for Image Classification
11. Sentiment Analysis with Recurrent Neural Networks
12. Exporting and Importing Models
13. Deploying Models on Mobile and Cloud Platforms

✦ Table of Contents

Cover
Title Page
Copyright and Credits
About Packt
Contributors
Table of Contents
Preface
Section 1: Overview of Machine Learning
Chapter 1: Introduction to Machine Learning with C++
Understanding the fundamentals of ML
Venturing into the techniques of ML
Supervised learning
Unsupervised learning 
Dealing with ML models
Model parameter estimation
An overview of linear algebra 
Learning the concepts of linear algebra
Basic linear algebra operations
Tensor representation in computing
Linear algebra API samples
Using Eigen
Using xtensor
Using Shark-ML
Using Dlib
An overview of linear regression
Solving linear regression tasks with different libraries
Solving linear regression tasks with Eigen
Solving linear regression tasks with Shogun
Solving linear regression tasks with Shark-ML
Linear regression with Dlib
Summary
Further reading
Chapter 2: Data Processing
Technical requirements
Parsing data formats to C++ data structures
Reading CSV files with the Fast-CPP-CSV-Parser library
Preprocessing CSV files
Reading CSV files with the Shark-ML library
Reading CSV files with the Shogun library
Reading CSV files with the Dlib library
Reading JSON files with the RapidJSON library
Writing and reading HDF5 files with the HighFive library
Initializing matrix and tensor objects from C++ data structures
Eigen
Shark-ML
Dlib
Shogun
Manipulating images with the OpenCV and Dlib libraries
Using OpenCV 
Using Dlib 
Transforming images into matrix or tensor objects of various libraries
Deinterleaving in OpenCV
Deinterleaving in Dlib
Normalizing data
Normalizing with Eigen
Normalizing with Shogun
Normalizing with Dlib
Normalizing with Shark-ML
Summary
Further reading
Chapter 3: Measuring Performance and Selecting Models
Technical requirements
Performance metrics for ML models
Regression metrics
Mean squared error and root mean squared error
Mean absolute error
R squared
Adjusted R squared
Classification metrics
Accuracy
Precision and recall
F-score
AUC–ROC
Log-Loss
Understanding the bias and variance characteristics
Bias
Variance
Normal training
Regularization
L1 regularization – Lasso
L2 regularization – Ridge
Data augmentation
Early stopping
Regularization for neural networks
Model selection with the grid search technique
Cross-validation
K-fold cross-validation
Grid search
Shogun example
Shark-ML example
Dlib example
Summary
Further reading
Section 2: Machine Learning Algorithms
Chapter 4: Clustering
Technical requirements
Measuring distance in clustering
Euclidean distance
Squared Euclidean distance
Manhattan distance
Chebyshev distance
Types of clustering algorithms
Partition-based clustering algorithms
Distance-based clustering algorithms
Graph theory-based clustering algorithms
Spectral clustering algorithms
Hierarchical clustering algorithms
Density-based clustering algorithms
Model-based clustering algorithms
Examples of using the Shogun library for dealing with the clustering task samples
GMM with Shogun
K-means clustering with Shogun
Hierarchical clustering with Shogun
Examples of using the Shark-ML library for dealing with the clustering task samples
Hierarchical clustering with Shark-ML
K-means clustering with Shark-ML
Examples of using the Dlib library for dealing with the clustering task samples
K-means clustering with Dlib
Spectral clustering with Dlib
Hierarchical clustering with Dlib
Newman modularity-based graph clustering algorithm with Dlib
Chinese Whispers – graph clustering algorithm with Dlib
Plotting data with C++
Summary
Further reading
Chapter 5: Anomaly Detection
Technical requirements
Exploring the applications of anomaly detection
Learning approaches for anomaly detection
Detecting anomalies with statistical tests
Detecting anomalies with the Local Outlier Factor method
Detecting anomalies with isolation forest
Detecting anomalies with One-Class SVM (OCSVM)
Density estimation approach (multivariate Gaussian distribution) for anomaly detection
C++ implementation of the isolation forest algorithm for anomaly detection
Using the Dlib library for anomaly detection
One-Cass SVM with Dlib
Multivariate Gaussian model with Dlib
OCSVM with Shogun
OCSVM with Shark-ML
Summary
Further reading
Chapter 6: Dimensionality Reduction
Technical requirements
An overview of dimension reduction methods
Feature selection methods
Dimensionality reduction methods
Exploring linear methods for dimension reduction
Principal component analysis
Singular value decomposition 
Independent component analysis
Linear discriminant analysis
Factor analysis
Multidimensional scaling 
Exploring non-linear methods for dimension reduction
Kernel PCA
IsoMap
Sammon mapping
Distributed stochastic neighbor embedding
Autoencoders
Understanding dimension reduction algorithms with various С++ libraries
Using the Dlib library
PCA
Data compression with PCA
LDA
Sammon mapping
Using the Shogun library
PCA
Kernel PCA
MDS
IsoMap
ICA
Factor analysis
t-SNE
Using the Shark-ML library
PCA
LDA
Summary
Further reading
Chapter 7: Classification
Technical requirements
An overview of classification methods
Exploring various classification methods
Logistic regression
KRR
SVM
kNN method
Multi-class classification
Examples of using C++ libraries for dealing with the classification task
Using the Shogun library
With logistic regression
With SVMs
With the kNN algorithm
Using the Dlib library
With KRR
With SVM
Using the Shark-ML library
With logistic regression
With SVM
With the kNN algorithm
Summary
Further reading
Chapter 8: Recommender Systems
Technical requirements
An overview of recommender system algorithms 
Non-personalized recommendations
Content-based recommendations
User-based collaborative filtering
Item-based collaborative filtering
Factorization algorithms
Similarity or preferences correlation
Pearson's correlation coefficient
Spearman's correlation
Cosine distance
Data scaling and standardization
Cold start problem
Relevance of recommendations
Assessing system quality 
Understanding collaborative filtering method details 
Examples of item-based collaborative filtering with C++
Using the Eigen library 
Using the mlpack library
Summary
Further reading
Chapter 9: Ensemble Learning
Technical requirements
An overview of ensemble learning
Using a bagging approach for creating ensembles
Using a gradient boosting method for creating ensembles
Using a stacking approach for creating ensembles
Using the random forest method for creating ensembles
Decision tree algorithm overview
Random forest method overview
Examples of using C++ libraries for creating ensembles
Ensembles with Shogun
Using gradient boosting with Shogun
Using random forest with Shogun
Ensembles with Shark-ML
Using random forest with Shark-ML
Using a stacking ensemble with Shark-ML
Summary
Further reading
Section 3: Advanced Examples
Chapter 10: Neural Networks for Image Classification
Technical requirements
An overview of neural networks 
Neurons
The perceptron and neural networks
Training with the backpropagation method
Backpropagation method modes
Stochastic mode
Batch mode
Mini-batch mode
Backpropagation method problems
The backpropagation method – an example
Loss functions
Activation functions
The stepwise activation function
The linear activation function
The sigmoid activation function
The hyperbolic tangent
Activation function properties
Regularization in neural networks
Different methods for regularization
Neural network initialization
Xavier initialization method
He initialization method
Delving into convolutional networks
Convolution operator
Pooling operation
Receptive field
Convolution network architecture
What is deep learning?
Examples of using C++ libraries to create neural networks
Simple network example for the regression task
Dlib
Shogun
Shark-ML
Architecture definition
Loss function definition
Network initialization
Optimizer configuration
Network training
The complete programming sample
Understanding image classification using the LeNet architecture
Reading the training dataset
Reading dataset files
Reading the image file
Neural network definition
Network training
Summary
Further reading
Chapter 11: Sentiment Analysis with Recurrent Neural Networks
Technical requirements
An overview of the RNN concept
Training RNNs using the concept of backpropagation through time
Exploring RNN architectures
LSTM 
GRUs
Bidirectional RNN
Multilayer RNN
Understanding natural language processing with RNNs
Word2Vec
GloVe
Sentiment analysis example with an RNN
Summary
Further reading
Section 4: Production and Deployment Challenges
Chapter 12: Exporting and Importing Models
Technical requirements
ML model serialization APIs in C++ libraries
Model serialization with Dlib
Model serialization with Shogun
Model serialization with Shark-ML
Model serialization with PyTorch
Neural network initialization
Using the torch::save and torch::load functions
Using PyTorch archive objects
Delving into ONNX format
Loading images into Caffe2 tensors
Reading the class definition file
Summary
Further reading
Chapter 13: Deploying Models on Mobile and Cloud Platforms
Technical requirements
Image classification on Android mobile
The mobile version of the PyTorch framework
Using TorchScript for a model snapshot
The Android Studio project
The UI and Java part of the project
The C++ native part of the project
Machine learning in the cloud – using Google Compute Engine
The server
The client
Service deployment
Summary
Further reading
Other Books You May Enjoy
Index