Adaptive Machine Learning Algorithms with Python: Solve Data Analytics and Machine Learning Problems on Edge Devices

✍ Scribed by Chanchal Chatterjee

• Publisher: Apress
• Year: 2022
• Tongue: English
• Leaves: 290
• Edition: 1
• Category: Library

✦ Synopsis

Learn to use adaptive algorithms to solve real-world streaming data problems. This book covers a multitude of data processing challenges, ranging from the simple to the complex. At each step, you will gain insight into real-world use cases, find solutions, explore code used to solve these problems, and create new algorithms for your own use.

Authors Chanchal Chatterjee and Vwani P. Roychowdhury begin by introducing a common framework for creating adaptive algorithms, and demonstrating how to use it to address various streaming data issues. Examples range from using matrix functions to solve machine learning and data analysis problems to more critical edge computation problems. They handle time-varying, non-stationary data with minimal compute, memory, latency, and bandwidth. 

Upon finishing this book, you will have a solid understanding of how to solve adaptive machine learning and data analytics problems and be able to derive new algorithms for your own use cases. You will also come away with solutions to high volume time-varying data with high dimensionality in a low compute, low latency environment.

What You Will Learn

• Apply adaptive algorithms to practical applications and examples
• Understand the relevant data representation features and computational models for time-varying multi-dimensional data
• Derive adaptive algorithms for mean, median, covariance, eigenvectors (PCA) and generalized eigenvectors with experiments on real data
• Speed up your algorithms and put them to use on real-world stationary and non-stationary data
• Master the applications of adaptive algorithms on critical edge device computation applications

Who This Book Is ForMachine learning engineers, data scientist and architects, software engineers and architects handling edge device computation and data management.

✦ Table of Contents

Table of Contents
About the Author
About the Technical Reviewer
Acknowledgments
Preface
Chapter 1: Introduction
1.1 Commonly Used Features Obtained by Linear Transform
Data Whitening
Principal Components
Linear Discriminant Features
Singular Value Features
Summary
1.2 Multi-Disciplinary Origin of Linear Features
Hebbian Learning or Neural Biology
Auto-Associative Networks
Hetero-Associative Networks
Statistical Pattern Recognition
Information Theory
Optimization Theory
1.3 Why Adaptive Algorithms?
Iterative or Batch Processing of Static Data
My Approach: Adaptive Processing of Streaming Data
Requirements of Adaptive Algorithms
Real-World Use of Adaptive Matrix Computation Algorithms and GitHub
1.4 Common Methodology for Derivations of Algorithms
Matrix Algebra Problems Solved Here
1.5 Outline of The Book
Chapter 2: General Theories and Notations
2.1 Introduction
2.2 Stationary and Non-Stationary Sequences
2.3 Use Cases for Adaptive Mean, Median, and Covariances
Handwritten Character Recognition
Anomaly Detection of Streaming Data
2.4 Adaptive Mean and Covariance of Nonstationary Sequences
2.5 Adaptive Covariance and Inverses
2.6 Adaptive Normalized Mean Algorithm
Variations of the Adaptive Normalized Mean Algorithm
2.7 Adaptive Median Algorithm
2.8 Experimental Results
Chapter 3: Square Root and  Inverse Square Root
3.1 Introduction and Use Cases
Various Solutions for A½ and A–½
Outline of This Chapter
3.2 Adaptive Square Root Algorithm: Method 1
Objective Function
Adaptive Algorithm
3.3 Adaptive Square Root Algorithm: Method 2
Objective Function
Adaptive Algorithm
3.4 Adaptive Square Root Algorithm: Method 3
Adaptive Algorithm
3.5 Adaptive Inverse Square Root Algorithm: Method 1
Objective Function
Adaptive Algorithm
3.6 Adaptive Inverse Square Root Algorithm: Method 2
Objective Function
Adaptive Algorithm
3.7 Adaptive Inverse Square Root Algorithm: Method 3
Adaptive Algorithm
3.8 Experimental Results
Experiments for Adaptive Square Root Algorithms
Experiments for Adaptive Inverse Square Root Algorithms
3.9 Concluding Remarks
Chapter 4: First Principal Eigenvector
4.1 Introduction and Use Cases
Outline of This Chapter
4.2 Algorithms and Objective Functions
Adaptive Algorithms
Objective Functions
4.3 OJA Algorithm
Objective Function
Adaptive Algorithm
Rate of Convergence
4.4 RQ, OJAN, and LUO Algorithms
Objective Function
Adaptive Algorithms
Rate of Convergence
4.5 IT Algorithm
Objective Function
Adaptive Algorithm
Rate of Convergence
Upper Bound of ηk
4.6 XU Algorithm
Objective Function
Adaptive Algorithm
Rate of Convergence
Upper Bound of ηk
4.7 Penalty Function Algorithm
Objective Function
Adaptive Algorithm
Rate of Convergence
Upper Bound of ηk
4.8 Augmented Lagrangian 1 Algorithm
Objective Function and Adaptive Algorithm
Rate of Convergence
Upper Bound of ηk
4.9 Augmented Lagrangian 2 Algorithm
Objective Function
Adaptive Algorithm
Rate of Convergence
Upper Bound of ηk
4.10 Summary of Algorithms
4.11 Experimental Results
Experiments with Various Starting Vectors w0
Experiments with Various Data Sets: Set 1
Experiments with Various Data Sets: Set 2
Experiments with Real-World Non-Stationary Data
4.12 Concluding Remarks
Chapter 5: Principal and Minor Eigenvectors
5.1 Introduction and Use Cases
Unified Framework
Outline of This Chapter
5.2 Algorithms and Objective Functions
Summary of Objective Functions for Adaptive Algorithms
5.3 OJA Algorithms
OJA Homogeneous Algorithm
OJA Deflation Algorithm
OJA Weighted Algorithm
OJA Algorithm Python Code
5.4 XU Algorithms
XU Homogeneous Algorithm
XU Deflation Algorithm
XU Weighted Algorithm
XU Algorithm Python Code
5.5 PF Algorithms
PF Homogeneous Algorithm
PF Deflation Algorithm
PF Weighted Algorithm
PF Algorithm Python Code
5.6 AL1 Algorithms
AL1 Homogeneous Algorithm
AL1 Deflation Algorithm
AL1 Weighted Algorithm
AL1 Algorithm Python Code
5.7 AL2 Algorithms
AL2 Homogeneous Algorithm
AL2 Deflation Algorithm
AL2 Weighted Algorithm
AL2 Algorithm Python Code
5.8 IT Algorithms
IT Homogeneous Function
IT Deflation Algorithm
IT Weighted Algorithm
IT Algorithm Python Code
5.9 RQ Algorithms
RQ Homogeneous Algorithm
RQ Deflation Algorithm
RQ Weighted Algorithm
RQ Algorithm Python Code
5.10 Summary of Adaptive Eigenvector Algorithms
5.11 Experimental Results
5.12 Concluding Remarks
Chapter 6: Accelerated Computation of Eigenvectors
6.1 Introduction
Objective Functions for Gradient-Based Adaptive PCA
Outline of This Chapter
6.2 Gradient Descent Algorithm
6.3 Steepest Descent Algorithm
Computation of  for Steepest Descent
Steepest Descent Algorithm Code
6.4 Conjugate Direction Algorithm
Conjugate Direction Algorithm Code
6.5 Newton-Raphson Algorithm
Newton-Raphson Algorithm Code
6.6 Experimental Results
Experiments with Stationary Data
Experiments with Non-Stationary Data
Comparison with State-of-the-Art Algorithms
6.7 Concluding Remarks
Chapter 7: Generalized Eigenvectors
7.1 Introduction and Use Cases
Application of GEVD in Pattern Recognition
Application of GEVD in Signal Processing
Methods for Generalized Eigen-Decomposition
Outline of This Chapter
7.2 Algorithms and Objective Functions
Summary of Objective Functions for Adaptive GEVD Algorithms
Summary of Generalized Eigenvector Algorithms
7.3 OJA GEVD Algorithms
OJA Homogeneous Algorithm
OJA Deflation Algorithm
OJA Weighted Algorithm
OJA Algorithm Python Code
7.4 XU GEVD Algorithms
XU Homogeneous Algorithm
XU Deflation Algorithm
XI Weighted Algorithm
XU Algorithm Python Code
7.5 PF GEVD Algorithms
PF Homogeneous Algorithm
PF Deflation Algorithm
PF Weighted Algorithm
PF Algorithm Python Code
7.6 AL1 GEVD Algorithms
AL1 Homogeneous Algorithm
AL1 Deflation Algorithm
AL1 Weighted Algorithm
AL1 Algorithm Python Code
7.7 AL2 GEVD Algorithms
AL2 Homogeneous Algorithm
AL2 Deflation Algorithm
AL2 Weighted Algorithm
AL2 Algorithm Python Code
7.8 IT GEVD Algorithms
IT Homogeneous Algorithm
IT Deflation Algorithm
IT Weighted Algorithm
IT Algorithm Python Code
7.9 RQ GEVD Algorithms
RQ Homogeneous Algorithm
RQ Deflation Algorithm
RQ Weighted Algorithm
RQ Algorithm Python Code
7.10 Experimental Results
7.11 Concluding Remarks
Chapter 8: Real-World Applications of Adaptive Linear Algorithms
8.1 Detecting Feature Drift
INSECTS-incremental_balanced_norm Dataset: Eigenvector Test
Adaptive EVD of Semi-Stationary Components
Adaptive EVD of Non-Stationary Components
INSECTS-incremental-abrupt_balanced _norm Dataset
Electricity Dataset
8.2 Adapting to Incoming Data Drift
8.3 Compressing High Volume and High Dimensional Data
Data Representation (PCA) Features
8.4 Detecting Feature Anomalies
Yahoo Real Dataset
NOAA Dataset
References
Index