Embedded Machine Learning for Cyber-Physical, IoT, and Edge Computing: Use Cases and Emerging Challenges

✍ Scribed by Sudeep Pasricha (editor), Muhammad Shafique (editor)

• Publisher: Springer
• Year: 2023
• Tongue: English
• Leaves: 571
• Category: Library

✦ Synopsis

This book presents recent advances towards the goal of enabling efficient implementation of machine learning models on resource-constrained systems, covering different application domains. The focus is on presenting interesting and new use cases of applying machine learning to innovative application domains, exploring the efficient hardware design of efficient machine learning accelerators, memory optimization techniques, illustrating model compression and neural architecture search techniques for energy-efficient and fast execution on resource-constrained hardware platforms, and understanding hardware-software codesign techniques for achieving even greater energy, reliability, and performance benefits.

• Discusses efficient implementation of machine learning in embedded, CPS, IoT, and edge computing;
• Offers comprehensive coverage of hardware design, software design, and hardware/software co-design and co-optimization;
• Describes real applications to demonstrate how embedded, CPS, IoT, and edge applications benefit from machine learning.

✦ Table of Contents

Preface
Acknowledgments
Contents
Part I Mobile, IoT, and Edge Application Use-Cases for Embedded Machine Learning
Convolutional Neural Networks for Efficient Indoor Navigation with Smartphones
1 Introduction
2 Related Works
3 Convolutional Neural Networks
4 CNNLOC Framework: Overview
4.1 Overview
4.2 Preprocessing of RSSI Data
4.3 RSSI Image Database
4.4 Hyperparameters
4.5 Integrating Hierarchy for Scalability
5 Experiments
5.1 Experimental Setup
5.2 Smartphone Implementation
5.3 Experimental Results
5.3.1 Indoor Localization Accuracy Comparison
5.3.2 CNNLOC Scalability Analysis
5.3.3 Accuracy Analysis with Other Approaches
6 Conclusion
References
An End-to-End Embedded Neural Architecture Searchand Model Compression Framework for Healthcare Applicationsand Use-Cases
1 Introduction
1.1 Deep Learning in Healthcare: Potential Use-Cases and Applications
2 Embedded Neural Architecture Search and Model Compression Framework for Healthcare Applications
2.1 User Specifications and Requirements
2.2 Platform Constraints
2.3 Dataset Construction
2.4 Deep Learning Model Generation
2.5 Deep Learning Model Training and Evaluation
2.6 Model Compression
2.6.1 Pruning
2.6.2 Quantization
3 Case Study: Bio-signal Anomaly Detection
3.1 Experimental Setup
3.2 Exhaustive Exploration
3.3 Selective Exploration: Time Benefits
3.4 Selective Exploration: Efficacy and Analysis
3.5 Selective Exploration: Weighted Exploration
3.6 Pruning and Quantization: Compression Efficacy and Receiver Operating Characteristics
4 Conclusion
References
Robust Machine Learning for Low-Power Wearable Devices: Challenges and Opportunities
1 Introduction
2 Edge Machine Learning in Wearable Devices
2.1 Edge Machine Learning Architectures
2.2 Edge Machine Learning Algorithms
2.2.1 Tree-Based Machine Learning Algorithms
2.2.2 Support Vector Machines
2.2.3 Neural Networks and Deep Learning
2.3 Challenges for On-Device Edge Machine Learning in Wearable Devices
2.4 Solutions to Address On-Device Learning Challenges
2.4.1 Quantization
2.4.2 Model Pruning
2.4.3 Energy Harvesting
2.5 Edge Machine Learning in Health Applications
2.5.1 Parkinson's Disease Diagnosis
2.5.2 Vital Sign Monitoring
2.5.3 Human–Computer Interaction
3 Robustness in Wearable Applications
3.1 Reliability Challenges for Wearable Devices
3.1.1 Sensor Shifts and Disturbances
3.1.2 Missing Sensor Data
3.1.3 Energy and Power Constraints
3.2 State-of-the-Art Methods for Robust Edge Machine Learning in Wearable Devices
3.3 Approaches to Address Sensor Shifts and Disturbances
3.4 Missing Data Recovery Algorithms
3.4.1 Energy Management in Wearable Devices
3.5 Future Opportunities
4 Human Activity Recognition Case Study
4.1 HAR Background
4.2 Generative Adversarial Imputation Networks
4.3 Experiments and Results
4.3.1 Experimental Setup
4.3.2 Design Space Exploration for GAIN
5 Conclusion
References
Efficient Deep Vision for Aerial Visual Understanding
1 Introduction
2 Domain-Specific Small ConvNets for UAV Applications
2.1 Disaster Classification
2.1.1 Network Design
2.1.2 Experiments and Results
2.2 Vehicle Detection
2.2.1 Network Design and Approach
2.2.2 Results
3 Processing Aerial Images with Tiling
3.1 Approach
3.1.1 Initial Position Estimation
3.1.2 Tiling and CNN Selection
3.1.3 Optical Flow-Based Tracker
3.2 Evaluation of EdgeNet Framework
3.2.1 Metrics
3.2.2 Configuration Analysis
3.2.3 Performance Analysis on CPU, Odroid, and Raspberry Platforms
4 Combining Tiling with Quantization
4.1 Quantization Techniques
4.2 Approach
4.3 Experimental Results
5 Conclusion
References
Edge-Centric Optimization of Multi-modal ML-Driven eHealth Applications
1 Introduction
1.1 ML in Smart eHealth Applications
1.2 Collaborative Edge Computing for Smart eHealth Applications
1.2.1 Example Scenario
1.3 Summary
1.3.1 Organization
2 Exemplar Case Study of Edge-ML-Driven Pain Assessment Application
2.1 Pain Assessment
2.2 Sensory Data Acquisition
2.2.1 Types of Signals
2.2.2 Commonly Used Sensors
2.2.3 Multi-modal Inputs
2.3 ML-Driven Objective Pain Assessment
2.3.1 iHurt Platform
2.3.2 Other Exemplar eHealth Applications
3 Edge-Centric Optimization of ML-Based eHealth Workloads
3.1 Dynamics of Compute Placement
3.2 Using RL for Optimization
3.2.1 Orchestration Framework
3.2.2 RL Agent for Orchestration
4 Sense–Compute Co-optimization of ML-driven eHealth Applications
4.1 Handling Input Data Perturbations
4.1.1 Sensing Phase Knobs
4.1.2 Sense-Making Phase Knobs
4.1.3 Co-optimization Knobs
4.1.4 Example Scenarios
4.2 Sense–Compute Co-optimization Framework
5 Conclusions
5.1 Key Insights
5.2 Open Research Directions
5.2.1 Data Quality Management
5.2.2 Contextual Edge Orchestration
5.2.3 Sense–Compute Co-optimization
References
A Survey of Embedded Machine Learning for Smart and Sustainable Healthcare Applications
1 Introduction
2 Overview of Embedded Machine Learning Frameworks
3 Embedded Machine Learning Applications for Healthcare
3.1 Freezing-of-Gait Identification in PD Patients
3.2 Human Activity Recognition
3.2.1 Processing Pipeline
3.2.2 Commonly Used ML Algorithms
3.2.3 Offline vs. Online Learning
3.3 Human Pose Estimation
3.3.1 Human Pose Estimation Using RGB Camera
3.3.2 Human Pose Estimation Using mmWave Radar
3.3.3 Human Pose Estimation Using Inertial Sensors
4 Energy Management
4.1 Energy Sources and Budget
4.2 Optimal Energy Management
5 Conclusions
References
Reinforcement Learning for Energy-Efficient Cloud Offloading of Mobile Embedded Applications
1 Introduction
2 Prior Work
3 Challenges with Offloading
4 Offloading Performance of Mobile Applications
4.1 Experimental Setup
4.2 Experimental Results
4.2.1 Matrix Operation App
4.2.2 Internet Browser App
4.2.3 Zipper App
4.2.4 Voice Recognition and Translation App
4.2.5 Torrents App
4.3 Summary of Findings
5 Adaptive Offloading
6 Middleware Framework for Efficient Offloading of Mobile Applications
6.1 Reinforcement Learning (RL)
6.2 RL Algorithm to Generate Q-Function
7 Experimental Results
8 Conclusions and Future Work
References
Part II Cyber-Physical Application Use-Cases for Embedded Machine Learning
Context-Aware Adaptive Anomaly Detection in IoT Systems
1 Introduction
1.1 Motivational Example
1.2 Threat Model
1.3 Research Challenges
1.4 Contributions
2 Related Works
3 Anomaly Detection Methodology
3.1 Context Generation
3.2 Sensor Association
3.3 Predictive Model
3.4 Anomaly Detection
3.5 Model Adaptation
4 Results and Evaluation
4.1 Fog Computing Architecture
4.2 Experimental Setup
4.3 Evaluation
4.3.1 Sensor Association Evaluation
4.3.2 Anomaly Detection Evaluation
4.4 Robustness
4.5 Case Study
4.6 Timing Analysis
4.7 Aliveness Assessment
5 Conclusion
References
Machine Learning Components for Autonomous Navigation Systems
1 Introduction
2 Sensor Data Fusion
3 Perception
3.1 RGB Object Detection
3.2 LiDAR Object Detection
3.3 RADAR Object Detection
3.4 Multi-Modal Object Detection
3.5 Perception-Driven Sensing
3.6 Adaptive Computational Load Control
4 Odometery, Localization, and Mapping
4.1 LiDAR Odometry and Mapping
4.2 Unsupervised VSLAM and VO
5 Planning
5.1 Overview of Reinforcement Learning Algorithms
5.2 Applications of Deep Reinforcement Learning Algorithms for Planning
6 End-to-End Learning Systems
7 Conclusion
References
Machine Learning for Efficient Perception in Automotive Cyber-Physical Systems
1 Introduction
2 Related Work
3 Background
3.1 ADAS Level 2 Autonomy Features
3.2 Sensor Placement and Orientation
3.3 Object Detection for Vehicle Environment Perception
3.4 Sensor Fusion
4 PASTA Architecture
4.1 Overview
4.2 Problem Formulation and Metrics
4.3 Design Space Encoder/Decoder
4.4 Design Space Exploration
4.4.1 Genetic Algorithm (GA)
4.4.2 Differential Evolution (DE)
4.4.3 Firefly Algorithm (FA)
4.5 Performance Evaluation
5 Experiments
5.1 Experimental Setup
5.2 Experimental Results
6 Conclusion
References
Machine Learning for Anomaly Detection in Automotive Cyber-Physical Systems
1 Introduction
2 Related Work
3 Sequence Learning Background
3.1 Sequence Models
3.1.1 Recurrent Neural Networks (RNNs)
3.1.2 Long-/Short-Term Memory (LSTM) Networks
3.1.3 Gated Recurrent Unit (GRU)
3.2 Autoencoders
4 Problem Definition
4.1 System Model
4.2 Communication Model
4.3 Attack Model
5 INDRA Framework Overview
5.1 Recurrent Autoencoder
5.1.1 Model Architecture
5.1.2 Training Process
5.2 Inference and Detection
6 Experiments
6.1 Experimental Setup
6.2 Anomaly Threshold Selection
6.3 Comparison of INDRA Variants
6.4 Comparison with Prior Works
6.5 ADS Overhead Analysis
6.6 Scalability Results
7 Conclusion
References
MELETI: A Machine-Learning-Based Embedded System Architecture for Infrastructure Inspection with UAVs
1 Introduction
2 Related Work
2.1 Power Line Infrastructure Inspection
2.2 Telecommunication Infrastructure Inspection
3 System Architecture
3.1 Key Performance Indicators (KPIs) Definition
3.2 Data Acquisition
3.3 Detection Algorithm
3.3.1 Performance Evaluation
3.3.2 Key Outcomes
3.3.3 Analysis
4 Application Examples
4.1 Experimental Equipment
4.2 Use Case: Telecommunication Infrastructure
4.2.1 Corrosion Detection
4.2.2 Antenna Tilting
4.2.3 Damage Control
4.2.4 Fire Prevention with Multispectral Imaging
4.3 Use Case: Power Infrastructure
4.3.1 Pole Detection with Position Correction
5 Conclusions and Future Work
References
Part III Security, Privacy and Robustness for Embedded Machine Learning
On the Vulnerability of Deep Reinforcement Learning to Backdoor Attacks in Autonomous Vehicles
1 Introduction
2 Deep Learning in Autonomous Vehicles
2.1 Deep Neural Networks in AVs
2.2 Deep Reinforcement Learning in AVs
2.2.1 The Reinforcement Learning Objective
2.2.2 DRL in AVs
3 Backdoor Attacks
3.1 Backdoor Attacks in Classification Problems
3.2 Backdoor Attacks in DRL
3.3 Backdoor Attacks in DRL-Based AV Controller
4 Backdoor Defenses
4.1 Analysis of Backdoor Defenses for DRL-Based Traffic Controller Attacks
5 Conclusion
References
Secure Indoor Localization on Embedded Devices with Machine Learning
1 Introduction
2 Background and Related Work
2.1 Received Signal Strength Indicator (RSSI)
2.2 Fingerprint-Based Indoor Localization
2.3 Challenges with Indoor Localization
3 CNNLOC Framework Overview
3.1 Convolutional Neural Networks
3.2 Indoor Localization with CNNLOC
4 Localization Security Analysis
5 Problem Formulation
6 S-CNNLOC Framework
6.1 Offline Fingerprint Database Extrapolation
6.2 Inducing Malicious Behavior
7 Experiments
7.1 Experimental Setup
7.2 Experimental Results
7.2.1 Analysis of Noise Induction Aggressiveness
7.2.2 Comparison of Attack Vulnerability
7.2.3 Extended Analysis on Additional Benchmark Paths
7.2.4 Generality of Proposed Approach
7.2.5 Denoising Autoencoder-Based DNN Framework
7.2.6 Security Aware DNN Training in the Offline Phase
8 Conclusions and Future Work
References
Considering the Impact of Noise on Machine Learning Accuracy
1 Introduction
2 Studying the Impact of Noise: A Brief Overview of the Existing Literature
2.1 Noise Generation
2.2 Formal Analysis
2.2.1 Linear Programming
2.2.2 Satisfiability Solving
2.2.3 Model Checking
2.2.4 Limitations in the Existing Literature
3 Effects of Noise on Machine Learning Accuracy
3.1 Decreasing Robustness
3.2 Noise Tolerance
3.3 Aggravating Bias
3.4 Varying Sensitivity Across Input Nodes
4 Modeling Noise
4.1 Lp Norms
4.1.1 L1 Norm (Manhattan Distance)
4.1.2 L2 Norm (Euclidean Distance)
4.1.3 L∞ Norm
4.2 Relative Noise
5 Case Study
5.1 FANNet: Formal Analysis of Neural Networks
5.2 Experimental Setup
5.2.1 Dataset
5.2.2 Neural Network
5.3 Results and Discussion
5.3.1 Robustness and Tolerance
5.3.2 Bias
5.3.3 Node Sensitivity
5.3.4 Discussion
6 Conclusion
References
Mitigating Backdoor Attacks on Deep Neural Networks
1 Background
2 Literature Survey
3 Preliminaries
4 Problem Description
5 Backdoor Defense by Training Attacker Imitator
5.1 Problem Formulation
5.2 Defense Methodology
5.3 Experimental Setup
5.3.1 Badnet-CW
5.3.2 Badnet-GY
5.3.3 Badnet-YS
5.3.4 Badnet-CR
5.4 Experimental Results
5.4.1 Attacker Imitator Configuration
5.4.2 BadNet vs. Benign Network Detection
5.4.3 Fine-Tuning the Network
6 RAID—An On-line Detection Method
6.1 Problem Formulation
6.2 Detection Algorithm
6.3 Experimental Setup
6.3.1 BadNet Trained on MNIST
6.3.2 BadNet Trained on GTSRB
6.3.3 BadNet Trained on CIFAR-10
6.3.4 BadNet Trained on YouTube Face
6.3.5 BadNet Trained on ImageNet
6.3.6 Hyperparameter Setting
6.4 Experimental Results
6.4.1 Performance of N and Gn
6.4.2 Performance of g(·)
6.4.3 Performance of the Anomaly Detector
6.4.4 Multiple Triggers and Adaptive Attacks
6.4.5 Experiments on Hyperparameters
6.4.6 More Advanced Attack
7 Benign Applications of the Backdoor Phenomena
8 Future Directions
References
Robustness for Embedded Machine Learning Using In-Memory Computing
1 Introduction
2 Background
2.1 Adversarial Attacks
2.2 Memristive Crossbars and Their Non-idealities and Non-linearities
3 Non-linearity Aware Training (NEAT): Mitigating the Impact of Crossbar Non-idealities and Non-linearities for Robust DNN Implementations
4 DetectX: Improving the Robustness of DNNs Using Hardware Signatures in Memristive Crossbar Arrays
5 Unleashing Robustness to Structure-Pruned DNNs Implemented on Crossbars with Non-idealities
5.1 Hardware Evaluation Framework for Non-ideality Integration During Inference
5.2 Are Structure-Pruned DNNs Also Robust on Hardware?
5.3 Non-ideality Mitigation Strategies for Increased Robustness of Structure-Pruned DNNs
6 Related Works
7 Conclusion
References
Adversarial ML for DNNs, CapsNets, and SNNs at the Edge
1 Introduction
2 Security Challenges for ML
2.1 ML Privacy
2.2 Fault Injection and Hardware Trojans on ML Systems
2.3 ML Systems Reliability Threats
3 Taxonomy of Adversarial ML
4 Security for DNNs
4.1 Adversarial Attacks
4.2 Adversarial Defenses
5 Security for Capsule Networks
5.1 Robustness Against Affine Transformations
5.2 Robustness Against Adversarial Attacks
5.2.1 Adversarial Attack Methodology
5.2.2 Evaluation Results
5.3 Discussion
6 Security for Spiking Neural Networks
6.1 Comparison DNNs vs. SNNs
6.2 Improving the SNN Robustness Through Inherent Structural Parameters
6.2.1 SNN Robustness Exploration Methodology
6.2.2 SNN Robustness Evaluation
6.3 Adversarial Attacks and Defenses on Event-Based Data
6.3.1 Gradient-Based Attack for Event Sequences
6.3.2 Background Activity Filter for Event Cameras
6.3.3 Evaluation of Gradient-Based Attack and Background Activity Filter
6.3.4 Dash Attack for Event Sequences
6.3.5 Mask Filter for Event Cameras
6.3.6 Evaluation of the Dash Attack Against Background Activity Filter and Mask Filter
6.3.7 Mask Filter-Aware Dash Attack for Event Sequences
6.3.8 Evaluation of the Mask Filter-Aware Dash Attack Against Background Activity Filter and Mask Filter
7 Conclusion
References
On the Challenge of Hardware Errors, Adversarial Attacks and Privacy Leakage for Embedded Machine Learning
1 Introduction
2 ML Robustness to Errors
2.1 Methodology
2.2 Results
3 ML Security
3.1 Adversarial Attacks
3.1.1 Defenses Against Adversarial Attacks
3.2 Embedded Systems-Friendly Defenses
3.2.1 Defensive Approximation
3.2.2 Undervolting as a Defense
3.3 Privacy
4 Conclusion
References
A Systematic Evaluation of Backdoor Attacks in Various Domains
1 Introduction
2 Background
2.1 Computer Vision
2.2 Natural Language Processing
2.3 Graph Data
2.4 Backdoor Attacks
2.4.1 Metrics
3 Methodology
3.1 Threat Model
3.2 Image Classification
3.3 Natural Language Processing
3.4 Speech Recognition
3.5 Graph Data
4 Experimental Results
4.1 Image Classification
4.2 Natural Language Processing
4.3 Speech Recognition
4.4 Graph Data
4.5 General Observations
5 Conclusions
References
Deep Learning Reliability: Towards Mitigating Reliability Threats in Deep Learning Systems by Exploiting Intrinsic Characteristics of DNNs
1 Introduction
2 Preliminaries
2.1 Deep Neural Networks
2.2 DNN Hardware Accelerators
3 Reliable Deep Learning
3.1 A Systematic Methodology for Building Reliable DNN Systems
3.2 Resilience of DNNs to Reliability Threats
3.2.1 Resilience of DNNs to Permanent Faults
3.2.2 Resilience of DNNs to Timing Faults
3.2.3 Resilience of DNNs to Memory Faults
3.3 Permanent Fault Mitigation
3.4 Timing Error Mitigation
3.5 Soft Error Mitigation
4 Conclusion
References
Index