Deep Learning Applications, Volume 3 (Advances in Intelligent Systems and Computing)

✍ Scribed by M. Arif Wani (editor), Bhiksha Raj (editor), Feng Luo (editor), Dejing Dou (editor)

• Publisher: Springer
• Year: 2021
• Tongue: English
• Leaves: 328
• Edition: 1st ed. 2022
• Category: Library

✦ Synopsis

This book presents a compilation of extended version of selected papers from the 19th IEEE International Conference on Machine Learning and Applications (IEEE ICMLA 2020) and focuses on deep learning networks in applications such as pneumonia detection in chest X-ray images, object detection and classification, RGB and depth image fusion, NLP tasks, dimensionality estimation, time series forecasting, building electric power grid for controllable energy resources, guiding charities in maximizing donations, and robotic control in industrial environments. Novel ways of using convolutional neural networks, recurrent neural network, autoencoder, deep evidential active learning, deep rapid class augmentation techniques, BERT models, multi-task learning networks, model compression and acceleration techniques, and conditional Feature Augmented and Transformed GAN (cFAT-GAN)  for the above applications are covered in this book. Readers will find insights to help them realize novel ways of using deep learning architectures and algorithms in real-world applications and contexts, making the book an essential reference guide for academic researchers, professionals, software engineers in the industry, and innovative product developers.

 

✦ Table of Contents

Preface
Contents
About the Editors
Deep Rapid Class Augmentation; A New Progressive Learning Approach that Eliminates the Issue of Catastrophic Forgetting
1 Introduction
1.1 Sequential Learning and the Problem of Catastrophic Forgetting
1.2 Proposed Solution to Catastrophic Forgetting in Sequential Learning Applications
2 Background on Progressive Gradient Descent Optimization
2.1 SGD Implementation
3 Illustration of Catastrophic Forgetting
3.1 The Sub-Optimality of Freezing the Old Classifier Weights
4 Related Work
4.1 Alternative Progressive Learning Approaches
4.2 Progressive Learning Approaches Related to Deep RCA
5 Deep Rapid Class Augmentation
5.1 Background on Recursive Least Squares
5.2 Adapting RLS to Model Augmentation
5.3 RCA Algorithm Implementation
6 Illustration of RCA’s Elimination of Catastrophic Forgetting
7 Progressive Learning on MNIST Data
7.1 Progressive SGD on MNIST DATA
7.2 Deep RCA Augmentation on MNIST Data
8 Deep RCA Augmentation on ImageNet
8.1 Augmentation of a New “Cat” Class
8.2 Augmentation Accuracy Comparison
9 Deep RCA on CIFAR 100 Data
9.1 Augmentation Timing Comparisons
9.2 Assessing Augmentation Time as a Function of Augmented Model Class Size
9.3 The Impact of Random New Class Weight Initialization
10 Summary
References
A Comprehensive Analysis of Subword Contextual Embeddings for Languages with Rich Morphology
1 Introduction
2 Related Work
3 Methodology
3.1 Word2Vec
3.2 FastText
3.3 Contextual Subword Embeddings
3.4 Input Layer
3.5 Highway LSTM
3.6 Dependency Parser
3.7 Named Entity Recognizer
3.8 Multi-task Learning Framework
3.9 Sentiment Analyzer
4 Experimental Settings
4.1 Datasets
4.2 Training Details
5 Results
5.1 Comparison of Embedding Types
5.2 Comparing Dependency on External Features
5.3 Analysis of Training Behavior
5.4 Effects of Multi-task Learning
5.5 Error Analysis for Monolingual BERT
5.6 Comparison with Related Work
5.7 Case Study: Sentiment Analysis
6 Guidelines
7 Conclusion
References
RGB and Depth Image Fusion for Object Detection Using Deep Learning
1 Introduction
2 Related Work
2.1 Deep Learning for Object Detection
2.2 Deep Learning for Fusion
2.3 Deep Learning for Depth Estimation
3 The Proposed Early Fusion Architecture
4 Depth Estimation Module
4.1 GeoNet
4.2 MonoDepth
4.3 SfMLearner
4.4 DF-Net
5 Detection Module
6 Experimental Setup
6.1 Datasets
6.2 Implementation Details
7 Experimental Results
7.1 Multi-modal Architectures Versus Uni-Modal Architectures
7.2 Qualitative Results
8 Conclusion and Future Work
References
Dimension Estimation Using Autoencoders and Application
1 Introduction
1.1 Contributions and Organization
2 Dimension Reduction
2.1 Linear Dimension Reduction Techniques
2.2 Example of DE: MNIST Hand-Written Digits
2.3 Autoencoders
3 Autoencoder—Singular Value Proxies
3.1 L1 Hidden Layer Regularizer
3.2 Latent Space Sorting
4 Dimensionality Estimation Algorithm
5 Experiments
5.1 Estimate Dimension of Synthetic Data
5.2 Estimate Dimension of Random Synthetic Polynomial
5.3 Estimate MNIST Digit Dimension
5.4 Estimate S&P 500 Dimension
5.5 Estimate Network Dimensionality
6 Conclusion
References
A New Clustering-Based Technique for the Acceleration of Deep Convolutional Networks
1 Introduction
2 Relevant Bibliography
3 Problem Formulation
4 Dictionary-Learning-Based Weight Clustering
4.1 Proposed Approximation
4.2 Computational Complexity Analysis
4.3 Proposed Algorithm
4.4 Initial Solution and Parameter Selection
5 Experimental Results
5.1 Experiment I. Quantization Error in Individual Layers
5.2 Experiment II. Accuracy Loss on Image Classification
5.3 Experiment III. MCA Performance on Object Detection
6 Conclusions
References
Deep Learning-Based Time Series Forecasting
1 Introduction
2 Problem Formalization and Settings
3 Forecasting Techniques
3.1 Classical Machine learning techniques
3.2 Deep Learning Techniques
4 Evaluation
4.1 Evaluation Metrics
4.2 Evaluation Datasets
4.3 Evaluation Results and Analysis
5 How to Use the Code
6 Conclusion and Future Directions
References
DEAL: Deep Evidential Active Learning for Image Classification
1 Introduction
2 Related Work and Positioning
3 Methodology
3.1 Theory of Evidence
3.2 Uncertainty-Based AL
4 Experiments
4.1 Implementation Details
4.2 Experimental Results
5 Conclusion
References
LB-CNN: Convolutional Neural Network with Latent Binarization for Large Scale Multi-class Classification
1 Introduction
2 Related Work
3 Proposed Method
3.1 Manifest Model
3.2 Latent Model
3.3 EM Algorithm
3.4 LB-CNN with Gibbs Sampling
4 Experiments
4.1 Description of Experiments
4.2 LB-CNN: Performance
4.3 LB-CNN Enhanced Interpretability
5 Conclusion
References
Efficient Deployment of Deep Learning Models on Autonomous Robots in the ROS Environment
1 Introduction
2 Related Work
3 Deep Learning Background
3.1 CNNs
3.2 Training
3.3 Inference
4 Deploying DL Models on Resource-Constrained Devices
4.1 EdgeLite Architecture
5 Architecture of EasyDLROS
5.1 ROS
5.2 EasyDLROS Framework
6 Dataset
7 How to Use the Code
8 Experiments
8.1 Experiments on Resource-Constrained Linux Environment Without ROS
8.2 Experiments on the ROS Environment
9 Conclusion
References
cFAT-GAN: Conditional Simulation of Electron–Proton Scattering Events with Variate Beam Energies by a Feature Augmented and Transformed Generative Adversarial Network
1 Introduction
2 Related Work
3 Methods
3.1 Feature-Augmented and Transformed GAN
3.2 cFAT-GAN Architecture
3.3 Beam Energy Representation
4 Results
4.1 Event Feature Distributions
4.2 Derived Physical Quantities
4.3 Latent Variable Analysis
5 Conclusion
6 How to Use the Code
References
Building Power Grid 2.0: Deep Learning and Federated Computations for Energy Decarbonization and Edge Resilience
1 Introduction
1.1 The Need for Power Grid Resiliency: ``A Song of Ice and Fire''
1.2 The Solution: Federated Computations and Deep Learning
2 Background: Grid Analytics
3 Future Plug-and-Play Grid
4 Identification and Coordination of Underutilized Energy Flexibility at the Grid Edge
4.1 Identifying Patterns at the Edge
4.2 Coordination Challenges
4.3 Energy Management in DER Networks
4.4 Inter-Agent Analytics
5 Physical Implementation
5.1 Testbed
5.2 Industry
6 Case Study 1: Load Forecasting Using Neural Networks
6.1 Approach
6.2 Results
7 Case Study 2: Coordinating Refrigeration Defrost
7.1 Asset Modeling
7.2 Building Load Prediction: Algorithm and Inputs
7.3 Results
8 Conclusions
References
Deep Learning the Donor Journey with Convolutional and Recurrent Neural Networks
1 Introduction
2 Related Research
2.1 Donation Predictions
2.2 Deep Learning
3 Problem Formulation
4 Our Approach
5 Empirical Evaluation
5.1 Adding Constituent Features
5.2 Varying Window Size
5.3 Investigating the No Action Effect
5.4 Combining Data
5.5 Summary of Best Results
5.6 Email Optimization
6 Discussion and Future Work
References
Author Index