Recurrent Neural Networks: Concepts and Applications

✍ Scribed by Ajith Abraham (editor), Amit Kumar Tyagi (editor)

• Publisher: CRC Pr I Llc
• Year: 2022
• Tongue: English
• Leaves: 413
• Category: Library

✦ Synopsis

The text discusses recurrent neural networks for prediction and offers new insights into the learning algorithms, architectures, and stability of recurrent neural networks. It discusses important topics including recurrent and folding networks, long short-term memory (LSTM) networks, gated recurrent unit neural networks, language modeling, neural network model, activation function, feed-forward network, learning algorithm, neural turning machines, and approximation ability. The text discusses diverse applications in areas including air pollutant modeling and prediction, attractor discovery and chaos, ECG signal processing, and speech processing. Case studies are interspersed throughout the book for better understanding.

FEATURES

• Covers computational analysis and understanding of natural languages

• Discusses applications of recurrent neural network in e-Healthcare

• Provides case studies in every chapter with respect to real-world scenarios

• Examines open issues with natural language, health care, multimedia (Audio/Video), transportation, stock market, and logistics

The text is primarily written for undergraduate and graduate students, researchers, and industry professionals in the fields of electrical, electronics and communication, and computer engineering/information technology.

✦ Table of Contents

Section I: Introduction
Chapter 1: A Road Map to Artificial Neural Network
1.1 Introduction
1.2 Biological Inspiration of Artificial Neural Network
1.3 The Architecture of Artificial Neural Network
1.4 Activation Functions for Artificial Neural Network
1.5 Types of Artificial Neural Network
1.6 Training Algorithms for Artificial Neural Network
1.7 Applications of Artificial Neural Network
1.8 Conclusion
References
Chapter 2: Applications of Recurrent Neural Network: Overview and Case Studies
2.1 Introduction
2.1.1 EEG Signal Analysis on Seizure Detection
2.1.1.1 Types of Recurrent Neural Networks
2.1.1.2 EEG Dataset
2.1.1.3 Methodology Implemented
2.1.1.4 Result
2.1.2 Recognizing Textual Entailment
2.1.2.1 Architecture
2.1.2.1.1 Decomposable Attention Model
2.1.2.1.2 Asymmetric Attention Model
2.1.2.2 LSTM
2.1.2.3 GRU
2.1.2.4 Evaluation
2.1.3 Dataset
2.1.4 Evaluation of Models
2.1.5 Analysis of Sentences
2.1.5.1 General Sentences
2.1.5.2 Active-Passive Sentences
2.1.6 Inference
2.2 Conclusion
2.2.1 EEG Analysis
2.2.2 RTE Analysis
References
Chapter 3: Image to Text Processing Using Convolution Neural Networks
3.1 Introduction
3.1.1 Convolutional Neural Networks
3.2 Literature Survey
3.3 Methodology
3.3.1 Recurrent Neural Networks
3.4 Implementation
3.5 Results and Discussion
3.6 Conclusion
References
Chapter 4: Fuzzy Orienteering Problem Using Genetic Search
4.1 Introduction
4.2 Chance-Constrained Programming
4.3 The Orienteering Problem
4.3.1 Deterministic Model
4.3.2 Fuzzy Model
4.4 The Proposed Method
4.4.1 Encoding Scheme
4.4.2 Fitness Function
4.4.3 Selection
4.4.4 Crossover
4.4.5 Mutation
4.4.6 Probability of Crossover and Mutation
4.4.7 The Proposed Genetic Algorithm
4.5 Result and Discussion
4.5.1 Data
4.5.2 Parameters of Proposed Algorithm
4.5.3 Results
4.6 Conclusion and Future Scope
References
Chapter 5: A Comparative Analysis of Stock Value Prediction Using Machine Learning Technique
5.1 Introduction
5.1.1 Stock Market
5.1.2 Deep Learning
5.1.3 Objective
5.2 Literature Review
5.3 Methodology and Analysis
5.3.1 Deep Neural Network
5.3.1.1 Input Layer
5.3.1.2 Hidden Layers
5.3.1.3 Activation Function
5.3.1.4 Neuron Weights
5.3.1.5 Output Layer
5.3.2 Recurrent Neural Network (RNN)
5.3.2.1 Back-Propagation through Time
5.3.2.2 Problems in RNN
5.3.3 Long Short-Term Memory Models (LSTM)
5.3.4 Activation Functions
5.3.4.1 Rectified Linear Unit Function
5.4 Experimentation and Results
5.4.1 Data Collection
5.4.2 Data Preprocessing
5.4.3 Analysis of Various Models on Stock Data
5.4.3.1 DNN Model
5.4.3.2 RNN Model
5.4.3.3 LSTM Model
5.5 Conclusion
References
Section II: Process and Methods
Chapter 6: Developing Hybrid Machine Learning Techniques to Forecast the Water Quality Index (DWM-Bat & DMARS)
6.1 Introduction
6.2 Literature Survey
6.3 Building IM12CP-WQI
6.3.1 Description of Dataset
6.3.2 Results of IM12CP-WQI
6.4 Conclusions and Recommendation for Future Works
References
Chapter 7: Analysis of RNNs and Different ML and DL Classifiers on Speech-Based Emotion Recognition System Using Linear and Nonlinear Features
7.1 Introduction
7.2 Methodology
7.2.1 Workflow
7.2.2 Preprocessing
7.2.2.1 Silence Removal
7.2.2.2 Zero Crossing Rate
7.2.2.3 Short Time Energy
7.2.2.4 Pre-emphasis
7.2.2.5 Framing
7.2.2.6 Windowing
7.2.3 Feature Extraction
7.2.4 Audio Features
7.2.4.1 Chromagram
7.2.4.2 Mel-Frequency Cepstrum (MFC)
7.2.4.3 Mel-Frequency Cepstrum Coefficients (MFCC)
7.3 Classification Models
7.3.1 MLP Classifier
7.3.2 SVC
7.3.3 Random Forest Classifier
7.3.4 Gradient-Boosting Classifier
7.3.5 K-Neighbors Classifier
7.3.6 Recurrent Neural Networks
7.3.7 Bagging Classifier
7.4 Experimentation
7.4.1 Dataset Description
7.4.1.1 EMODB
7.4.1.2 RAVDESS
7.4.2 Training Process
7.5 Results
7.5.1 Recurrent Neural Network (RNN)
7.6 Conclusion
7.7 Future Directions
References
Chapter 8: Web Service User Diagnostics with Deep Learning Architectures
8.1 Introduction
8.2 Convolution Neural Networks
8.3 Recurrent Neural Networks
8.4 Importance of Deep Learning versus Machine Learning
8.5 Feature Extraction and Feature Engineering
8.6 Model Representation and Generation
8.7 Related Work
8.7.1 Deep Learning Architectures
8.8 Deep Learning and Web Services
8.9 Deep Learning Performance Evaluation
8.10 CNN and Web Service Diagnostics
8.11 Convolution Neural Network
8.12 Recurrent Neural Network and Web Services
8.12.1 Recurrent Neural Network
8.13 Long Short-Term Memory and Web Service State Diagnostics
8.13.1 Long Short-Term Memory
8.14 Gated Recurrent Units and Web Service State Diagnostics
8.15 Dataset
8.16 Results and Discussion
8.16.1 Convolution Neural Network
8.16.2 Recurrent Neural Network
8.16.3 Comparison of CNN, LSTM, GRU, RNN
8.17 Summary
References
Chapter 9: D-SegNet: A Modified Encoder-Decoder Approach for Pixel-Wise Classification of Brain Tumor from MRI Images
9.1 Introduction
9.2 Literature Review
9.3 System Model
9.3.1 Database
9.3.2 Data Preprocessing
9.3.3 Patch Extraction
9.3.4 Encoder
9.3.5 Feature Fusion
9.3.6 Decoder
9.3.7 Training
9.3.8 Evaluation
9.4 Results and Discussion
9.5 Conclusion
References
Chapter 10: Data Analytics for Intrusion Detection System Based on Recurrent Neural Network and Supervised Machine Learning Methods
10.1 Introduction
10.2 Related Work
10.3 Proposed System
10.3.1 Methodology
10.3.2 Description of the Dataset
10.3.3 Particle Swarm Optimization
10.3.4 Recurrent Neural Network
10.3.5 Extra Tree
10.3.6 Cat Boost
10.3.7 Random Forest
10.3.8 Gradient Boosting
10.4 Results and Discussion
10.4.1 Experimental Findings of PSO-RNN
10.4.2 Experimental Findings of PSO-Extra Tree, PSO-Cat Boost, PSO-RF, PSO-GB
10.4.3 Comparison with Existing Studies Reported in the Literature
10.5 Conclusion and Future Work
References
Section III: Applications
Chapter 11: Triple Steps for Verifying Chemical Reaction Based on Deep Whale Optimization Algorithm (VCR-WOA)
11.1 Introduction
11.2 Main Concepts Related to Problem
11.2.1 Tokenization Process
11.2.2 Coding
11.2.3 Selection Algorithms
11.2.4 Optimization
11.3 Building VCR-WOA
11.3.1 The VCR-WOA Design Stages
11.3.1.1 Preprocessing Stage
11.3.1.2 Building VCR-WOA Predictor
11.3.1.3 Evaluation Stage
11.4 Implementation and Results of VCR-WOA
11.4.1 Description of Database
11.4.2 Coding Elements of Periodic Table
11.4.3 Tokenization
11.4.4 Applying Whale Optimization Steps
11.4.5 Evaluation the Results
11.5 Conclusion and Future Works
References
Chapter 12: Structural Health Monitoring of Existing Building Structures for Creating Green Smart Cities Using Deep Learning
12.1 Introduction
12.2 Fundamental Objectives of Monitoring of Civil Structures
12.2.1 Cost-Effectiveness
12.2.2 Detecting Early Risk
12.2.3 Improved Public Safety
12.2.4 Increased Life Span of Structure
12.3 Artificial Intelligence
12.3.1 Machine Learning
12.3.2 Deep Learning
12.4 Artificial Intelligence in Civil Engineering
12.4.1 Artificial Intelligence in SHM
12.4.1.1 Image Binarization (IB)
12.4.1.2 Threshold Values
12.4.1.3 Concerns in Crack-Sensing Images
12.5 Case Study of an Unoccupied Building at CSIR-CBRI Campus
12.5.1 Results of Cracks Identification from the Proposed Images in MATLAB
12.6 Conclusion
Abbreviation Used
References
Chapter 13: Artificial Intelligence–Based Mobile Bill Payment System Using Biometric Fingerprint
13.1 Overview of Payment Transactions
13.2 Literature Review
13.2.1 Current Scenario
13.2.1.1 Disadvantages of Current Payment Methods
13.2.2 Mobile Banking
13.2.3 e-Wallets
13.2.3.1 NFC Chips
13.2.3.2 Face Recognition
13.2.4 Biometrics Method
13.2.4.1 QR Code Method
13.2.4.2 IoT Method
13.2.4.3 Fingerprint Method
13.2.4.4 E-Cash Transaction
13.2.4.5 Merits and Demerits of Biometric
13.3 Bill Payment System Using Biometric Fingerprint
13.3.1 Architecture Description
13.3.2 Minutiae Extraction and Comparison Algorithm
13.3.2.1 Fingerprint Acquisition
13.3.2.2 Fingerprint Preprocessing
13.3.2.3 Fingerprint Enhancement
13.3.2.4 Feature Extraction
13.3.2.5 Minutiae Matching
13.4 Implementation
13.5 Experimental Results and Discussion
13.6 Conclusion and Future Scope
References
Chapter 14: An Efficient Transfer Learning–Based CNN Multi-Label Classification and ResUNET Based Segmentation of Brain Tumor in MRI
14.1 Introduction
14.2 Review of Related Work
14.3 Dataset and Preprocessing
14.4 Methods Used
14.4.1 VGG16 Model
14.4.2 Convolutional Neural Network
14.5 Transfer Learning (TL)
14.5.1 ResUNET Model
14.6 Implementation
14.6.1 Classification Model
14.6.2 Classifier Metrics and Evaluation
14.6.3 Comparison with Related Classification Models
14.6.4 Segmentation Model
14.6.5 Segmentation Metrics and Evaluation
14.6.6 Comparison with Some Exiting Segmentation Models
14.7 Results
14.8 Conclusion
References
Chapter 15: Deep Learning–Based Financial Forecasting of NSE Using Sentiment Analysis
15.1 Introduction
15.2 Related Work
15.3 System Design
15.4 Data Collection and Processing
15.4.1 Stock Price Prediction Using Historical Data
15.4.2 Textual Data Collection for Sentiment Analysis
15.5 Evaluation Methodology
15.5.1 Accuracy
15.5.2 F1 Score
15.5.3 Mean Absolute Error
15.5.4 Root Mean Square Error
15.6 Experimental Setup
15.6.1 Stock Price Prediction
15.6.1.1 Decision Tree Regressor
15.6.1.2 Random Forest Regressor
15.6.1.3 Gradient Boosting Regressor
15.6.1.4 Long Short-Term Memory
15.6.2 Sentiment Analysis
15.6.2.1 Logistic Regression
15.6.2.2 Support Vector Machine
15.6.3 Hyperparameter Optimization
15.7 Result and Analysis
15.7.1 Feature Selection
15.7.2 Historical Data Analysis
15.7.2.1 Historical Data Analysis Using LSTM
15.8 Conclusion and Future Scope
References
Chapter 16: An Efficient Convolutional Neural Network with Image Augmentation for Cassava Leaf Disease Detection
16.1 Introduction
16.2 Related Works
16.3 Materials and Methods
16.3.1 Data Set Description
16.3.2 Methodology
16.3.2.1 Preprocessing and Augmentation
16.3.2.2 CNN Feature Extractors
16.3.2.3 EfficientB4 Transfer Learning Architecture
16.3.2.4 GPU Training
16.3.2.5 TPU Training
16.3.2.6 Parameters on GPU and TPU
16.3.2.7 Loss Functions Used
16.3.2.8 Optimizer
16.3.2.9 Learning Rate Function
16.4 Experiment Results and Discussion
16.4.1 Performance Analysis
16.5 Conclusion
References
Section IV: Post–COVID-19 Futuristic Scenarios–Based Applications: Issues and Challenges
Chapter 17: AI-Based Classification and Detection of COVID-19 on Medical Images Using Deep Learning
17.1 Introduction
17.2 Literature Survey
17.2.1 Methodology
17.3 Proposed Model
17.3.1 Dataset Description
17.4 Result and Discussion
17.5 Conclusion
References
Chapter 18: An Innovative Electronic Sterilization System (S-Vehicle, NaOCI.5H 2 O and CeO 2 NP)
18.1 Introduction
18.2 Related Works
18.3 Main Tools and Materials
18.3.1 Platform Waspmote
18.3.2 LoRa Modem
18.3.3 Autonomous Vehicles (AVs)
18.3.4 GPS Module
18.3.5 Control System
18.3.6 Arduino UNO
18.3.7 Arduino Uno components
18.4 Designed System
18.5 Applications
18.6 Conclusions
Declarations
References
Chapter 19: Comparative Forecasts of Confirmed COVID-19 Cases in Botswana Using Box-Jenkin’s ARIMA and Exponential Smoothing State-Space Models
19.1 Introduction
19.2 Literature Review
19.3 Methodology
19.3.1 Dataset Description
19.3.2 Time Series Analysis
19.3.3 ARIMA Algorithm
19.3.4 Exponential Smoothing Algorithm
19.3.5 Testing for Goodness of Fit
19.3.6 Models Prediction Accuracy Measurement
19.3.7 Models Forecast Accuracy Measurement
19.4 Results and Discussions
19.4.1 Visualization of Time Series
19.4.2 ARIMA Modeling
19.4.2.1 Forecasting with the Best ARIMA Model
19.4.3 ETS Modeling
19.4.3.1 Forecasting with the Best ETS Model
19.4.4 Comparison of ARIMA and ETS Models
19.4.5 Summary on the Best-Performing Model
19.5 Conclusion and Future Works
References
Chapter 20: Recent Advancement in Deep Learning: Open Issues, Challenges, and a Way Forward
20.1 Introduction
20.2 Evolution
20.3 Motivation
20.4 Popular Applications Using Natural Language Processing (NLP)
20.4.1 Deep Learning Libraries and Framework
20.4.1.1 Tensor Flow
20.4.1.2 PyTorch
20.4.1.3 Keras
20.4.1.4 Sonnet
20.4.1.5 MXnet
20.4.1.6 DL4j (Deep Learning for Java)
20.5 Benefits and Pitfalls of Existing Algorithms
20.6 Related Work
20.7 Conclusion
Appendix A
References
Index