Deep Learning Foundations

✍ Scribed by Taeho Jo

• Publisher: Springer
• Year: 2023
• Tongue: English
• Leaves: 433
• Edition: 1st ed. 2023
• Category: Library

✦ Synopsis

This book provides a conceptual understanding of deep learning algorithms. The book consists of the four parts: foundations, deep machine learning, deep neural networks, and textual deep learning. The first part provides traditional supervised learning, traditional unsupervised learning, and ensemble learning, as the preparation for studying deep learning algorithms. The second part deals with modification of existing machine learning algorithms into deep learning algorithms. The book’s third part deals with deep neural networks, such as Multiple Perceptron, Recurrent Networks, Restricted Boltzmann Machine, and Convolutionary Neural Networks. The last part provides deep learning techniques that are specialized for text mining tasks. The book is relevant for researchers, academics, students, and professionals in machine learning.

✦ Table of Contents

Preface
Part I: Foundation
Part II: Deep Machine Learning
Part III: Deep Neural Networks
Part IV: Textual Deep Learning
Contents
Part I Foundation
1 Introduction
1.1 Definition of Deep Learning
1.2 Swallow Learning
1.2.1 Supervised Learning
1.2.2 Unsupervised Learning
1.2.3 Semi-supervised Learning
1.2.4 Reinforcement Learning
1.3 Deep Supervised Learning
1.3.1 Input Encoding
1.3.2 Output Encoding
1.3.3 Unsupervised Layer
1.3.4 Convolution
1.4 Advanced Learning Types
1.4.1 Ensemble Learning
1.4.2 Local Learning
1.4.3 Kernel-Based Learning
1.4.4 Incremental Learning
1.5 Summary and Further Discussions
References
2 Supervised Learning
2.1 Introduction
2.2 Simple Supervised Learning Algorithms
2.2.1 Rule-Based Approach
2.2.2 Naive Retrieval
2.2.3 Data Similarity
2.2.4 One Nearest Neighbor
2.3 Neural Networks
2.3.1 Artificial Neuron
2.3.2 Activation Functions
2.3.3 Neural Connection
2.3.4 Perceptron
2.4 Advanced Supervised Learning Algorithms
2.4.1 Naive Bayes
2.4.2 Decision Tree
2.4.3 Random Forest
2.4.4 Support Vector Machine
2.5 Summary and Further Discussions
References
3 Unsupervised Learning
3.1 Introduction
3.2 Simple Unsupervised Learning Algorithms
3.2.1 AHC Algorithm
3.2.2 Divisive Algorithm
3.2.3 Online Linear Clustering Algorithm
3.2.4 K Means Algorithm
3.3 Kohonen Networks
3.3.1 Initial Version
3.3.2 Learning Vector Quantization
3.3.3 Semi-supervised Model
3.3.4 Self-Organizing Map
3.4 EM Algorithm
3.4.1 Cluster Distributions
3.4.2 Notations
3.4.3 E-Step
3.4.4 M-Step
3.5 Summary and Further Discussions
Reference
4 Ensemble Learning
4.1 Introduction
4.2 Partition
4.2.1 Training Set
4.2.2 Attribute Set
4.2.3 Array Partition
4.2.4 Partition Schemes
4.3 Supervised Combination Schemes
4.3.1 Voting
4.3.2 Expert Gate
4.3.3 Cascading
4.3.4 Cellular Learning
4.4 Multiple Viewed Learning
4.4.1 Views
4.4.2 Multiple Encodings
4.4.3 Multiple Viewed Supervised Learning
4.4.4 Multiple Viewed Unsupervised Learning
4.5 Summary and Further Discussions
Part II Deep Machine Learning
5 Deep KNN Algorithm
5.1 Introduction
5.2 Swallow Version
5.2.1 KNN Algorithm
5.2.2 KNN Variants
5.2.3 Trainable KNN Algorithm
5.2.4 Radius Nearest Neighbor
5.3 Basic Deep Versions
5.3.1 Feature Reduction
5.3.2 Kernel-Based KNN Algorithm
5.3.3 Output Decoded KNN
5.3.4 Pooled KNN
5.4 Advanced Deep Versions
5.4.1 Unsupervised Layer
5.4.2 Unsupervised KNN
5.4.3 Stacked KNN
5.4.4 Convolutional KNN Algorithm
5.5 Summary and Further Discussions
Reference
6 Deep Probabilistic Learning
6.1 Introduction
6.2 Swallow Version
6.2.1 Normal Distribution
6.2.2 Bayes Classifier
6.2.3 Naive Bayes
6.2.4 Bayesian Networks
6.3 Basic Deep Versions
6.3.1 Kernel-Based Bayes Classifier
6.3.2 Pooling-Based Bayes Classifier
6.3.3 Output Decoded Naive Bayes
6.3.4 Pooled Naive Bayes
6.4 Advanced Deep Versions
6.4.1 Unsupervised Bayes Classifier
6.4.2 Unsupervised Naive Bayes
6.4.3 Stacked Bayes Classifier
6.4.4 Stacked Bayes Classifier
6.5 Summary and Further Discussions
Reference
7 Deep Decision Tree
7.1 Introduction
7.2 Swallow Version
7.2.1 Graphical View
7.2.2 Classification Process
7.2.3 Root Node Selection
7.2.4 Learning Process
7.3 Basic Deep Versions
7.3.1 Random Forest
7.3.2 Clustering-Based Multiple Decision Trees
7.3.3 Output-Decoded Decision Tree
7.3.4 Pooled Naive Bayes
7.4 Advanced Deep Versions
7.4.1 Unsupervised Decision Tree
7.4.2 Stacked Decision Tree
7.4.3 Unsupervised Random Forest
7.4.4 Stacked Random Forest
7.5 Summary and Further Discussions
Reference
8 Deep Linear Classifier
8.1 Introduction
8.2 Support Vector Machine
8.2.1 Linear Classifier
8.2.2 Kernel Function
8.2.3 SVM Classifier
8.2.4 Dual Constraints
8.3 Basic Deep Versions
8.3.1 SVM as Deep Learning Algorithm
8.3.2 Multiple Kernel-Based SVM
8.3.3 SVM for Multiple Classification
8.3.4 Pooled SVM
8.4 Advanced Deep Versions
8.4.1 Unsupervised Linear Classifier
8.4.2 Stacked Linear Classifier
8.4.3 Unsupervised SVM
8.4.4 Stacked SVM
8.5 Summary and Further Discussions
Part III Deep Neural Networks
9 Multiple Layer Perceptron
9.1 Introduction
9.2 Perceptron
9.2.1 Architecture
9.2.2 Classification Process
9.2.3 Learning Process
9.2.4 Perceptron for Regression
9.3 Multiple Layer Perceptrons
9.3.1 Architecture
9.3.2 Input Layer
9.3.3 Hidden Layer
9.3.4 Output Layer
9.4 Learning Process
9.4.1 Weight Update Between Hidden Layer and Output Layer
9.4.2 Weight Update Between Input Layer and Hidden Layer
9.4.3 Entire Learning Process
9.4.4 Stochastic Gradient Descent
9.5 Summary and Further Discussions
Reference
10 Recurrent Neural Networks
10.1 Introduction
10.2 Recurrent Architecture
10.2.1 Forward Connection
10.2.2 Recurrent Connection
10.2.3 Hybrid Architecture
10.2.4 Hidden Recurrency
10.3 Recurrent Neural Networks
10.3.1 Basic Recurrent Neural Networks
10.3.2 RNN Variants
10.3.3 LSTM (Long Short-Term Memory)
10.3.4 LSTM Variants
10.4 Applications
10.4.1 Time Series Prediction
10.4.2 Sentimental Analysis
10.4.3 Entire Learning Process
10.4.4 Machine Translation
10.5 Summary and Further Discussions
Reference
11 Restricted Boltzmann Machine
11.1 Introduction
11.2 Associative Memory
11.2.1 Input Restoration
11.2.2 Associative MLP
11.2.3 Hopfield Networks
11.2.4 Boltzmann Machine
11.3 Single RBM
11.3.1 Architecture
11.3.2 Input Layer
11.3.3 Learning Process
11.3.4 Classification Model
11.4 Stacked RBM
11.4.1 Multiple Stacked RBM
11.4.2 Input Encoding
11.4.3 Output Decoding
11.4.4 Evolutionary RBM
11.5 Summary and Further Discussions
12 Convolutional Neural Networks
12.1 Introduction
12.2 Pooling
12.2.1 Pooling Concepts
12.2.2 Pooling Types
12.2.3 Dimensionality Downsizing
12.2.4 Pooling for Ensemble Learning
12.3 Convolution
12.3.1 Tensor
12.3.2 Single-Channeled Convolution
12.3.3 Tensor Convolution
12.3.4 Convolution Variants
12.4 CNN Design
12.4.1 ReLU
12.4.2 Pooling + ReLU
12.4.3 Convolution + ReLU
12.4.4 Pooling + Convolution + ReLU
12.5 Summary and Further Discussions
Reference
Part IV Textual Deep Learning
13 Index Expansion
13.1 Introduction
13.2 Text Indexing
13.2.1 Tokenization
13.2.2 Pooling Types
13.2.3 Stop Word Removal
13.2.4 Additional Filtering
13.3 Semantic Similarity
13.3.1 Word Representation
13.3.2 Cosine Similarity
13.3.3 Euclidean Distances
13.3.4 Table Similarity
13.4 Expansion Schemes
13.4.1 Associated Words
13.4.2 Associated Text
13.4.3 Information Retrieval-Based Scheme
13.4.4 Index Optimization
13.5 Summary and Further Discussions
References
14 Text Summarization
14.1 Introduction
14.2 Abstracting
14.2.1 Phrase-Based Abstracting
14.2.2 Keyword-Based Abstracting
14.2.3 Mapping Abstracting into Binary Classification
14.2.4 Machine Learning-Based Abstracting
14.3 Query-Based Text Summarization
14.3.1 Query
14.3.2 Word-Based Summarization
14.3.3 Sentence-Based Summarization
14.3.4 ML-Based Text Summarization
14.4 Multiple Text Summarization
14.4.1 Group Cohesion
14.4.2 Keyword-Based Summarization
14.4.3 Machine Learning-Based Text Summarization
14.4.4 Textual Cluster Prototype
14.5 Summary and Further Discussions
References
15 Textual Deep Operations
15.1 Introduction
15.2 Numerical Deep Operations
15.2.1 Text Encoding
15.2.2 Convolution
15.2.3 Pooling
15.2.4 Virtual Examples
15.3 Textual Convolution
15.3.1 Raw Text Structure
15.3.2 Random Part Selection
15.3.3 Hierarchical Indexing
15.3.4 Temporal Topic Analysis
15.4 Textual Pooling
15.4.1 Text Partition
15.4.2 Sub-dimensional Down-sampling
15.4.3 Keyword Extraction
15.4.4 Text Summarization
15.5 Summary and Further Discussions
References
16 Text Classification System
16.1 Introduction
16.2 System Architecture
16.2.1 Input Layer
16.2.2 Convolution Layer
16.2.3 Pooling Layer
16.2.4 Design
16.3 Text Classification Process
16.3.1 Convolutional KNN
16.3.2 Convolutional Naive Bayes
16.3.3 Restricted Boltzmann Machine
16.3.4 Convolutional Neural Networks
16.4 Learning Process
16.4.1 Convolutional KNN
16.4.2 Convolutional Naive Bayes
16.4.3 Restricted Boltzmann Machine
16.4.4 Convolutional Neural Networks
16.5 Summary and Further Discussions
Reference
Index