Deep Learning from the Basics (2021) [Saitoh] [9781800206137]

✍ Scribed by Saitoh

• Year: 2021
• Tongue: English
• Leaves: 317
• Category: Library

✦ Table of Contents

Cover
FM
Copyright
Table of Contents
Preface
Introduction
Chapter 1: Introduction to Python
What is Python?
Installing Python
Python Versions
External Libraries That We Use
Anaconda Distribution
Python Interpreter
Mathematical Operations
Data Types
Variables
Lists
Dictionaries
Boolean
if Statements
for Statements
Functions
Python Script Files
Saving in a File
Classes
NumPy
Importing NumPy
Creating a NumPy Array
Mathematical Operations in NumPy
N-Dimensional NumPy Arrays
Broadcasting
Accessing Elements
Matplotlib
Drawing a Simple Graph
Features of pyplot
Displaying Images
Summary
Chapter 2: Perceptrons
What Is a Perceptron?
Simple Logic Circuits
AND Gate
NAND and OR gates
Implementing Perceptrons
Easy Implementation
Introducing Weights and Bias
Implementation with Weights and Bias
Limitations of Perceptrons
XOR Gate
Linear and Nonlinear
Multilayer Perceptrons
Combining the Existing Gates
Implementing an XOR Gate
From NAND to a Computer
Summary
Chapter 3: Neural Networks
From Perceptrons to Neural Networks
Neural Network Example
Reviewing the Perceptron
Introducing an Activation Function
Activation Function
Sigmoid Function
Implementing a Step Function
Step Function Graph
Implementing a Sigmoid Function
Comparing the Sigmoid Function and the Step Function
Nonlinear Function
ReLU Function
Calculating Multidimensional Arrays
Multidimensional Arrays
Matrix Multiplication
Matrix Multiplication in a Neural Network
Implementing a Three-Layer Neural Network
Examining the Symbols
Implementing Signal Transmission in Each Layer
Implementation Summary
Designing the Output Layer
Identity Function and Softmax Function
Issues when Implementing the Softmax Function
Characteristics of the Softmax Function
Number of Neurons in the Output Layer
Handwritten Digit Recognition
MNIST Dataset
Inference for Neural Network
Batch Processing
Summary
Chapter 4: Neural Network Training
Learning from Data
Data-Driven
Training Data and Test Data
Loss Function
Sum of Squared Errors
Cross-Entropy Error
Mini-Batch Learning
Implementing Cross-Entropy Error (Using Batches)
Why Do We Configure a Loss Function?
Numerical Differentiation
Derivative
Examples of Numerical Differentiation
Partial Derivative
Gradient
Gradient Method
Gradients for a Neural Network
Implementing a Training Algorithm
A Two-Layer Neural Network as a Class
Implementing Mini-Batch Training
Using Test Data for Evaluation
Summary
Chapter 5: Backpropagation
Computational Graphs
Using Computational Graphs to Solve Problems
Local Calculation
Why Do We Use Computational Graphs?
Chain Rule
Backward Propagation in a Computational Graph
What Is the Chain Rule?
The Chain Rule and Computational Graphs
Backward Propagation
Backward Propagation in an Addition Node
Backward Propagation in a Multiplication Node
Apples Example
Implementing a Simple Layer
Implementing a Multiplication Layer
Implementing an Addition Layer
Implementing the Activation Function Layer
ReLU Layer
Sigmoid Layer
Implementing the Affine and Softmax Layers
Affine Layer
Batch-Based Affine Layer
Softmax-with-Loss Layer
Implementing Backpropagation
Overall View of Neural Network Training
Presupposition
Implementing a Neural Network That Supports Backpropagation
Gradient Check-in Backpropagation
Training Using Backpropagation
Summary
Chapter 6: Training Techniques
Updating Parameters
Story of an Adventurer
SGD
Disadvantage of SGD
Momentum
AdaGrad
Adam
Which Update Technique Should We Use?
Using the MNIST Dataset to Compare the Update Techniques
Initial Weight Values
How About Setting the Initial Weight Values to 0?
Distribution of Activations in the Hidden Layers
Initial Weight Values for ReLU
Using the MNIST Dataset to Compare the Weight Initializers
Batch Normalization
Batch Normalization Algorithm
Evaluating Batch Normalization
Regularization
Overfitting
Weight Decay
Dropout
Validating Hyperparameters
Validation Data
Optimizing Hyperparameters
Implementing Hyperparameter Optimization
Summary
Chapter 7: Convolutional Neural Networks
Overall Architecture
The Convolution Layer
Issues with the Fully Connected Layer
Convolution Operations
Padding
Stride
Performing a Convolution Operation on Three-Dimensional Data
Thinking in Blocks
Batch Processing
The Pooling Layer
Characteristics of a Pooling Layer
Implementing the Convolution and Pooling Layers
Four-Dimensional Arrays
Expansion by im2col
Implementing a Convolution Layer
Implementing a Pooling Layer
Implementing a CNN
Visualizing a CNN
Visualizing the Weight of the First Layer
Using a Hierarchical Structure to Extract Information
Typical CNNs
LeNet
AlexNet
Summary
Chapter 8: Deep Learning
Making a Network Deeper
Deeper Networks
Improving Recognition Accuracy
Motivation for a Deeper Network
A Brief History of Deep Learning
ImageNet
VGG
GoogLeNet
ResNet
Accelerating Deep Learning
Challenges to Overcome
Using GPUs for Acceleration
Distributed Training
Reducing the Bit Number for Arithmetic Precision
Practical Uses of Deep Learning
Object Detection
Segmentation
Generating Image Captions
The Future of Deep Learning
Converting Image Styles
Generating Images
Automated Driving
Deep Q-Networks (Reinforcement Learning)
Summary
Appendix A
Index