Deep Learning Illustrated: A Visual, Interactive Guide to Artificial Intelligence

✍ Scribed by Jon Krohn, Grant Beyleveld, Aglaé Bassens

• Publisher: Addison-Wesley Professional
• Year: 2019
• Tongue: English
• Leaves: 416
• Series: Addison-Wesley Data & Analytics Series
• Edition: 1
• Category: Library

✦ Synopsis

Deep learning is one of today’s hottest fields. This approach to machine learning is achieving breakthrough results in some of today’s highest profile applications, in organizations ranging from Google to Tesla, Facebook to Apple. Thousands of technical professionals and students want to start leveraging its power, but previous books on deep learning have often been non-intuitive, inaccessible, and dry. In Deep Learning Illustrated, three world-class instructors and practitioners present a uniquely visual, intuitive, and accessible high-level introduction to the techniques and applications of deep learning. Packed with vibrant, full-color illustrations, it abstracts away much of the complexity of building deep learning models, making the field more fun to learn, and accessible to a far wider audience.

 

Part I’s high-level overview explains what Deep Learning is, why it has become so ubiquitous, and how it relates to concepts and terminology such as Artificial Intelligence, Machine Learning, Artificial Neural Networks, and Reinforcement Learning. These opening chapters are replete with vivid illustrations, easy-to-grasp analogies, and character-focused narratives.

 

Building on this foundation, the authors then offer a practical reference and tutorial for applying a wide spectrum of proven deep learning techniques. Essential theory is covered with as little mathematics as possible, and illuminated with hands-on Python code. Theory is supported with practical “run-throughs” available in accompanying Jupyter notebooks, delivering a pragmatic understanding of all major deep learning approaches and their applications: machine vision, natural language processing, image generation, and videogaming.

 

To help readers accomplish more in less time, the authors feature several of today’s most widely-used and innovative deep learning libraries, including TensorFlow and its high-level API, Keras; PyTorch, and the recently-released high-level Coach, a TensorFlow API that abstracts away the complexity typically associated with building Deep Reinforcement Learning algorithms.

✦ Table of Contents

Cover
Title Page
Copyright Page
Contents
Figures
Tables
Examples
Foreword
Preface
Acknowledgments
About the Authors
Part I: Introducing Deep Learning
1 Biological and Machine Vision
Biological Vision
Machine Vision
The Neocognitron
LeNet-5
The Traditional Machine Learning Approach
ImageNet and the ILSVRC
AlexNet
TensorFlow Playground
Quick, Draw!
Summary
2 Human and Machine Language
Deep Learning for Natural Language Processing
Deep Learning Networks Learn Representations Automatically
Natural Language Processing
A Brief History of Deep Learning for NLP
Computational Representations of Language
One-Hot Representations of Words
Word Vectors
Word-Vector Arithmetic
word2viz
Localist Versus Distributed Representations
Elements of Natural Human Language
Google Duplex
Summary
3 Machine Art
A Boozy All-Nighter
Arithmetic on Fake Human Faces
Style Transfer: Converting Photos into Monet (and Vice Versa)
Make Your Own Sketches Photorealistic
Creating Photorealistic Images from Text
Image Processing Using Deep Learning
Summary
4 Game-Playing Machines
Deep Learning, AI, and Other Beasts
Artificial Intelligence
Machine Learning
Representation Learning
Artificial Neural Networks
Deep Learning
Machine Vision
Natural Language Processing
Three Categories of Machine Learning Problems
Supervised Learning
Unsupervised Learning
Reinforcement Learning
Deep Reinforcement Learning
Video Games
Board Games
AlphaGo
AlphaGo Zero
AlphaZero
Manipulation of Objects
Popular Deep Reinforcement Learning Environments
OpenAI Gym
DeepMind Lab
Unity ML-Agents
Three Categories of AI
Artificial Narrow Intelligence
Artificial General Intelligence
Artificial Super Intelligence
Summary
Part II: Essential Theory Illustrated
5 The (Code) Cart Ahead of the (Theory) Horse
Prerequisites
Installation
A Shallow Network in Keras
The MNIST Handwritten Digits
A Schematic Diagram of the Network
Loading the Data
Reformatting the Data
Designing a Neural Network Architecture
Training a Deep Learning Model
Summary
6 Artificial Neurons Detecting Hot Dogs
Biological Neuroanatomy 101
The Perceptron
The Hot Dog / Not Hot Dog Detector
The Most Important Equation in This Book
Modern Neurons and Activation Functions
The Sigmoid Neuron
The Tanh Neuron
ReLU: Rectified Linear Units
Choosing a Neuron
Summary
Key Concepts
7 Artificial Neural Networks
The Input Layer
Dense Layers
A Hot Dog-Detecting Dense Network
Forward Propagation Through the First Hidden Layer
Forward Propagation Through Subsequent Layers
The Softmax Layer of a Fast Food-Classifying Network
Revisiting Our Shallow Network
Summary
Key Concepts
8 Training Deep Networks
Cost Functions
Quadratic Cost
Saturated Neurons
Cross-Entropy Cost
Optimization: Learning to Minimize Cost
Gradient Descent
Learning Rate
Batch Size and Stochastic Gradient Descent
Escaping the Local Minimum
Backpropagation
Tuning Hidden-Layer Count and Neuron Count
An Intermediate Net in Keras
Summary
Key Concepts
9 Improving Deep Networks
Weight Initialization
Xavier Glorot Distributions
Unstable Gradients
Vanishing Gradients
Exploding Gradients
Batch Normalization
Model Generalization (Avoiding Overfitting)
L1 and L2 Regularization
Dropout
Data Augmentation
Fancy Optimizers
Momentum
Nesterov Momentum
AdaGrad
AdaDelta and RMSProp
Adam
A Deep Neural Network in Keras
Regression
TensorBoard
Summary
Key Concepts
Part III: Interactive Applications of Deep Learning
10 Machine Vision
Convolutional Neural Networks
The Two-Dimensional Structure of Visual Imagery
Computational Complexity
Convolutional Layers
Multiple Filters
A Convolutional Example
Convolutional Filter Hyperparameters
Pooling Layers
LeNet-5 in Keras
AlexNet and VGGNet in Keras
Residual Networks
Vanishing Gradients: The Bête Noire of Deep CNNs
Residual Connections
ResNet
Applications of Machine Vision
Object Detection
Image Segmentation
Transfer Learning
Capsule Networks
Summary
Key Concepts
11 Natural Language Process
Preprocessing Natural Language Data
Tokenization
Converting All Characters to Lowercase
Removing Stop Words and Punctuation
Stemming
Handling n-grams
Preprocessing the Full Corpus
Creating Word Embeddings with word2vec
The Essential Theory Behind word2vec
Evaluating Word Vectors
Running word2vec
Plotting Word Vectors
The Area under the ROC Curve
The Confusion Matrix
Calculating the ROC AUC Metric
Natural Language Classification with Familiar Networks
Loading the IMDb Film Reviews
Examining the IMDb Data
Standardizing the Length of the Reviews
Dense Network
Convolutional Networks
Networks Designed for Sequential Data
Recurrent Neural Networks
Long Short-Term Memory Units
Bidirectional LSTMs
Stacked Recurrent Models
Seq2seq and Attention
Transfer Learning in NLP
Non-sequential Architectures: The Keras Functional API
Summary
Key Concepts
12 Generative Adversarial Networks
Essential GAN Theory
The Quick, Draw! Dataset
The Discriminator Network
The Generator Network
The Adversarial Network
GAN Training
Summary
Key Concepts
13 Deep Reinforcement Learning
Essential Theory of Reinforcement Learning
The Cart-Pole Game
Markov Decision Processes
The Optimal Policy
Essential Theory of Deep Q-Learning Networks
Value Functions
Q-Value Functions
Estimating an Optimal Q-Value
Defining a DQN Agent
Initialization Parameters
Building the Agent’s Neural Network Model
Remembering Gameplay
Training via Memory Replay
Selecting an Action to Take
Saving and Loading Model Parameters
Interacting with an OpenAI Gym Environment
Hyperparameter Optimization with SLM Lab
Agents Beyond DQN
Policy Gradients and the REINFORCE Algorithm
The Actor-Critic Algorithm
Summary
Key Concepts
Part IV: You and AI
14 Moving Forward with Your Own Deep Learning Projects
Ideas for Deep Learning Projects
Machine Vision and GANs
Natural Language Processing
Deep Reinforcement Learning
Converting an Existing Machine Learning Project
Resources for Further Projects
Socially Beneficial Projects
The Modeling Process, Including Hyperparameter Tuning
Automation of Hyperparameter Search
Deep Learning Libraries
Keras and TensorFlow
PyTorch
MXNet, CNTK, Caffe, and So On
Software 2.0
Approaching Artificial General Intelligence
Summary
Part V: Appendices
Appendix A: Formal Neural Network Notation
Appendix B: Backpropagation
Appendix C: PyTorch
PyTorch Features
Autograd System
Define-by-Run Framework
PyTorch Versus TensorFlow
PyTorch in Practice
PyTorch Installation
The Fundamental Units Within PyTorch
Building a Deep Neural Network in PyTorch
Index
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z