Cover......Page 1
Copyright......Page 4
Table of Contents......Page 5
Conventions Used in This Book......Page 11
Safari® Books Online......Page 12
Acknowledgements......Page 13
Building Intelligent Machines......Page 15
The Limits of Traditional Computer Programs......Page 16
The Mechanics of Machine Learning......Page 17
The Neuron......Page 21
Expressing Linear Perceptrons as Neurons......Page 22
Feed-Forward Neural Networks......Page 23
Linear Neurons and Their Limitations......Page 26
Sigmoid, Tanh, and ReLU Neurons......Page 27
Looking Forward......Page 29
The Fast-Food Problem......Page 31
Gradient Descent......Page 33
The Delta Rule and Learning Rates......Page 35
Gradient Descent with Sigmoidal Neurons......Page 36
The Backpropagation Algorithm......Page 37
Stochastic and Minibatch Gradient Descent......Page 39
Test Sets, Validation Sets, and Overfitting......Page 41
Preventing Overfitting in Deep Neural Networks......Page 48
Summary......Page 51
What Is TensorFlow?......Page 53
How Does TensorFlow Compare to Alternatives?......Page 54
Installing TensorFlow......Page 55
Creating and Manipulating TensorFlow Variables......Page 57
Placeholder Tensors......Page 59
Sessions in TensorFlow......Page 60
Navigating Variable Scopes and Sharing Variables......Page 62
Managing Models over the CPU and GPU......Page 65
Specifying the Logistic Regression Model in TensorFlow......Page 66
Logging and Training the Logistic Regression Model......Page 69
Leveraging TensorBoard to Visualize Computation Graphs and Learning......Page 72
Building a Multilayer Model for MNIST in TensorFlow......Page 73
Summary......Page 76
The Challenges with Gradient Descent......Page 77
Local Minima in the Error Surfaces of Deep Networks......Page 78
Model Identifiability......Page 79
How Pesky Are Spurious Local Minima in Deep Networks?......Page 80
Flat Regions in the Error Surface......Page 83
When the Gradient Points in the Wrong Direction......Page 85
Momentum-Based Optimization......Page 88
A Brief View of Second-Order Methods......Page 91
Learning Rate Adaptation......Page 92
AdaGrad—Accumulating Historical Gradients......Page 93
RMSProp—Exponentially Weighted Moving Average of Gradients......Page 94
Adam—Combining Momentum and RMSProp......Page 95
Summary......Page 97
Neurons in Human Vision......Page 99
The Shortcomings of Feature Selection......Page 100
Vanilla Deep Neural Networks Don’t Scale......Page 103
Filters and Feature Maps......Page 104
Full Description of the Convolutional Layer......Page 109
Max Pooling......Page 112
Full Architectural Description of Convolution Networks......Page 113
Closing the Loop on MNIST with Convolutional Networks......Page 115
Image Preprocessing Pipelines Enable More Robust Models......Page 117
Accelerating Training with Batch Normalization......Page 118
Building a Convolutional Network for CIFAR-10......Page 121
Visualizing Learning in Convolutional Networks......Page 123
Leveraging Convolutional Filters to Replicate Artistic Styles......Page 127
Learning Convolutional Filters for Other Problem Domains......Page 128
Summary......Page 129
Learning Lower-Dimensional Representations......Page 131
Principal Component Analysis......Page 132
Motivating the Autoencoder Architecture......Page 134
Implementing an Autoencoder in TensorFlow......Page 135
Denoising to Force Robust Representations......Page 148
Sparsity in Autoencoders......Page 151
When Context Is More Informative than the Input Vector......Page 154
The Word2Vec Framework......Page 157
Implementing the Skip-Gram Architecture......Page 160
Summary......Page 166
Analyzing Variable-Length Inputs......Page 167
Tackling seq2seq with Neural N-Grams......Page 169
Implementing a Part-of-Speech Tagger......Page 170
Dependency Parsing and SyntaxNet......Page 178
Beam Search and Global Normalization......Page 182
A Case for Stateful Deep Learning Models......Page 186
Recurrent Neural Networks......Page 187
The Challenges with Vanishing Gradients......Page 190
Long Short-Term Memory (LSTM) Units......Page 192
TensorFlow Primitives for RNN Models......Page 197
Implementing a Sentiment Analysis Model......Page 199
Solving seq2seq Tasks with Recurrent Neural Networks......Page 203
Augmenting Recurrent Networks with Attention......Page 205
Dissecting a Neural Translation Network......Page 208
Summary......Page 231
Neural Turing Machines......Page 233
Attention-Based Memory Access......Page 235
NTM Memory Addressing Mechanisms......Page 237
Differentiable Neural Computers......Page 240
Interference-Free Writing in DNCs......Page 243
DNC Memory Reuse......Page 244
Temporal Linking of DNC Writes......Page 245
The DNC Controller Network......Page 246
Visualizing the DNC in Action......Page 248
Implementing the DNC in TensorFlow......Page 251
Teaching a DNC to Read and Comprehend......Page 256
Summary......Page 258
Deep Reinforcement Learning Masters Atari Games......Page 259
What Is Reinforcement Learning?......Page 261
Markov Decision Processes (MDP)......Page 262
Policy......Page 263
Future Return......Page 264
Explore Versus Exploit......Page 265
Policy Versus Value Learning......Page 267
OpenAI Gym......Page 268
Creating an Agent......Page 269
Keeping Track of History......Page 271
Policy Gradient Main Function......Page 272
PGAgent Performance on Pole-Cart......Page 274
The Bellman Equation......Page 275
Approximating the Q-Function......Page 276
Learning Stability......Page 277
From Q-Function to Policy......Page 278
Playing Breakout wth DQN......Page 279
Setting Up Training Operations......Page 282
Implementing Experience Replay......Page 283
DQN Main Loop......Page 284
DQNAgent Results on Breakout......Page 286
Deep Recurrent Q-Networks (DRQN)......Page 287
Asynchronous Advantage Actor-Critic Agent (A3C)......Page 288
UNsupervised REinforcement and Auxiliary Learning (UNREAL)......Page 289
Summary......Page 290
Index......Page 291
Colophon......Page 298