Applied Deep Learning. A Case-based Approach to Understanding Deep Neural Networks

✍ Scribed by Umberto Michelucci

• Publisher: Apress
• Year: 2018
• Tongue: English
• Leaves: 419
• Category: Library

✦ Table of Contents

Contents......Page 3
Intro......Page 9
Set up Python Environment......Page 14
Basic Intro to TensorFlow......Page 27
Structure of Neuron......Page 43
Example of Logistic Regression......Page 82
Refs......Page 92
Feedforward Neural Networks......Page 94
Network Architecture......Page 95
sof tmax Function for Multiclass Classification......Page 101
Brief Digression - Overfitting......Page 102
Zalando Dataset......Page 111
Building Model with tensorflow......Page 116
Gradient Descent Variations......Page 125
Examples of Wrong Predictions......Page 134
Weight Initialization......Page 136
Adding many Layers efficiently......Page 138
Advantages of Additional Hidden Layers......Page 141
Comparing Different Networks......Page 142
Tips for Choosing the Right Network......Page 146
Dynamic Learning Rate Decay......Page 148
Common Optimizers......Page 174
Example of Self-Developed Optimizer......Page 190
Complex Networks & Overfitting......Page 196
Regularization......Page 201
ℓp Regularization......Page 203
ℓ2 Regularization......Page 216
Dropout......Page 222
Early Stopping......Page 226
Additional Methods......Page 227
Metric Analysis......Page 228
Human-Level Performance & Bayes Error......Page 229
Human-Level Performance......Page 232
Bias......Page 234
Training Set Overfitting......Page 236
Test Set......Page 239
Split Dataset......Page 241
Unbalanced Class Distribution......Page 245
Precision, Recall & F1 Metrics......Page 250
Datasets with Different Distributions......Page 256
K-Fold Cross-Validation......Page 264
Manual Metric Analysis - Example......Page 274
Black-Box Optimization......Page 282
Notes on Black-Box Functions......Page 284
Problem of Hyperparameter Tuning......Page 285
Sample Black-Box Problem......Page 286
Grid Search......Page 288
Random Search......Page 293
Coarse-to-Fine Optimization......Page 296
Bayesian Optimization......Page 300
Sampling on Logarithmic Scale......Page 321
Hyperparameter Tuning with Zalando Dataset......Page 323
Quick Note on Radial Basis Function......Page 332
Kernels & Filters......Page 334
Convolution......Page 336
Examples of Convolution......Page 345
Pooling......Page 353
Building Blocks of CNN......Page 357
Intro to RNNs......Page 366
Problem Description......Page 376
Regression Problem......Page 380
Dataset Preparation......Page 386
Model Training......Page 395
Logistic Regression from Scratch......Page 401
Mathematics of Logistic Regression......Page 402
Python Implementation......Page 405
Test of the Model......Page 408
Conclusion......Page 411
Index......Page 412