Deep Learning Pipeline: Building a Deep Learning Model with TensorFlow

✍ Scribed by Hisham El-amir, Mahmoud Hamdy

• Publisher: Apress
• Year: 2019
• Tongue: English
• Leaves: 563
• Edition: 1
• Category: Library

✦ Synopsis

Build your own pipeline based on modern TensorFlow approaches rather than outdated engineering concepts. This book shows you how to build a deep learning pipeline for real-life TensorFlow projects. 

You'll learn what a pipeline is and how it works so you can build a full application easily and rapidly. Then troubleshoot and overcome basic Tensorflow obstacles to easily create functional apps and deploy well-trained models. Step-by-step and example-oriented instructions help you understand each step of the deep learning pipeline while you apply the most straightforward and effective tools to demonstrative problems and datasets.  

You'll also develop a deep learning project by preparing data, choosing the model that fits that data, and debugging your model to get the best fit to data all using Tensorflow techniques. Enhance your skills by accessing some of the most powerful recent trends in data science. If you've ever considered building your own image or text-tagging solution or entering a Kaggle contest,Deep Learning Pipeline is for you!
 

What You'll Learn

• Develop a deep learning project using data
• Study and apply various models to your data
• Debug and troubleshoot the proper model suited for your data

Who This Book Is For

Developers, analysts, and data scientists looking to add to or enhance their existing skills by accessing some of the most powerful recent trends in data science. Prior experience in Python or other TensorFlow related languages and mathematics would be helpful.

✦ Table of Contents

Table of Contents
About the Authors
About the Technical Reviewer
Introduction
Part I: Introduction
Chapter 1: A Gentle Introduction
Information Theory, Probability Theory, and Decision Theory
Information Theory
Probability Theory
Decision Theory
Introduction to Machine Learning
Predictive Analytics and Its Connection with Machine learning
Machine Learning Approaches
Supervised Learning
Unsupervised Learning
Semisupervised Learning
Checkpoint
Reinforcement Learning
From Machine Learning to Deep Learning
Lets’ See What Some Heroes of Machine Learning Say About the Field
Connections Between Machine Learning and Deep Learning
Difference Between ML and DL
In Machine Learning
In Deep Learning
What Have We Learned Here?
Why Should We Learn About Deep Learning (Advantages of Deep learning)?
Disadvantages of Deep Learning (Cost of Greatness)
Introduction to Deep Learning
Machine Learning Mathematical Notations
Summary
Chapter 2: Setting Up Your Environment
Background
Python 2 vs. Python 3
Installing Python
Python Packages
IPython
Installing IPython
Jupyter
Installing Jupyter
What Is an ipynb File?
Packages Used in the Book
NumPy
SciPy
Pandas
Matplotlib
NLTK
Scikit-learn
Gensim
TensorFlow
Installing on Mac or Linux distributions
Installing on Windows
Keras
Summary
Chapter 3: A Tour Through the Deep Learning Pipeline
Deep Learning Approaches
What Is Deep Learning
Biological Deep Learning
What Are Neural Networks Architectures?
Deep Learning Pipeline
Define and Prepare Problem
Summarize and Understand Data
Process and Prepare Data
Evaluate Algorithms
Improve Results
Fast Preview of the TensorFlow Pipeline
Tensors—the Main Data Structure
First Session
Data Flow Graphs
Tensor Properties
Tensor Rank
Tensor Shape
Summary
Chapter 4: Build Your First Toy TensorFlow app
Basic Development of TensorFlow
Hello World with TensorFlow
Simple Iterations
Prepare the Input Data
Doing the Gradients
Linear Regression
Why Linear Regression?
What Is Linear Regression?
Dataset Description
Full Source Code
XOR Implementation Using TensorFlow
Full Source Code
Summary
Part II: Data
Chapter 5: Defining Data
Defining Data
Why Should You Read This Chapter?
Structured, Semistructured, and Unstructured Data
Tidy Data
Divide and Conquer
Tabular Data
Quantitative​ vs. ​Qualitative​ Data
Example—the Titanic
Divide and Conquer
Making a Checkpoint
The Four Levels of Data
Measure of Center
The Nominal Level
Mathematical Operations Allowed for Nominal
Measures of Center for Nominal
What Does It Mean to be a Nominal Level Type?
The Ordinal Level
Examples of Being Ordinal
What Data Is Like at the Ordinal Level
Mathematical Operations Allowed for Ordinal
Measures of Center for Ordinal
Quick Recap and Check
The Interval Level
Examples of Interval Level Data
What Data Is Like at the Interval Level
Mathematical Operations Allowed for Interval
Measures of Center for Interval
Measures of Variation for Interval
Standard Deviation
The Ratio Level
Examples
Measures of Center for Ratio
Problems with the Ratio Level
Summarizing All Levels Table 5-1
Text Data
What Is Text Processing and What Is the Level of Importance of Text Processing?
IMDB—Example
Images Data
Type of Images (2-D, 3-D, 4-D)
2-D Data
3-D Data
4-D Data
Example—MNIST
Example—CIFAR-10
Summary
Chapter 6: Data Wrangling and Preprocessing
The Data Fields Pipelines Revisited
Giving You a Reason
Where Is Data Cleaning in the Process?
Data Loading and Preprocessing
Fast and Easy Data Loading
Missing Data
Empties
Is It Ever Useful to Fill Missing Data Using a Zero Instead of an Empty or Null?
Managing Missing Features
Dealing with Big Datasets
Accessing Other Data Formats
Data Preprocessing
Data Augmentation
Image Crop
Crop and Resize
Crop to Bounding Box
Flipping
Rotate Image
Translation
Transform
Adding Salt and Pepper Noise
Convert RGB to Grayscale
Change Brightness
Adjust Contrast
Adjust Hue
Adjust Saturation
Categorical and Text data
Data Encoding
Performing One-Hot Encoding on Nominal Features
Can You Spot the Problem?
A Special Type of Data: Text
So Far, Everything Has Been Pretty Good, Hasn’t It?
Tokenization, Stemming, and Stop Words
What Are Tokenizing and Tokenization?
The Bag-of-Words (BoW) Model
What is the BoW?
Summary
Chapter 7: Data Resampling
Creating Training and Test Sets
Cross-Validation
Validation Set Technique
Leave-One-Out Cross-Validation (LOOCV)
K-Fold Cross-Validation
Bootstrap
Bootstrap in Statistics
Tips to Use Bootstrap (Resampling with Replacement)
Generators
What Are Keras Generators?
Data Generator
Callback
Summary
Chapter 8: Feature Selection and Feature Engineering
Dataset Used in This Chapter
Dimensionality Reduction—Questions to Answer
What Is Dimensionality Reduction?
When Should I Use Dimensionality Reduction?
Unsupervised Dimensionality Reduction via Principal Component Analysis (PCA)
Total and Explained Variance
Feature Selection and Filtering
Principal Component Analysis
Nonnegative Matrix Factorization
Sparse PCA
Kernel PCA
Atom Extraction and Dictionary Learning
Latent Dirichlet Allocation (LDA)
Latent Dirichlet Allocation (LDA in NLP)
Code Example Using gensim
LDA vs. PCA
ZCA Whitening
Summary
Part III: TensorFlow
Chapter 9: Deep Learning Fundamentals
Perceptron
Single Perceptron
Multilayer Perceptron
Recap
Different Neural Network Layers
Input Layer
Hidden Layer(s)
Output Layer
Shallow vs. Deep Neural Networks
Activation Functions
Types of Activation Functions
Recap
Gradient Descent
Recap
Batch vs. Stochastic vs. Mini-Batch Gradient Descent
Batch Gradient Descent
Stochastic Gradient Descent
Mini-batch Gradient Descent
Recap
Loss function and Backpropagation
Loss Function
Backpropagation
The Four Fundamental Equations Behind Backpropagation
Exploding Gradients
Re-Design the Network Model
Use Long Short-Term Memory Networks
Use Gradient Clipping
Use Weight Regularization
Vanishing Gradients
Vanishing Gradients Problem
TensorFlow Basics
Placeholder vs. Variable vs. Constant
Gradient-Descent Optimization Methods from a Deep-Learning Perspective
Learning Rate in the Mini-batch Approach to Stochastic Gradient Descent
Summary
Chapter 10: Improving Deep Neural Networks
Optimizers in TensorFlow
The Notation to Use
Momentum
Nesterov Accelerated Gradient
Adagrad
Adadelta
RMSprop
Adam
Nadam (Adam + NAG)
Choosing the Learning Rate
Dropout Layers and Regularization
Normalization Techniques
Batch Normalization
Weight Normalization
Layer Normalization
Instance Normalization
Group Normalization
Summary
Chapter 11: Convolutional Neural Network
What is a Convolutional Neural Network
Convolution Operation
One-Dimensional Convolution
Two-Dimensional Convolution
Padding and Stride
Common Image-Processing Filters
Mean and Median Filters
Gaussian Filter
Sobel Edge-Detection Filter
Identity Transform
Convolutional Neural Networks
Layers of Convolutional Neural Networks
Input Layer
Convolutional layer
Pooling Layer
Backpropagation Through the Convolutional and Pooling Layers
Weight Sharing Through Convolution and Its Advantages
Translation Equivariance and Invariance
Case Study—Digit Recognition on the CIFAR-10 Dataset
Summary
Chapter 12: Sequential Models
Recurrent Neural Networks
Language Modeling
Backpropagation Through Time
Vanishing and Exploding Gradient Problems in RNN
The Solution to Vanishing and Exploding Gradients Problems in RNNs
Long Short-Term Memory
Case Study—Digit Identification on the MNIST Dataset
Gated Recurrent Unit
Bidirectional RNN (Bi-RNN)
Summary
Part IV: Applying What You’ve Learned
Chapter 13: Selected Topics in Computer Vision
Different Architectures in Convolutional Neural Networks
LeNet
AlexNet
VGG
ResNet
Transfer Learning
What Is a Pretrained Model, and Why Use It?
How to Use a Pretrained Model?
Ways to Fine-Tune the Model
Pretrained VGG19
Summary
Chapter 14: Selected Topics in Natural Language Processing
Vector Space Model
Vector Representation of Words
Word2Vec
Continuous Bag of Words
Implementing Continuous Bag of Words
Skip-Gram Model for Word Embeddings
Implementing Skip-Gram
GloVe
Summary
Chapter 15: Applications
Case Study—Tabular Dataset
Understanding the Dataset
Scratching the Surface
Digging Deeper
Preprocessing Dataset
Building the Model
Case Study—IMDB Movie Review Data with Word2Vec
Case Study—Image Segmentation
Summary
Index