Deep Learning Projects Using TensorFlow 2: Neural Network Development with Python and Keras

✍ Scribed by Vinita Silaparasetty

• Publisher: Apress
• Year: 2020
• Tongue: English
• Leaves: 439
• Edition: 1
• Category: Library

✦ Synopsis

Work through engaging and practical deep learning projects using TensorFlow 2.0. Using a hands-on approach, the projects in this book will lead new programmers through the basics into developing practical deep learning applications. 

Deep learning is quickly integrating itself into the technology landscape. Its applications range from applicable data science to deep fakes and so much more. It is crucial for aspiring data scientists or those who want to enter the field of AI to understand deep learning concepts. 

The best way to learn is by doing. You'll develop a working knowledge of not only TensorFlow, but also related technologies such as Python and Keras. You'll also work with Neural Networks and other deep learning concepts. By the end of the book, you'll have a collection of unique projects that you can add to your GitHub profiles and expand on for professional application. 

What You'll Learn

• Grasp the basic process of neural networks through projects, such as creating music
  
• Restore and colorize black and white images with deep learning processes

Who This Book Is For

Beginners new to TensorFlow and Python. 

✦ Table of Contents

Table of Contents
About the Author
About the Technical Reviewer
Acknowledgments
Preface
Chapter 1: Getting Started: Installation and Troubleshooting
Installing Python 3
Method 1: Direct Installation from the Official Python Website
Troubleshooting Tips
Method 2: Using Anaconda
Troubleshooting Tips
Installing Jupyter Notebook
Dependencies
Method 1: Using the PIP Installation Package
Troubleshooting Tips
Method 2: Using Anaconda
Troubleshooting Tips
Installing TensorFlow 2.0
Dependencies
Method 1: Using the PIP Installation Package
Troubleshooting Tips
Method 2: Using Anaconda
Troubleshooting Tips
Installing Keras
Dependencies
Using the PIP Installation Package
Troubleshooting Tips
Installing Python Libraries
Installing NumPy
Dependencies
Using the PIP Installation Package
Troubleshooting Tips
Installing SciPy
Dependencies
Using the PIP Installation Package
Troubleshooting Tips
Installing Matplotlib
Dependencies
Using the PIP Installation Package
Troubleshooting Tips
Installing Pandas
Dependencies
Using the PIP Installation Package
Troubleshooting Tips
Installing Scikit-Learn
Dependencies
Using the PIP Installation Package
Troubleshooting Tips
Summary
Chapter 2: Perceptrons
Biological Neurons
Artificial Neurons
Perceptrons
Perceptron Learning Rule
Types of Activation Functions
The Sigmoid Activation Function
The ReLU Function
The Softmax Function
Perceptrons in Action
Stage 1: Forward Propagation of Inputs
Stage 2: Calculation of the Net Input
Weights
Bias
Net Input
Stage 3: Activation Function
Stage 4: Backward Propagation
Project Description
Important Terminology
Required Libraries
Procedure
Step 1. Import Libraries
Step 2. Declare Parameters
Step 3. Declare the Weights and Bias
Step 4. Define the Perceptron Function
Step 5. Define the Loss Function and Optimizer
Step 6. Read in the Data
Step 7. Visualization of Labels
Step 8. Prepare Inputs
Step 9. Initialize Variables
Step 10. Train the Model
Step 11. New Values for Weights and Bias
Step 12. View the Final Loss
Step 13. Predicting Using the Trained Model
Step 14. Evaluate the Model
Summary
Chapter 3: Neural Networks
What Is a Neural Network?
Neural Network Components
Advantages of Neural Networks
Disadvantages of a Neural Networks
How a Neural Network Works
Forward Propagation
Backward Propagation
Types of Neural Networks
Feedforward Neural Network
Convolutional Neural Networks
Recurrent Neural Network (RNN)
Radial Basis Function Neural Network (RBNN)
Project Description
Flattening Data
About the Dataset
Required Libraries
Neural Network Architecture
Procedure
Summary
References
Chapter 4: Sentiment Analysis
LSTM Review
How an LSTM Works
Layers in an LSTM
Project Description
About the Dataset
Understanding Sentiment Analysis
Types of Sentiment Analysis
Fine-Grained Sentiment Analysis
Emotion Detection
Aspect-Based Sentiment Analysis
Intent Analysis
Multilingual Sentiment Analysis
Sentiment Analysis Algorithms
Sentiment Analysis Metrics for Evaluation
Cross-Validation
Precision
Recall
Accuracy
Hybrid Approaches
Parameters Affecting Model Performance
Subjectivity and Tone
Context and Polarity
Irony and Sarcasm
Comparisons
Defining Neutral Emotions
Tokenizer
H5 File
JSON File
Required Libraries
LSTM Architecture
Procedure
Step 1. Import Libraries
Step 2. Load the Data
Step 3. Prepare the Data
Step 4. Clean the Data
Step 5. Structure the Model
Step 6. Compile the Model
Step 7. Train the Model
Step 8. Save the Model (Optional)
Step 9. Import the Pretrained Model (Optional)
Further Tests
Troubleshooting
Summary
References
Further Reading
Chapter 5: Music Generation
GRU Overview
How a GRU Works
GRU Stages
Stage 1
Stage 2
Stage 3
Stage 4
GRU Layers
Comparing GRU and LSTM
Project Description
About the Dataset
Important Terminology and Concepts
Required Libraries
Installation Instructions
Using PIP
Using Windows
Using macOS
Using Linux
Installation Troubleshooting
GRU Architecture
Procedure
Step 1. Import Libraries
Step 2. Load the Data
Step 3. Feature Extraction
Step 4. Exploratory Data Analysis (EDA)
Step 5. Data Preparation (Input)
Step 6. Structure the Model
Step 7. Train the Model
Step 8. Prediction
Step 9. Data Preparation (Offset)
Step 10. Store the Output as a MIDI File
Further Tests
Troubleshooting
Summary
References
Resources
Further Reading
Chapter 6: Image Colorization
Human Vision Review
Computer Vision Review
How a CNN Works
Input Layer
Convolution Layer: The Kernel
Upsampling Layer
DepthwiseConv2D
Pooling Layer
Fully Connected Layer
Project Description
About the Dataset
Important Terminology
Color Space
Image Colorization
VGG-16
MAPE Loss Functions
Required Libraries
Installation Instructions
Installing PIL
Troubleshooting PIL
Installing CV2
Troubleshooting CV2
Installing Scikit-Image
Troubleshooting Scikit-Image
CNN+VGG-16 Architecture
Procedure
Step 1. Import the Libraries
Step 2. Convert the Images to Grayscale
Step 3. Load the Data
Step 4. Structure the Model
Step 5. Set the Model Parameters
Step 6. Data Preparation
Step 7. Train the Model
Step 8. Obtain Predictions
Step 9. View the Results
Troubleshooting
Further Tests
Summary
References
Further Reading
Chapter 7: Image Deblurring
What Is a GAN?
Types of GANs
How a GAN Works
The Generative Model
Process Within the Generator
The Discriminator Model
Process Within the Discriminator
Project Description
About the Dataset
Important Terminology and Concepts
Image Deblurring
Defocusing
Motion Smudging
Convolution
Deconvolution
GAN Architecture
Required Libraries
GAN Architecture
Generator
Discriminator
Procedure
Step 1. Import the Libraries
Step 2. Dataset Preparation
Step 3. Exploratory Data Analysis
Step 4. Structure the Model
Step 5. Input Preparation
Step 6. View the Images
Step 7. Save Results
Troubleshooting
Further Tests
Summary
References
Further Reading
Chapter 8: Image Manipulation
Project Description
Important Terminology and Concepts
Multimedia Forensics
Acquisition
Coding
Editing
Saving
Copy-Move Forgeries
About the Dataset
Required Libraries
Troubleshooting
CNN Architecture
Procedure
Step 1. Import the Libraries
Step 2. Preparing the Dataset
Step 2a. Sort and Collect the Authentic Data
Step 2b. Sort and Collect the Manipulated Data
Step 2c. Transform and Convert the Data to an Array
Step 2d. Create the Combined Dataset
Step 2e. Define the Optimizer
Step 3. Structure the Model
Step 4. Train the Model
Step 5. Test the Model
Step 6. Check the Results
Further Tests
Summary
References
Further Reading
Chapter 9: Neural Network Collection
Neural Network Zoo Primer
Neural Networks
Recurrent Neural Networks (RNNs)
Reservoir Computing
Multiplicative LSTM
ANNs with Attention
Transformers
Autoencoder
Variational Autoencoders
Denoising Autoencoders
Recurrent Autoencoders
Sparse Autoencoders
Stacked Autoencoders
Convolutional Autoencoders
Stacked Denoising Autoencoders
Stochastic Corruption in SDAs
Contractive Autoencoders
Markov Chains
Hopfield Networks
How Human Memory Works
Bidirectional Associative Memory
Boltzmann Machines
Restricted Boltzmann Machines
Deep Belief Networks
Deconvolutional Networks
Deep Convolutional Inverse Graphics Networks
Liquid State Machines
Human Brain Spiking
Echo State Networks (ESNs)
Deep Residual Network (ResNet)
ResNeXt
Neural Turing Machines
Reading
Capsule Networks
CAPSNet Architecture
LeNet-5
AlexNet
GoogLeNet
Xception
Optimizers
Stochastic Gradient Descent
RMSProp
AdaGrad
AdaDelta
Adam
Adamax
Nesterov Accelerated Gradient (NAG)
Nadam
Loss Functions
Mean Squared Error (MSE)
Mean Absolute Error (MAE)
Mean Absolute Percentage Error (MAPE)
Mean Squared Logarithmic Error (MSLE)
Squared Hinge
Hinge
Categorical Hinge
Log Cosh
Huber Loss
Categorical Cross-Entropy
Sparse Categorical Cross-Entropy
Binary Cross-Entropy
Kullback-Leibler Divergence
Poisson
References
Further Reading
Appendix: Portfolio Tips
Data Analyst Portfolios
LinkedIn Profile
GitHub Profile
Kaggle
Blogging Platforms
Sharing Your Portfolio
Twitter
Facebook
LinkedIn Groups
Tableau Public (Optional)
Types of Projects
Data Cleaning Project
Data Storytelling Project
Explanatory Post
Projects to Avoid
Selecting a Topic
Defining Problem Statements
Phase 1: Understanding the Goals and Expectations
Phase 2: Translating the Goals to Data Analysis Goals
Phase 3: Framing the Problem Statement
Phase 4: Choosing a Success Metric
Using Design Thinking
Benefits of Design Thinking
Phase 1: Planning
Phase 2: Model Design
Phase 3: Prototyping
Phase 4: Black-Box Testing
Black-Box Testing Techniques for Machine Learning Models
Solution Implementation
Phase 1: Data Collection
Phase 2: Data Exploration
Why EDA?
Univariate EDA
Multivariate EDA
Phase 3: Data Handling
Phase 4: Data Mining
Underfitting
Overcoming Underfitting
Overfitting
Overcoming Overfitting
Phase 5: Prototyping
Phase 6. Storytelling
What Is Color Theory?
Maintenance
Uploading Your Project to GitHub
Tips for Documenting Projects
Appendix Checklist
References
Further Reading
Resources for Building Your Portfolio
Read.me Template
Project Title
Problem Statement
Road Map Template
Data Cleaning Checklist
Index