Deep Learning By Example: A hands-on guide to implementing advanced machine learning algorithms and neural networks

✍ Scribed by Ahmed Menshawy

• Publisher: Packt Publishing
• Year: 2018
• Tongue: English
• Leaves: 442
• Category: Library

✦ Synopsis

Grasp the fundamental concepts of deep learning using Tensorflow in a hands-on manner

Key Features

• Get a first-hand experience of the deep learning concepts and techniques with this easy-to-follow guide
• Train different types of neural networks using Tensorflow for real-world problems in language processing, computer vision, transfer learning, and more
• Designed for those who believe in the concept of 'learn by doing', this book is a perfect blend of theory and code examples

Book Description

Deep learning is a popular subset of machine learning, and it allows you to build complex models that are faster and give more accurate predictions. This book is your companion to take your first steps into the world of deep learning, with hands-on examples to boost your understanding of the topic.

This book starts with a quick overview of the essential concepts of data science and machine learning which are required to get started with deep learning. It introduces you to Tensorflow, the most widely used machine learning library for training deep learning models. You will then work on your first deep learning problem by training a deep feed-forward neural network for digit classification, and move on to tackle other real-world problems in computer vision, language processing, sentiment analysis, and more. Advanced deep learning models such as generative adversarial networks and their applications are also covered in this book.

By the end of this book, you will have a solid understanding of all the essential concepts in deep learning. With the help of the examples and code provided in this book, you will be equipped to train your own deep learning models with more confidence.

What you will learn

• Understand the fundamentals of deep learning and how it is different from machine learning
• Get familiarized with Tensorflow, one of the most popular libraries for advanced machine learning
• Increase the predictive power of your model using feature engineering
• Understand the basics of deep learning by solving a digit classification problem of MNIST
• Demonstrate face generation based on the CelebA database, a promising application of generative models
• Apply deep learning to other domains like language modeling, sentiment analysis, and machine translation

Who This Book Is For

This book targets data scientists and machine learning developers who wish to get started with deep learning. If you know what deep learning is but are not quite sure of how to use it, this book will help you as well. An understanding of statistics and data science concepts is required. Some familiarity with Python programming will also be beneficial.

Table of Contents

1. Data science: Bird's-eye view
2. Data Modeling in Action - The Titanic Example
3. Feature Engineering and Model Complexity - The Titanic Example Revisited
4. Get Up and Running with TensorFlow
5. Tensorflow in Action - Some Basic Examples
6. Deep Feed-forward Neural Networks - Implementing Digit Classification
7. Introduction to Convolutional Neural Networks
8. Object Detection - CIFAR-10 Example
9. Object Detection - Transfer Learning with CNNs
10. Recurrent-Type Neural Networks - Language modeling
11. Representation Learning - Implementing Word Embeddings
12. Neural sentiment Analysis
13. Autoencoders - Feature Extraction and Denoising
14. Generative Adversarial Networks in Action - Generating New Images
15. Face Generation and Handling Missing Labels
16. Appendix - Implementing Fish Recognition

✦ Table of Contents

Cover
Copyright and Credits
Packt Upsell
Contributors
Table of Contents
Preface
Chapter 1: Data Science - A Birds' Eye View
Understanding data science by an example
Design procedure of data science algorithms
Data pre-processing
Data cleaning
Data pre-processing
Feature selection
Model selection
Learning process
Evaluating your model
Getting to learn
Challenges of learning
Feature extraction – feature engineering
Noise
Overfitting
Selection of a machine learning algorithm
Prior knowledge
Missing values
Implementing the fish recognition/detection model
Knowledge base/dataset
Data analysis pre-processing
Model building
Model training and testing
Fish recognition – all together
Different learning types
Supervised learning
Unsupervised learning
Semi-supervised learning
Reinforcement learning
Data size and industry needs
Summary
Chapter 2: Data Modeling in Action - The Titanic Example
Linear models for regression
Motivation
Advertising – a financial example
Dependencies
Importing data with pandas
Understanding the advertising data
Data analysis and visualization
Simple regression model
Learning model coefficients
Interpreting model coefficients
Using the model for prediction
Linear models for classification
Classification and logistic regression
Titanic example – model building and training
Data handling and visualization
Data analysis – supervised machine learning
Different types of errors
Apparent (training set) error
Generalization/true error
Summary
Chapter 3: Feature Engineering and Model Complexity – The Titanic Example Revisited
Feature engineering
Types of feature engineering
Feature selection
Dimensionality reduction
Feature construction
Titanic example revisited
Missing values
Removing any sample with missing values in it
Missing value inputting
Assigning an average value
Using a regression or another simple model to predict the values of missing variables
Feature transformations
Dummy features
Factorizing
Scaling
Binning
Derived features
Name
Cabin
Ticket
Interaction features
The curse of dimensionality
Avoiding the curse of dimensionality
Titanic example revisited – all together
Bias-variance decomposition
Learning visibility
Breaking the rule of thumb
Summary
Chapter 4: Get Up and Running with TensorFlow
TensorFlow installation
TensorFlow GPU installation for Ubuntu 16.04
Installing NVIDIA drivers and CUDA 8
Installing TensorFlow
TensorFlow CPU installation for Ubuntu 16.04
TensorFlow CPU installation for macOS X
TensorFlow GPU/CPU installation for Windows
The TensorFlow environment
Computational graphs
TensorFlow data types, variables, and placeholders
Variables
Placeholders
Mathematical operations
Getting output from TensorFlow
TensorBoard – visualizing learning
Summary
Chapter 5: TensorFlow in Action - Some Basic Examples
Capacity of a single neuron
Biological motivation and connections
Activation functions
Sigmoid
Tanh
ReLU
Feed-forward neural network
The need for multilayer networks
Training our MLP – the backpropagation algorithm
Step 1 – forward propagation
Step 2 – backpropagation and weight updation
TensorFlow terminologies – recap
Defining multidimensional arrays using TensorFlow
Why tensors?
Variables
Placeholders
Operations
Linear regression model – building and training
Linear regression with TensorFlow
Logistic regression model – building and training
Utilizing logistic regression in TensorFlow
Why use placeholders?
Set model weights and bias
Logistic regression model
Training
Cost function
Summary
Chapter 6: Deep Feed-forward Neural Networks - Implementing Digit Classification
Hidden units and architecture design
MNIST dataset analysis
The MNIST data
Digit classification – model building and training
Data analysis
Building the model
Model training
Summary
Chapter 7: Introduction to Convolutional Neural Networks
The convolution operation
Motivation
Applications of CNNs
Different layers of CNNs
Input layer
Convolution step
Introducing non-linearity
The pooling step
Fully connected layer
Logits layer
CNN basic example – MNIST digit classification
Building the model
Cost function
Performance measures
Model training
Summary
Chapter 8: Object Detection – CIFAR-10 Example
Object detection
CIFAR-10 – modeling, building, and training
Used packages
Loading the CIFAR-10 dataset
Data analysis and preprocessing
Building the network
Model training
Testing the model
Summary
Chapter 9: Object Detection – Transfer Learning with CNNs
Transfer learning
The intuition behind TL
Differences between traditional machine learning and TL
CIFAR-10 object detection – revisited
Solution outline
Loading and exploring CIFAR-10
Inception model transfer values
Analysis of transfer values
Model building and training
Summary
Chapter 10: Recurrent-Type Neural Networks - Language Modeling
The intuition behind RNNs
Recurrent neural networks architectures
Examples of RNNs
Character-level language models
Language model using Shakespeare data
The vanishing gradient problem
The problem of long-term dependencies
LSTM networks
Why does LSTM work?
Implementation of the language model
Mini-batch generation for training
Building the model
Stacked LSTMs
Model architecture
Inputs
Building an LSTM cell
RNN output
Training loss
Optimizer
Building the network
Model hyperparameters
Training the model
Saving checkpoints
Generating text
Summary
Chapter 11: Representation Learning - Implementing Word Embeddings
Introduction to representation learning
Word2Vec
Building Word2Vec model
A practical example of the skip-gram architecture
Skip-gram Word2Vec implementation
Data analysis and pre-processing
Building the model
Training
Summary
Chapter 12: Neural Sentiment Analysis
General sentiment analysis architecture
RNNs – sentiment analysis context
Exploding and vanishing gradients - recap
Sentiment analysis – model implementation
Keras
Data analysis and preprocessing
Building the model
Model training and results analysis
Summary
Chapter 13: Autoencoders – Feature Extraction and Denoising
Introduction to autoencoders
Examples of autoencoders
Autoencoder architectures
Compressing the MNIST dataset
The MNIST dataset
Building the model
Model training
Convolutional autoencoder
Dataset
Building the model
Model training
Denoising autoencoders
Building the model
Model training
Applications of autoencoders
Image colorization
More applications
Summary
Chapter 14: Generative Adversarial Networks
An intuitive introduction
Simple implementation of GANs
Model inputs
Variable scope
Leaky ReLU
Generator
Discriminator
Building the GAN network
Model hyperparameters
Defining the generator and discriminator
Discriminator and generator losses
Optimizers
Model training
Generator samples from training
Sampling from the generator
Summary
Chapter 15: Face Generation and Handling Missing Labels
Face generation
Getting the data
Exploring the Data
Building the model
Model inputs
Discriminator
Generator
Model losses
Model optimizer
Training the model
Semi-supervised learning with Generative Adversarial Networks (GANs)
Intuition
Data analysis and preprocessing
Building the model
Model inputs
Generator
Discriminator
Model losses
Model optimizer
Model training
Summary
Appendix: Implementing Fish Recognition
Code for fish recognition
Other Books You May Enjoy
Index