Deep Learning with TensorFlow: Explore neural networks and build intelligent systems with Python, 2nd Edition

✍ Scribed by Giancarlo Zaccone, Md. Rezaul Karim

• Publisher: Packt Publishing
• Year: 2018
• Tongue: English
• Leaves: 484
• Series: Expert Insight
• Edition: 2
• Category: Library

✦ Synopsis

Delve into neural networks, implement deep learning algorithms, and explore layers of data abstraction with the help of this comprehensive TensorFlow guide

Key Features

• Learn how to implement advanced techniques in deep learning with Google's brainchild, TensorFlow
• Explore deep neural networks and layers of data abstraction with the help of this comprehensive guide
• Real-world contextualization through some deep learning problems concerning research and application

Book Description

Deep learning is the step that comes after machine learning, and has more advanced implementations. Machine learning is not just for academics anymore, but is becoming a mainstream practice through wide adoption, and deep learning has taken the front seat. As a data scientist, if you want to explore data abstraction layers, this book will be your guide. This book shows how this can be exploited in the real world with complex raw data and has been fully updated to the latest version of TensorFlow 1.x.

Throughout the book, you'll learn how to implement deep learning algorithms for machine learning systems and integrate them into your product offerings, including search, image recognition, and language processing. Additionally, you'll learn how to analyze and improve the performance of deep learning models. This can be done by comparing algorithms against benchmarks, along with machine intelligence, to learn from the information and determine ideal behaviors within a specific context.

After finishing the book, you will be familiar with machine learning techniques, in particular the use of TensorFlow for deep learning, and will be ready to apply your knowledge to research or commercial projects.

What you will learn

• Learn about machine learning landscapes along with the historical development and progress of deep learning
• Learn about deep machine intelligence and GPU computing with the latest TensorFlow 1.x
• Access public datasets and utilize them using TensorFlow to load, process, and transform data
• Use TensorFlow on real-world datasets, including images, text, and more
• Learn how to evaluate the performance of your deep learning models
• Using deep learning for scalable object detection and mobile computing
• Train machines quickly to learn from data by exploring reinforcement learning techniques
• Explore active areas of deep learning research and applications

Who This Book Is For

The book is intended for a general audience of people interested in machine learning and machine intelligence. A rudimentary level of programming in one language is assumed, as is a basic familiarity with computer science techniques and technologies, including a basic awareness of computer hardware and algorithms. Some competence in mathematics is needed to the level of elementary linear algebra and calculus.

✦ Table of Contents

Cover
Copyright
Packt Upsell
Contributors
Table of Contents
Preface
Chapter 1: Getting Started with Deep Learning
A soft introduction to machine learning
Supervised learning
Unbalanced data
Unsupervised learning
Reinforcement learning
What is deep learning?
Artificial neural networks
The biological neurons
The artificial neuron
How does an ANN learn?
ANNs and the backpropagation algorithm
Weight optimization
Stochastic gradient descent
Neural network architectures
Deep Neural Networks (DNNs)
Multilayer perceptron
Deep Belief Networks (DBNs)
Convolutional Neural Networks (CNNs)
AutoEncoders
Recurrent Neural Networks (RNNs)
Emergent architectures
Deep learning frameworks
Summary
Chapter 2: A First Look at TensorFlow
A general overview of TensorFlow
What's new in TensorFlow v1.6?
Nvidia GPU support optimized
Introducing TensorFlow Lite
Eager execution
Optimized Accelerated Linear Algebra (XLA)
Installing and configuring TensorFlow
TensorFlow computational graph
TensorFlow code structure
Eager execution with TensorFlow
Data model in TensorFlow
Tensor
Rank and shape
Data type
Variables
Fetches
Feeds and placeholders
Visualizing computations through TensorBoard
How does TensorBoard work?
Linear regression and beyond
Linear regression revisited for a real dataset
Summary
Chapter 3: Feed-Forward Neural Networks with TensorFlow
Feed-forward neural networks (FFNNs)
Feed-forward and backpropagation
Weights and biases
Activation functions
Using sigmoid
Using tanh
Using ReLU
Using softmax
Implementing a feed-forward neural network
Exploring the MNIST dataset
Softmax classifier
Implementing a multilayer perceptron (MLP)
Training an MLP
Using MLPs
Dataset description
Preprocessing
A TensorFlow implementation of MLP for client-subscription assessment
Deep Belief Networks (DBNs)
Restricted Boltzmann Machines (RBMs)
Construction of a simple DBN
Unsupervised pre-training
Supervised fine-tuning
Implementing a DBN with TensorFlow for client-subscription assessment
Tuning hyperparameters and advanced FFNNs
Tuning FFNN hyperparameters
Number of hidden layers
Number of neurons per hidden layer
Weight and biases initialization
Selecting the most suitable optimizer
GridSearch and randomized search for hyperparameters tuning
Regularization
Dropout optimization
Summary
Chapter 4: Convolutional Neural Networks
Main concepts of CNNs
CNNs in action
LeNet5
Implementing a LeNet-5 step by step
AlexNet
Transfer learning
Pretrained AlexNet
Dataset preparation
Fine-tuning implementation
VGG
Artistic style learning with VGG-19
Input images
Content extractor and loss
Style extractor and loss
Merger and total loss
Training
Inception-v3
Exploring Inception with TensorFlow
Emotion recognition with CNNs
Testing the model on your own image
Source code
Summary
Chapter 5: Optimizing TensorFlow Autoencoders
How does an autoencoder work?
Implementing autoencoders with TensorFlow
Improving autoencoder robustness
Implementing a denoising autoencoder
Implementing a convolutional autoencoder
Encoder
Decoder
Fraud analytics with autoencoders
Description of the dataset
Problem description
Exploratory data analysis
Training, validation, and testing set preparation
Normalization
Autoencoder as an unsupervised feature learning algorithm
Evaluating the model
Summary
Chapter 6: Recurrent Neural Networks
Working principles of RNNs
Implementing basic RNNs in TensorFlow
RNN and the long-term dependency problem
Bi-directional RNNs
RNN and the gradient vanishing-exploding problem
LSTM networks
GRU cell
Implementing an RNN for spam prediction
Data description and preprocessing
Developing a predictive model for time series data
Description of the dataset
Pre-processing and exploratory analysis
LSTM predictive model
Model evaluation
An LSTM predictive model for sentiment analysis
Network design
LSTM model training
Visualizing through TensorBoard
LSTM model evaluation
Human activity recognition using LSTM model
Dataset description
Workflow of the LSTM model for HAR
Implementing an LSTM model for HAR
Summary
Chapter 7: Heterogeneous and Distributed Computing
GPGPU computing
The GPGPU history
The CUDA architecture
The GPU programming model
The TensorFlow GPU setup
Update TensorFlow
GPU representation
Using a GPU
GPU memory management
Assigning a single GPU on a multi-GPU system
The source code for GPU with soft placement
Using multiple GPUs
Distributed computing
Model parallelism
Data parallelism
The distributed TensorFlow setup
Summary
Chapter 8: Advanced TensorFlow Programming
tf.estimator
Estimators
Graph actions
Parsing resources
Flower predictions
TFLearn
Installation
Titanic survival predictor
PrettyTensor
Chaining layers
Normal mode
Sequential mode
Branch and join
Digit classifier
Keras
Keras programming models
Sequential model
Functional API
Summary
Chapter 9: Recommendation Systems Using Factorization Machines
Recommendation systems
Collaborative filtering approaches
Content-based filtering approaches
Hybrid recommender systems
Model-based collaborative filtering
Movie recommendation using collaborative filtering
The utility matrix
Description of the dataset
Ratings data
Movies data
Users data
Exploratory analysis of the MovieLens dataset
Implementing a movie RE
Training the model with the available ratings
Inferencing the saved model
Generating the user-item table
Clustering similar movies
Movie rating prediction by users
Finding top k movies
Predicting top k similar movies
Computing user-user similarity
Evaluating the recommender system
Factorization machines for recommendation systems
Factorization machines
Cold-start problem and collaborative-filtering approaches
Problem definition and formulation
Dataset description
Workflow of the implementation
Preprocessing
Training the FM model
Improved factorization machines
Neural factorization machines
Dataset description
Using NFM for the movie recommendation
Summary
Chapter 10: Reinforcement Learning
The RL problem
OpenAI Gym
OpenAI environments
The env class
Installing and running OpenAI Gym
The Q-Learning algorithm
The FrozenLake environment
Deep Q-learning
Deep Q neural networks
The Cart-Pole problem
Deep Q-Network for the Cart-Pole problem
The Experience Replay method
Exploitation and exploration
The Deep Q-Learning training algorithm
Summary
Other Books You May Enjoy
Leave a review – let other readers know what you think
Index