Deep Learning with Hadoop

✍ Scribed by Dipayan Dev

• Publisher: Packt Publishing
• Tongue: English
• Leaves: 200
• Category: Library

✦ Synopsis

Key Features

• Get to grips with the deep learning concepts and set up Hadoop to put them to use
• Implement and parallelize deep learning models on Hadoop's YARN framework
• A comprehensive tutorial to distributed deep learning with Hadoop

Book Description

This book will teach you how to deploy large-scale dataset in deep neural networks with Hadoop for optimal performance.

Starting with understanding what deep learning is, and what the various models associated with deep neural networks are, this book will then show you how to set up the Hadoop environment for deep learning. In this book, you will also learn how to overcome the challenges that you face while implementing distributed deep learning with large-scale unstructured datasets. The book will also show you how you can implement and parallelize the widely used deep learning models such as Deep Belief Networks, Convolutional Neural Networks, Recurrent Neural Networks, Restricted Boltzmann Machines and autoencoder using the popular deep learning library deeplearning4j.

Get in-depth mathematical explanations and visual representations to help you understand the design and implementations of Recurrent Neural network and Denoising AutoEncoders with deeplearning4j. To give you a more practical perspective, the book will also teach you the implementation of large-scale video processing, image processing and natural language processing on Hadoop.

By the end of this book, you will know how to deploy various deep neural networks in distributed systems using Hadoop.

What you will learn

• Explore Deep Learning and various models associated with it
• Understand the challenges of implementing distributed deep learning with Hadoop and how to overcome it
• Implement Convolutional Neural Network (CNN) with deeplearning4j
• Delve into the implementation of Restricted Boltzmann Machines (RBM)
• Understand the mathematical explanation for implementing Recurrent Neural Networks (RNN)
• Get hands on practice of deep learning and their implementation with Hadoop.

About the Author

Dipayan Dev has completed his M.Tech from National Institute of Technology, Silchar with a first class first and is currently working as a software professional in Bengaluru, India. He has extensive knowledge and experience in non-relational database technologies, having primarily worked with large-scale data over the last few years. His core expertise lies in Hadoop Framework. During his postgraduation, Dipayan had built an infinite scalable framework for Hadoop, called Dr. Hadoop, which got published in top-tier SCI-E indexed journal of Springer. Dr. Hadoop has recently been cited by Goo Wikipedia in their Apache Hadoop article. Apart from that, he registers interest in a wide range of distributed system technologies, such as Redis, Apache Spark, Elasticsearch, Hive, Pig, Riak, and other NoSQL databases. Dipayan has also authored various research papers and book chapters, which are published by IEEE and top-tier Springer Journals. To know more about him, you can also visit his LinkedIn profile dipayandev.

Table of Contents

1. Introduction to Deep Learning
2. Distributed Deep Learning for Large-Scale Data
3. Convolutional Neural Network
4. Recurrent Neural Network
5. Restricted Boltzmann Machines
6. Autoencoders
7. Miscellaneous Deep Learning Operations using Hadoop
8. References

✦ Table of Contents

Cover
Copyright
Credits
About the Author
About the Reviewers
www.PacktPub.com
Customer Feedback
Table of Contents
Preface
Chapter 1: Introduction to Deep Learning
Getting started with deep learning
Deep feed-forward networks
Various learning algorithms
Unsupervised learning
Supervised learning
Semi-supervised learning
Deep learning terminologies
Deep learning: A revolution in Artificial Intelligence
Motivations for deep learning
The curse of dimensionality
The vanishing gradient problem
Distributed representation
Classification of deep learning networks
Deep generative or unsupervised models
Deep discriminate models
Summary
Chapter 2: Distributed Deep Learning for Large-Scale Data
Deep learning for massive amounts of data
Challenges of deep learning for big data
Challenges of deep learning due to massive volumes of data (first V)
Challenges of deep learning from a high variety of data (second V)
Challenges of deep learning from a high velocity of data (third V)
Challenges of deep learning to maintain the veracity of data (fourth V)
Distributed deep learning and Hadoop
Map-Reduce
Iterative Map-Reduce
Yet Another Resource Negotiator (YARN)
Important characteristics for distributed deep learning design
Deeplearning4j – an open source distributed framework for deep learning
Major features of Deeplearning4j
Summary of functionalities of Deeplearning4j
Setting up Deeplearning4j on Hadoop YARN
Getting familiar with Deeplearning4j
Integration of Hadoop YARN and Spark for distributed deep learning
Rules to configure memory allocation for Spark on Hadoop YARN
Summary
Chapter 3: Convolutional Neural Network
Understanding convolution
Background of a CNN
Architecture overview
Basic layers of CNN
Importance of depth in a CNN
Convolutional layer
Sparse connectivity
Improved time complexity
Parameter sharing
Improved space complexity
Equivariant representations
Choosing the hyperparameters for Convolutional layers
Depth
Stride
Zero-padding
Mathematical formulation of hyperparameters
Effect of zero-padding
ReLU (Rectified Linear Units) layers
Advantages of ReLU over the sigmoid function
Pooling layer
Where is it useful, and where is it not?
Fully connected layer
Distributed deep CNN
Most popular aggressive deep neural networks and their configurations
Training time – major challenges associated with deep neural networks
Hadoop for deep CNNs
Convolutional layer using Deeplearning4j
Loading data
Model configuration
Training and evaluation
Summary
Chapter 4: Recurrent Neural Network
What makes recurrent networks distinctive from others?
Recurrent neural networks(RNNs)
Unfolding recurrent computations
Advantages of a model unfolded in time
Memory of RNNs
Architecture
Backpropagation through time (BPTT)
Error computation
Long short-term memory
Problem with deep backpropagation with time
Long short-term memory
Bi-directional RNNs
Shortfalls of RNNs
Solutions to overcome
Distributed deep RNNs
RNNs with Deeplearning4j
Summary
Chapter 5: Restricted Boltzmann Machines
Energy-based models
Boltzmann machines
How Boltzmann machines learn
Shortfall
Restricted Boltzmann machine
The basic architecture
How RBMs work
Convolutional Restricted Boltzmann machines
Stacked Convolutional Restricted Boltzmann machines
Deep Belief networks
Greedy layer-wise training
Distributed Deep Belief network
Distributed training of Restricted Boltzmann machines
Distributed training of Deep Belief networks
Distributed back propagation algorithm
Performance evaluation of RBMs and DBNs
Drastic improvement in training time
Implementation using Deeplearning4j
Restricted Boltzmann machines
Deep Belief networks
Summary
Chapter 6: Autoencoders
Autoencoder
Regularized autoencoders
Sparse autoencoders
Sparse coding
Sparse autoencoders
The k-Sparse autoencoder
How to select the sparsity level k
Effect of sparsity level
Deep autoencoders
Training of deep autoencoders
Implementation of deep autoencoders using Deeplearning4j
Denoising autoencoder
Architecture of a Denoising autoencoder
Stacked denoising autoencoders
Implementation of a stacked denoising autoencoder using Deeplearning4j
Applications of autoencoders
Summary
Chapter 7: Miscellaneous Deep Learning Operations using Hadoop
Distributed video decoding in Hadoop
Large-scale image processing using Hadoop
Application of Map-Reduce jobs
Natural language processing using Hadoop
Web crawler
Extraction of keyword and module for natural language processing
Estimation of relevant keywords from a page
Summary
References
Index