Hands-On Natural Language Processing with PyTorch 1.x: Build smart, AI-driven linguistic applications using deep learning and NLP techniques

✍ Scribed by Thomas Dop

• Publisher: Packt Publishing
• Tongue: English
• Leaves: 277
• Category: Library

✦ Synopsis

Become a proficient NLP data scientist by developing deep learning models for NLP and extract valuable insights from structured and unstructured data

Key Features

• Get to grips with word embeddings, semantics, labeling, and high-level word representations using practical examples
• Learn modern approaches to NLP and explore state-of-the-art NLP models using PyTorch
• Improve your NLP applications with innovative neural networks such as RNNs, LSTMs, and CNNs

Book Description

In the internet age, where an increasing volume of text data is generated daily from social media and other platforms, being able to make sense of that data is a crucial skill. With this book, you'll learn how to extract valuable insights from text by building deep learning models for natural language processing (NLP) tasks.

Starting by understanding how to install PyTorch and using CUDA to accelerate the processing speed, you'll explore how the NLP architecture works with the help of practical examples. This PyTorch NLP book will guide you through core concepts such as word embeddings, CBOW, and tokenization in PyTorch. You'll then learn techniques for processing textual data and see how deep learning can be used for NLP tasks. The book demonstrates how to implement deep learning and neural network architectures to build models that will allow you to classify and translate text and perform sentiment analysis. Finally, you'll learn how to build advanced NLP models, such as conversational chatbots.

By the end of this book, you'll not only have understood the different NLP problems that can be solved using deep learning with PyTorch, but also be able to build models to solve them.

What you will learn

• Use NLP techniques for understanding, processing, and generating text
• Understand PyTorch, its applications and how it can be used to build deep linguistic models
• Explore the wide variety of deep learning architectures for NLP
• Develop the skills you need to process and represent both structured and unstructured NLP data
• Become well-versed with state-of-the-art technologies and exciting new developments in the NLP domain
• Create chatbots using attention-based neural networks

Who this book is for

This PyTorch book is for NLP developers, machine learning and deep learning developers, and anyone interested in building intelligent language applications using both traditional NLP approaches and deep learning architectures. If you're looking to adopt modern NLP techniques and models for your development projects, this book is for you. Working knowledge of Python programming, along with basic working knowledge of NLP tasks, is required.

Table of Contents

1. Fundamentals of Machine Learning and Deep Learning
2. Getting Started with PyTorch 1.x for NLP
3. NLP and Text Embeddings
4. Text Preprocessing, Stemming, and Lemmatization
5. Recurrent Neural Networks and Sentiment Analysis
6. Convolutional Neural Networks for Text Classification
7. Text Translation using Sequence to Sequence Neural Networks
8. Building a Chatbot Using Attention-based Neural Networks
9. The Road Ahead

✦ Table of Contents

Cover
Title Page
Copyright and Credits
Dedication
About Packt
Contributors
Table of Contents
Preface
Section 1: Essentials of PyTorch 1.x for NLP
Chapter 1: Fundamentals of Machine Learning and Deep Learning
Overview of machine learning
Supervised learning
Unsupervised learning
How do models learn?
Neural networks
Structure of neural networks
Activation functions
How do neural networks learn?
Overfitting in neural networks
NLP for machine learning
Bag-of-words
Sequential representation
Summary
Chapter 2: Getting Started with PyTorch 1.x for NLP
Technical requirements
Installing and using PyTorch 1.x
Tensors
Enabling PyTorch acceleration using CUDA
Comparing PyTorch to other deep learning frameworks
Building a simple neural network in PyTorch
Loading the data
Building the classifier
Implementing dropout
Defining the forward pass
Setting the model parameters
Training our network
Making predictions
Evaluating our model
NLP for PyTorch
Setting up the classifier
Training the classifier
Summary
Section 2: Fundamentals of Natural Language Processing
In this section, you will learn about the fundamentals of building a Natural Language Processing (NLP) application. You will also learn how to use various NLP techniques, such as word embeddings, CBOW, and tokenization in PyTorch in this section.
Chapter 3: NLP and Text Embeddings
Technical requirements
Embeddings for NLP
GLoVe
Embedding operations
Exploring CBOW
CBOW architecture
Building CBOW
Exploring n-grams
N-gram language modeling
Tokenization
Tagging and chunking for parts of speech
Tagging
Chunking
TF-IDF
Calculating TF-IDF
Implementing TF-IDF
Calculating TF-IDF weighted embeddings
Summary
Chapter 4: Text Preprocessing, Stemming, and Lemmatization
Technical requirements
Text preprocessing
Removing HTML
Converting text into lowercase
Removing punctuation
Replacing numbers
Stemming and lemmatization
Stemming
Lemmatization
Uses of stemming and lemmatization
Differences in lemmatization and stemming
Summary
Section 3: Real-World NLP Applications Using PyTorch 1.x
Chapter 5: Recurrent Neural Networks and Sentiment Analysis
Technical requirements
Building RNNs
Using RNNs for sentiment analysis
Exploding and shrinking gradients
Introducing LSTMs
Working with LSTMs
LSTM cells
Bidirectional LSTMs
Building a sentiment analyzer using LSTMs
Preprocessing the data
Model architecture
Training the model
Using our model to make predictions
Deploying the application on Heroku
Introducing Heroku
Creating an API using Flask – file structure
Creating an API using Flask – API file
Creating an API using Flask – hosting on Heroku
Summary
Chapter 6: Convolutional Neural Networks for Text Classification
Technical requirements
Exploring CNNs
Convolutions
Convolutions for NLP
Building a CNN for text classification
Defining a multi-class classification dataset
Creating iterators to load the data
Constructing the CNN model
Training the CNN
Making predictions using the trained CNN
Summary
Chapter 7: Text Translation Using Sequence-to-Sequence Neural Networks
Technical requirements
Theory of sequence-to-sequence models
Encoders
Decoders
Using teacher forcing
Building a sequence-to-sequence model for text translation
Preparing the data
Building the encoder
Building the decoder
Constructing the full sequence-to-sequence model
Training the model
Evaluating the model
Next steps
Summary
Chapter 8: Building a Chatbot Using Attention-Based Neural Networks
Technical requirements
The theory of attention within neural networks
Comparing local and global attention
Building a chatbot using sequence-to-sequence neural networks with attention
Acquiring our dataset
Processing our dataset
Creating the vocabulary
Loading the data
Removing rare words
Transforming sentence pairs to tensors
Constructing the model
Defining the training process
Defining the evaluating process
Training the model
Summary
Chapter 9: The Road Ahead
Exploring state-of-the-art NLP machine learning
BERT
BERT–Architecture
Applications of BERT
GPT-2
Comparing self-attention and masked self-attention
GPT-2 – Ethics
Future NLP tasks
Constituency parsing
Semantic role labeling
Textual entailment
Machine comprehension
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index