Mastering PyTorch: Create and deploy deep learning models from CNNs to multimodal models, LLMs and beyond, 2nd Edition

✍ Scribed by Ashish Ranjan Jha

• Publisher: Packt Publishing
• Year: 2024
• Tongue: English
• Leaves: 559
• Series: EXPERT INSIGHT
• Edition: 2
• Category: Library

✦ Synopsis

Master advanced techniques and algorithms for machine learning with PyTorch using real-world examples

Updated for PyTorch 2.x, including integration with Hugging Face, mobile deployment, diffusion models, and graph neural networks

Purchase of the print or Kindle book includes a free eBook in PDF format
Key Features

• Understand how to use PyTorch to build advanced neural network models

• Get the best from PyTorch by working with Hugging Face, fastai, PyTorch Lightning, PyTorch Geometric, Flask, and Docker

• Unlock faster training with multiple GPUs and optimize model deployment using efficient inference frameworks
  Book Description

PyTorch is making it easier than ever before for anyone to build deep learning applications. This PyTorch deep learning book will help you uncover expert techniques to get the most from your data and build complex neural network models.

You'll build convolutional neural networks for image classification and recurrent neural networks and transformers for sentiment analysis. As you advance, you'll apply deep learning across different domains, such as music, text, and image generation using generative models, including diffusion models. You'll not only build and train your own deep reinforcement learning models in PyTorch but also learn to optimize model training using multiple CPUs, GPUs, and mixed-precision training. You'll deploy PyTorch models to production, including mobile devices. Finally, you'll discover the PyTorch ecosystem and its rich set of libraries. These libraries will add another set of tools to your deep learning toolbelt, teaching you how to use fastai for prototyping models to training models using PyTorch Lightning. You'll discover libraries for AutoML and explainable AI (XAI), create recommendation systems, and build language and vision transformers with Hugging Face.

By the end of this book, you'll be able to perform complex deep learning tasks using PyTorch to build smart artificial intelligence models.
What You Will Learn

• Implement text, vision, and music generating models using PyTorch

• Build a deep Q-network (DQN) model in PyTorch

• Deploy PyTorch models on mobile devices (Android and iOS)

• Become well-versed with rapid prototyping using PyTorch with fast.ai

• Perform neural architecture search effectively using AutoML

• Easily interpret machine learning models using Captum

• Design ResNets, LSTMs, and graph neural networks (GNNs)

• Create language and vision transformer models using Hugging Face
  Who this book is for

This deep learning with PyTorch book is for data scientists, machine learning engineers, machine learning researchers, and deep learning practitioners looking to implement advanced deep learning models using PyTorch. This book is ideal for those looking to switch from TensorFlow to PyTorch. Working knowledge of deep learning with Python is required.

✦ Table of Contents

Cover
Copyright
Contributors
Table of Contents
Preface
Chapter 1: Overview of Deep Learning Using PyTorch
A refresher on deep learning
Activation functions
Optimization schedule
Exploring the PyTorch library in contrast to TensorFlow
Tensor modules
PyTorch modules
torch.nn
torch.optim
torch.utils.data
Training a neural network using PyTorch
Summary
Reference list
Chapter 2: Deep CNN Architectures
Why are CNNs so powerful?
Evolution of CNN architectures
Developing LeNet from scratch
Using PyTorch to build LeNet
Training LeNet
Testing LeNet
Fine-tuning the AlexNet model
Using PyTorch to fine-tune AlexNet
Running a pretrained VGG model
Exploring GoogLeNet and Inception v3
Inception modules
1x1 convolutions
Global average pooling
Auxiliary classifiers
Inception v3
Discussing ResNet and DenseNet architectures
ResNet
DenseNet
Understanding EfficientNets and the future of CNN architectures
Summary
References
Chapter 3: Combining CNNs and LSTMs
Building a neural network with CNNs and LSTMs
Text encoding demo
Building an image caption generator using PyTorch
Downloading the image captioning datasets
Preprocessing caption (text) data
Preprocessing image data
Defining the image captioning data loader
Defining the CNN-LSTM model
Training the CNN-LSTM model
Generating image captions using the trained model
Summary
References
Chapter 4: Deep Recurrent Model Architectures
Exploring the evolution of recurrent networks
Types of recurrent neural networks
RNNs
Bidirectional RNNs
LSTMs
Extended and bidirectional LSTMs
Multi-dimensional RNNs
Stacked LSTMs
GRUs
Grid LSTMs
Gated orthogonal recurrent units
Training RNNs for sentiment analysis
Loading and preprocessing the text dataset
Instantiating and training the model
Building a bidirectional LSTM
Loading and preprocessing the text dataset
Instantiating and training the LSTM model
Discussing GRUs and attention-based models
GRUs and PyTorch
Attention-based models
Summary
References
Chapter 5: Advanced Hybrid Models
Building a transformer model for language modeling
Reviewing language modeling
Understanding the transformer model architecture
Defining a transformer model in PyTorch
Loading and processing the dataset
Training the transformer model
Developing a RandWireNN model from scratch
Understanding RandWireNNs
Developing RandWireNNs using PyTorch
Defining a training routine and loading data
Defining the randomly wired graph
Defining RandWireNN model modules
Transforming a random graph into a neural network
Training the RandWireNN model
Evaluating and visualizing the RandWireNN model
Summary
References
Chapter 6: Graph Neural Networks
Introduction to GNNs
Understanding the intuition behind GNNs
Using regular NNs on graph data – a thought experiment
Understanding the power of GNNs with computational graphs
Types of graph learning tasks
Understanding node-level tasks
Understanding edge-level tasks
Understanding graph-level tasks
Reviewing prominent GNN models
Understanding graph convolutions with GCNs
Using attention in graphs with GAT
Performing graph sampling with GraphSAGE
Building a GCN model using PyTorch Geometric
Loading and exploring the citation networks dataset
Building a simple NN-based node classifier
Building a GCN model for node classification
Training a GAT model with PyTorch Geometric
Summary
Reference list
Chapter 7: Music and Text Generation with PyTorch
Building a transformer-based text generator with PyTorch
Training the transformer-based language model
Saving and loading the language model
Using the language model to generate text
Using GPT models as text generators
Out-of-the-box text generation with GPT-2
Text generation strategies using PyTorch
Greedy search
Beam search
Top-k and top-p sampling
Text generation with GPT-3
Generating MIDI music with LSTMs using PyTorch
Loading the MIDI music data
Defining the LSTM model and training routine
Training and testing the music generation model
Summary
References
Chapter 8: Neural Style Transfer
Understanding how to transfer style between images
Implementing neural style transfer using PyTorch
Loading the content and style images
Loading and trimming the pretrained VGG19 model
Building the neural style transfer model
Training the style transfer model
Experimenting with the style transfer system
Summary
References
Chapter 9: Deep Convolutional GANs
Defining the generator and discriminator networks
Understanding the DCGAN generator and discriminator
Training a DCGAN using PyTorch
Defining the generator
Defining the discriminator
Loading the image dataset
Training loops for DCGANs
Using GANs for style transfer
Understanding the pix2pix architecture
Exploring the pix2pix generator
Exploring the pix2pix discriminator
Summary
References
Chapter 10: Image Generation Using Diffusion
Understanding image generation using diffusion
Understanding how diffusion works
Training a forward diffusion model
Performing reverse diffusion or denoising
Training a diffusion model for image generation
Loading the dataset using Hugging Face datasets
Processing the dataset using torchvision transforms
Adding noise to images using diffusers
Defining the UNet model
Training the UNet model
Defining the optimizer and learning schedule
Using Hugging Face Accelerate to accelerate training
Running the model training loop
Generating realistic anime images using (reverse) diffusion
Understanding text-to-image generation using diffusion
Encoding text input into an embedding vector
Ingesting additional text data in the (conditional) UNet model
Using the Stable Diffusion model to generate images from text
Summary
Reference list
Chapter 11: Deep Reinforcement Learning
Reviewing RL concepts
Types of RL algorithms
Model-based
Model-Free
Discussing Q-learning
Understanding deep Q-learning
Using two separate DNNs
Experience replay buffer
Building a DQN model in PyTorch
Initializing the main and target CNN models
Defining the experience replay buffer
Setting up the environment
Defining the CNN optimization function
Managing and running episodes
Training the DQN model to learn Pong
Summary
Reference list
Chapter 12: Model Training Optimizations
Distributed training with PyTorch
Training the MNIST model in a regular fashion
Training the MNIST model in a distributed fashion
Distributed training on GPUs with CUDA
Automatic mixed precision training
Regular model training on a GPU
Mixed precision training on a GPU
Summary
Reference list
Chapter 13: Operationalizing PyTorch Models into Production
Model serving in PyTorch
Creating a PyTorch model inference pipeline
Saving and loading a trained model
Building the inference pipeline
Building a basic model server
Writing a basic app using Flask
Using Flask to build our model server
Setting up model inference for Flask serving
Building a Flask app to serve model
Using a Flask server to run predictions
Creating a model microservice
Serving a PyTorch model using TorchServe
Installing TorchServe
Launching and using a TorchServe server
Exporting universal PyTorch models using TorchScript and ONNX
Understanding the utility of TorchScript
Model tracing with TorchScript
Model scripting with TorchScript
Running a PyTorch model in C++
Using ONNX to export PyTorch models
Serving PyTorch models in the cloud
Using PyTorch with AWS
Serving a PyTorch model using an AWS instance
Using TorchServe with Amazon SageMaker
Serving PyTorch models on Google Cloud
Serving PyTorch models with Azure
Working with Azure’s DSVMs
Discussing Azure Machine Learning Service
Summary
Reference list
Chapter 14: PyTorch on Mobile Devices
Deploying a PyTorch model on Android
Converting the PyTorch model to a mobile-friendly format
Setting up the Android app development environment
Using the phone camera in the Android app to capture images
Enabling the camera during app startup
Handling camera permissions in Android
Opening the camera for image capture
Capturing images using the phone camera
Running ML model inference on camera-captured images
Validating the ML model binary path
Performing image classification on camera-captured images
Launching the app on an Android mobile device
Building PyTorch apps on iOS
Setting up the iOS development environment
Using a phone camera in the iOS app to capture images
Running ML model inference on camera-captured images
Summary
Reference list
Chapter 15: Rapid Prototyping with PyTorch
Using fastai to set up model training in a few minutes
Setting up fastai and loading data
Training an MNIST model using fastai
Evaluating and interpreting the model using fastai
Training models on any hardware using PyTorch Lightning
Defining the model components in PyTorch Lightning
Training and evaluating the model using PyTorch Lightning
Profiling MNIST model inference using PyTorch Profiler
Profiling on a CPU
Profiling model inference on the GPU
Visualizing model profiling results
Summary
Reference list
Chapter 16: PyTorch and AutoML
Finding the best neural architectures with AutoML
Using Auto-PyTorch for optimal MNIST model search
Loading the MNIST dataset
Running a neural architecture search with Auto-PyTorch
Visualizing the optimal AutoML model
Using Optuna for hyperparameter search
Defining the model architecture and loading the dataset
Defining the model training routine and optimization schedule
Running Optuna’s hyperparameter search
Summary
Reference list
Chapter 17: PyTorch and Explainable AI
Model interpretability in PyTorch
Training the handwritten digits classifier – a recap
Visualizing the convolutional filters of the model
Visualizing the feature maps of the model
Using Captum to interpret models
Setting up Captum
Exploring Captum’s interpretability tools
Summary
Reference List
Chapter 18: Recommendation Systems with PyTorch
Using deep learning for recommendation systems
Understanding a movie recommendation system dataset
Understanding embedding-based recommender systems
Understanding and processing the MovieLens dataset
Downloading the MovieLens dataset
Loading and analyzing the MovieLens dataset
Processing the MovieLens dataset
Creating the MovieLens dataloader
Training and evaluating a recommendation system model
Defining the EmbeddingNet architecture
Training EmbeddingNet
Evaluating the trained EmbeddingNet model
Building a recommendation system using the trained model
Summary
Reference list
Chapter 19: PyTorch and Hugging Face
Understanding Hugging Face within the PyTorch context
Exploring Hugging Face components relevant to PyTorch
Integrating Hugging Face with PyTorch
Using the Hugging Face Hub for pre-trained models
Using the Hugging Face Datasets library with PyTorch
Using Accelerate to speed up PyTorch model training
Using Optimum to optimize PyTorch model deployment
Summary
Reference list
Other Books You May Enjoy
Index