Deep Learning and Computational Physics

✍ Scribed by Deep Ray; Orazio Pinti; Assad A. Oberai

• Publisher: Springer Nature Switzerland
• Year: 2024
• Tongue: English
• Leaves: 160
• Category: Library

✦ Table of Contents

Preface
Acknowledgements
Contents
About the Authors
1 Introduction
1.1 Computational Physics
1.2 Machine Learning
1.2.1 Examples of ML
1.2.2 Types of ML Algorithms Based Tasks
1.3 Artificial Intelligence, Machine Learning and Deep Learning
1.4 Machine Learning and Computational Physics
1.5 Computational Exercise
2 Introduction to Deep Neural Networks
2.1 MLP Architecture
2.2 Activation Functions
2.2.1 Linear Activation
2.2.2 Rectified Linear Unit (ReLU)
2.2.3 Leaky ReLU
2.2.4 Logistic Function
2.2.5 Tanh
2.2.6 Sine
2.3 Expressivity of a Network
2.3.1 Universal Approximation Results
2.4 Training, Validation and Testing of Neural Networks
2.5 Overfitting and How to Avoid It
2.5.1 Regularization
2.6 Gradient Descent
2.6.1 Convergence
2.7 Some Advanced Optimization Algorithms
2.7.1 Momentum Methods
2.7.2 Adam
2.7.3 Stochastic Optimization
2.8 Calculating Gradients Using Back-Propagation
2.9 Regression Versus Classification
2.10 Computational Exercise
2.10.1 Expressivity of Deep Neural Networks
2.10.2 Training an MLP for a Regression Problem
3 Residual Neural Networks
3.1 Vanishing Gradients in Deep Networks
3.2 ResNets
3.3 Connections with ODEs
3.4 Neural ODEs
4 Convolutional Neural Networks
4.1 Functions and Images
4.2 Convolutions of Functions
4.2.1 Example 1
4.2.2 Example 2
4.3 Discrete Convolutions
4.4 Connection to Finite Difference Approximations
4.5 Convolution Layers
4.5.1 Average and Max Pooling
4.5.2 Convolution for Inputs with Multiple Channels
4.6 Convolution Neural Network (CNN)
4.7 Transpose Convolution Layers
4.8 UpSampling
4.9 Image-to-Image Transformations
4.10 Computational Exercise: Convolutional Neural Networks (CNNs)
5 Solving PDEs with Neural Networks
5.1 Finite Difference Method
5.2 Spectral Collocation Method
5.3 Physics-Informed Neural Networks (PINNs)
5.4 Extending PINNs to a More General PDE
5.5 Error Analysis for PINNs
5.6 Data Assimilation Using PINNs
5.7 Some Existing PINN Formulations
5.8 Computational Exercise: Physics Informed Neural Networks (PINNs)
6 Operator Networks
6.1 Parametrized PDEs
6.2 Operators
6.3 Deep Operator Network (DeepONet) Architecture
6.3.1 Training DeepONets
6.3.2 Error Analysis for DeepONets
6.4 Physics-Informed DeepONets
6.5 DeepONets and Their Applications
6.6 Fourier Neural Operator (FNO)
6.6.1 Discretization of the Fourier Neural Operator
6.6.2 The Use of Fourier Transforms
6.7 Variationally Mimetic Operator Network (VarMiON)
6.7.1 Background
6.7.2 VarMiON Architecture
6.7.3 Training the VarMiON
6.7.4 Error Estimates of VarMiON Approximation
6.8 Mesh Graph Networks
6.8.1 Background
6.8.2 Architecture of MGNs
6.8.3 Training MGNs
6.9 Computational Exercise: Deep Operator Networks (DeepONets)
7 Generative Deep Learning
7.1 Generative Algorithms
7.2 Introductory Concepts in Probability
7.2.1 Random Variables
7.2.2 Cumulative Distribution Function
7.2.3 Probability Density Function
7.2.4 Examples of Important Random Variables
7.2.5 Expectation and Variance of RVs
7.2.6 Random Vectors
7.2.7 Joint Probability Density Function
7.2.8 Examples of Important Random Vectors
7.2.9 Expectation and Covariance of Random Vectors
7.2.10 Marginal and Conditional Probability Density Functions
7.3 Pure Generative Problem
7.3.1 GANs
7.3.2 Score-Based Diffusion Models
7.4 Conditional Generative Algorithms
7.4.1 Conditional GANs
7.4.2 Conditional Diffusion Models
Appendix References