Acknowledgements
Copyright
License for Code
Readme
Introducing Neural Networks
A Brief History
What is a Neural Network?
Coding Our First Neurons
A Single Neuron
A Layer of Neurons
Tensors, Arrays and Vectors
Dot Product and Vector Addition
A Single Neuron with NumPy
A Batch of Data
Matrix Product
Transposition for the Matrix Product
A Layer of Neurons & Batch of Data w/ NumPy
Adding Layers
Training Data
Dense Layer Class
Full code up to this point:
Activation Functions
The Step Activation Function
The Linear Activation Function
The Sigmoid Activation Function
The Rectified Linear Activation Function
Why Use Activation Functions?
Linear Activation in the Hidden Layers
ReLU Activation in a Pair of Neurons
ReLU Activation in the Hidden Layers
ReLU Activation Function Code
The Softmax Activation Function
Full code up to this point:
Calculating Network Error with Loss
Categorical Cross-Entropy Loss
The Categorical Cross-Entropy Loss Class
Combining everything up to this point:
Accuracy Calculation
Introducing Optimization
Full code up to this point:
Derivatives
The Impact of a Parameter on the Output
The Slope
The Numerical Derivative
The Analytical Derivative
Summary
Gradients, Partial Derivatives, and the Chain Rule
The Partial Derivative
The Partial Derivative of a Sum
The Partial Derivative of Multiplication
The Partial Derivative of Max
The Gradient
The Chain Rule
Summary
Backpropagation
Categorical Cross-Entropy loss derivative
Categorical Cross-Entropy loss derivative code implementation
Softmax activation derivative
Softmax activation derivative code implementation
Common Categorical Cross-Entropy loss and Softmax activation derivative
Common Categorical Cross-Entropy loss and Softmax activation derivative - code implementation
Full code up to this point:
Optimizers
Stochastic Gradient Descent (SGD)
Learning Rate
Learning Rate Decay
Stochastic Gradient Descent with Momentum
AdaGrad
RMSProp
Adam
Full code up to this point:
Testing with Out-of-Sample Data
Validation Data
Training Dataset
L1 and L2 Regularization
Forward Pass
Backward pass
Dropout
Forward Pass
Backward Pass
The Code
Binary Logistic Regression
Sigmoid Activation Function
Sigmoid Function Derivative
Sigmoid Function Code
Binary Cross-Entropy Loss
Binary Cross-Entropy Loss Derivative
Binary Cross-Entropy Code
Implementing Binary Logistic Regression and Binary Cross-Entropy Loss
Full code up to this point:
Regression
Linear Activation
Mean Squared Error Loss
Mean Squared Error Loss Derivative
Mean Squared Error (MSE) Loss Code
Mean Absolute Error Loss
Mean Absolute Error Loss Derivative
Mean Absolute Error Loss Code
Accuracy in Regression
Regression Model Training
Full code up to this point:
Model Object
Full code up to this point:
A Real Dataset
Data preparation
Data loading
Data preprocessing
Data Shuffling
Batches
Training
Full code up to now:
Model Evaluation
Saving and Loading Models and Their Parameters
Retrieving Parameters
Setting Parameters
Saving Parameters
Loading Parameters
Saving the Model
Loading the Model
Prediction / Inference
Full code:
Closing