Deep Learning with PyTorch

Tensors and Operations

Creating, indexing and reshaping tensors on CPU and GPU

Code: tensor exercises

Autograd

Automatic differentiation in PyTorch

Code: gradients by hand and by autograd

Building Models with nn.Module

Layers, parameters and forward methods

Code: a multi-layer network

Datasets and DataLoaders

Batching, shuffling and transforms

Code: a custom dataset

The Training Loop

Loss, optimiser, training and validation steps

Code: a complete training loop

Backpropagation in Detail

A full numerical example through a network

Code: verify gradients numerically

Weight Initialisation

Why it matters, and Xavier and He initialisation

Code: compare initialisations

Var(w) = 2 / n

Vanishing and Exploding Gradients

Causes, residual connections and gradient clipping

Code: diagnose a failing network

Regularisation in Deep Learning

Dropout, weight decay, data augmentation and early stopping

Code: reduce overfitting

Normalisation Layers

Batch, layer and group normalisation

Code: compare normalisations

Learning Rate Schedules

Warm-up, step and cosine schedules

Code: compare schedules

Debugging Training

Overfitting one batch, checking loss curves and common bugs

Lab: fix five broken training runs

GPU Training and Mixed Precision

Using GPUs efficiently, and lower-precision arithmetic

Code: mixed-precision training

Saving, Loading and Checkpointing

Resuming training and sharing models

Code: checkpoint and resume

Keras, TensorFlow and JAX

Other frameworks compared

Code: the same model in Keras

Module Review and Assessment

Coding tasks and a scored quiz

Graded assessment