Advanced Mathematics for AI

Linear Transformations and Rank

Matrices as transformations, span, basis and rank

Code: visualise a transformation

Eigenvalues and Eigenvectors

Definition, computation and meaning

Code: eigen-decomposition

Av = λv

Singular Value Decomposition

Decomposition and low-rank approximation

Code: compress an image with SVD

A = UΣVᵀ

Multivariable Calculus and Gradients

Partial derivatives and the gradient vector

Code: numerical gradients

∇f = [∂f/∂x₁, …, ∂f/∂xₙ]

The Chain Rule and Automatic Differentiation

How backpropagation follows from the chain rule

Code: a tiny autograd engine

∂L/∂x = ∂L/∂y · ∂y/∂x

Jacobians and Hessians

Second-order information and curvature

Code: compute a Hessian

Probability Distributions

Bernoulli, binomial, Poisson, normal, multivariate normal and others

Code: sample and plot distributions

Maximum Likelihood and MAP Estimation

Where common loss functions come from

Code: derive and fit a model by MLE

θ̂ = argmax Σ log p(xᵢ|θ)

Bayesian Inference

Priors, posteriors and uncertainty

Code: Bayesian coin estimation

Entropy, Cross-Entropy and KL Divergence

Measuring uncertainty and the difference between distributions

Code: compute all three

H(p,q) = −Σ p log q

Convex Optimisation

Convex functions, minima and why convexity helps

Interactive: convex and non-convex surfaces

Optimisers: SGD, Momentum, RMSProp and Adam

How modern optimisers adjust each step

Interactive: race the optimisers

θ ← θ − η·m̂ / (√v̂ + ε)

Constrained Optimisation

Lagrange multipliers and duality

Code: solve a constrained problem

L = f(x) + λg(x)

Numerical Stability

Overflow, underflow and stable softmax

Code: make a function stable

Module Review and Assessment

Derivations and a scored quiz

Graded problem set