Supervised Machine Learning in Depth

The End-to-End Supervised Workflow

Problem framing, baselines, splitting, training, evaluation and iteration

Code: a complete workflow on real data

Data Preprocessing

Scaling, encoding, imputing missing values and handling outliers

Code: a ColumnTransformer for mixed data

x′ = (x − μ) / σ

Data Leakage and Validation Design

Train, validation and test sets, and the ways leakage creeps in

Code: catch three leakage bugs

Machine Learning Pipelines

Chaining preprocessing and models so nothing leaks

Code: a scikit-learn Pipeline

Linear Regression and Least Squares

The model, the cost function and the normal equation

Code: fit from scratch with NumPy

w = (XᵀX)⁻¹Xᵀy

Gradient Descent for Linear Regression

Deriving and running the update rule

Interactive: watch the line learn

w ← w − η·(2/n)Xᵀ(Xw − y)

Multiple and Polynomial Regression

Many features, curved relationships and choosing the degree

Code: degree compared with validation error

ŷ = w₀ + w₁x + w₂x²

Regularisation: Ridge, Lasso and Elastic Net

Penalising large weights to reduce overfitting

Interactive: change λ and watch the weights

J = MSE + λΣw² , J = MSE + λΣ|w|

Regression Metrics

MAE, MSE, RMSE, R² and residual analysis

Code: residual plots

R² = 1 − SS_res / SS_tot

Logistic Regression

The sigmoid, log loss and the decision boundary

Code: from scratch, then scikit-learn

L = −[y log p + (1 − y) log(1 − p)]

k-Nearest Neighbours and Distance Metrics

Choosing k, distance measures and the curse of dimensionality

Interactive: k and the decision boundary

d = √Σ(xᵢ − yᵢ)²

Naive Bayes Classifiers

Gaussian, multinomial and Bernoulli variants, and Laplace smoothing

Code: a text classifier from scratch

P(c|x) ∝ P(c) Π P(xᵢ|c)

Support Vector Machines and Kernels

Maximum margin, soft margins and the kernel trick

Interactive: linear and RBF kernels

min ½‖w‖² + CΣξᵢ

Decision Trees

Gini, entropy, information gain, CART and pruning

Code: a tree from scratch

Gini = 1 − Σpₖ²

Bagging and Random Forests

Bootstrap samples, feature randomness and out-of-bag error

Code: forest compared with a single tree

Boosting: AdaBoost and Gradient Boosting

Learning from the previous model's mistakes

Interactive: boosting step by step

Fₘ = Fₘ₋₁ + η·hₘ

XGBoost, LightGBM and CatBoost

Modern gradient boosting libraries and when to use each

Code: tune a gradient-boosted model

Stacking and Voting Ensembles

Combining different models

Code: a stacked ensemble

Classification Metrics in Depth

Confusion matrix, precision, recall, F1 and specificity

Code: metrics for every threshold

P = TP/(TP+FP) , R = TP/(TP+FN)

ROC-AUC and Precision-Recall Curves

Comparing models across all thresholds

Code: plot both curves

Imbalanced Data and Threshold Tuning

Resampling, class weights and choosing a threshold by cost

Code: fraud detection with 1 per cent positives

Cross-Validation Strategies

k-fold, stratified, grouped and time-series splits

Code: compare strategies

The Bias-Variance Trade-off

Decomposing error, learning curves and model complexity

Interactive: bias and variance by model size

Error = Bias² + Variance + Noise

Hyperparameter Tuning

Grid, random and Bayesian search

Code: tune with Optuna

Model Interpretability

Permutation importance, partial dependence and SHAP values

Code: explain a model's predictions

Project: An End-to-End Tabular Prediction System

From raw data to a tuned, explained and documented model

Project notebook with a written report

Module Review and Assessment

Derivations, coding tasks and a scored quiz

Graded assessment