Regression Metrics | AI Pathshala

Definition

Is an error of 5 good or bad?

Regression metrics summarise how far predictions are from true values. MAE is the mean absolute error; MSE the mean squared error; RMSE its square root, in the target's units; and R² the share of the target's variance explained by the model compared with predicting the mean. Residual plots show whether errors have structure the model missed.

In simple words

An error of 5 means nothing until you know the scale: 5 rupees on a phone price is tiny, 5 degrees on body temperature is huge. RMSE keeps the target's units, punishing big misses more. MAE treats every rupee of error equally. R² says how much better you are than always guessing the average.

Why it matters

The metric decides which model wins. A delivery app that hates long delays should look at RMSE; one that cares about typical error, MAE. Reporting R² alone hides whether errors are acceptable in absolute terms.

How it works

Metric	Units	Sensitive to outliers	Best value
MAE	Target units	Less	0
MSE	Squared units	More	0
RMSE	Target units	More	0
R²	None	Moderate	1 (can be negative)

The maths

Mean absolute error and root mean squared error

MAE = \frac{1}{n} i \sum ∣ y_{i} - \overset{y}{^}_{i} ∣, RMSE = \frac{1}{n} i \sum (y_{i} - \overset{y}{^}_{i})^{2}

Coefficient of determination

R^{2} = 1 - \frac{S S _{res}}{S S _{tot}} = 1 - \frac{\sum _{i} ( y _{i} - y ^ _{i} ) ^{2}}{\sum _{i} ( y _{i} - y ˉ ) ^{2}}

In code

All four metrics by hand and from scikit-learn, plus a simple residual check by quartile of the prediction.

regression_metrics.py

import numpy as np
from sklearn.datasets import load_diabetes
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
from sklearn.model_selection import train_test_split

X, y = load_diabetes(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0)
pred = LinearRegression().fit(X_tr, y_tr).predict(X_te)

mae = np.mean(np.abs(y_te - pred))
rmse = np.sqrt(np.mean((y_te - pred) ** 2))
r2 = 1 - np.sum((y_te - pred) ** 2) / np.sum((y_te - y_te.mean()) ** 2)
print(f"By hand:      MAE {mae:.1f}  RMSE {rmse:.1f}  R2 {r2:.3f}")
print(f"scikit-learn: MAE {mean_absolute_error(y_te, pred):.1f}  RMSE {np.sqrt(mean_squared_error(y_te, pred)):.1f}  R2 {r2_score(y_te, pred):.3f}")
print("Target range:", int(y.min()), "to", int(y.max()))

residuals = y_te - pred
for i, chunk in enumerate(np.array_split(np.argsort(pred), 4), start=1):
    print(f"Prediction quartile {i}: mean residual {residuals[chunk].mean():6.1f}")

Output

By hand:      MAE 44.6  RMSE 55.7  R2 0.393
scikit-learn: MAE 44.6  RMSE 55.7  R2 0.393
Target range: 25 to 346
Prediction quartile 1: mean residual    8.6
Prediction quartile 2: mean residual  -13.4
Prediction quartile 3: mean residual    8.5
Prediction quartile 4: mean residual  -25.2

Try it yourself

Choose the metric

Match each requirement to the metric that fits it best.

Report error in rupees, treating all errors equally
Big misses are much worse than small ones
Compare with always predicting the average
A smooth loss to optimise during training

Common mistakes

Reporting R² without RMSE or MAE, which hides the size of errors.
Comparing RMSE across datasets with different scales.
Ignoring residual patterns, which reveal a missing feature or curve.

Check yourself

1. What does R² = 0 mean?

2. Which is in the same units as the target?

3. Can R² be negative on a test set?

Summary

MAE and RMSE are in target units; RMSE punishes large errors more.
R² compares the model with predicting the mean.
Always check residuals for patterns.

Finished reading?

Sources: scikit-learn: Regression metrics · James et al., An Introduction to Statistical Learning