Training vs inference | AI Pathshala

Definition

Training is the phase in which a model learns, by adjusting its parameters using data. Inference is the phase in which the trained model is used to produce an output for new input. Training happens occasionally and is expensive, and inference happens every time someone uses the model.

In simple words

Years of study to become a doctor is training. Each patient you later see is inference. Studying is long and costly, while one consultation is quick. But over a career, many thousands of consultations add up.

Why it matters

It decides your costs and hardware. A startup may never train a large model, and only pay for inference through an API. It also explains why a model doesn't know events after its training cutoff unless it's given that information.

How it works

	Training	Inference
Goal	Learn the parameters	Use the parameters
Do parameters change?	Yes	No
Maths	Forward pass, loss, backward pass, update	Forward pass only
Data	Large, with labels	One input, or a small batch
Hardware	Many GPUs, for large models	GPU, CPU or even a phone
Happens	Once, or at each retrain	With every request

In code

train_vs_infer.py

import time
from sklearn.datasets import load_digits
from sklearn.neural_network import MLPClassifier

X, y = load_digits(return_X_y=True)
model = MLPClassifier(hidden_layer_sizes=(64,), max_iter=200, random_state=0)

start = time.perf_counter()
model.fit(X, y)                       # training: parameters change
train_time = time.perf_counter() - start

start = time.perf_counter()
model.predict(X[:1])                  # inference: parameters stay fixed
infer_time = time.perf_counter() - start

print("Training took", "much longer" if train_time > 100 * infer_time else "longer", "than one prediction")
print("Number of parameters:", sum(w.size for w in model.coefs_) + sum(b.size for b in model.intercepts_))

Output

Training took much longer than one prediction
Number of parameters: 4810

Common mistakes

Assuming a model keeps learning as it's used.
Budgeting only for training. For a popular product, inference usually costs more over time.

Check yourself

1. During inference, the model's parameters:

2. Why might a model not know last week's news?

Summary

Training learns the parameters, and inference uses them.
Training is rare and costly. Inference is frequent and cheap per request.
Parameters don't change at inference.