Definition
These are the words used throughout ML. A sample is one row of data. A feature is one input column, and the label is the output to predict. A hypothesis is the function a model has learned. Loss measures error on one example, while cost averages it across the data. A batch is a group of samples, and an epoch is one full pass over the training data.
In simple words
You have 1,000 flats in a spreadsheet. Each flat is a sample, area and floor are features, and price is the label. If training looks at 100 flats at a time, that's a batch of 100. It takes 10 batches to see all flats once, and that one full round is an epoch.
Why it matters
Every library, paper and interview uses these words. Mixing them up leads to wrong settings, such as confusing batch size with epochs.
How it works
| Term | Also called | Symbol |
|---|---|---|
| Sample | Example, row, instance | x⁽ⁱ⁾ |
| Feature | Input, attribute, column, predictor | xⱼ |
| Label | Target, output, y | y |
| Prediction | Output of the model | ŷ |
| Weight | Parameter, coefficient | w or θ |
| Learning rate | Step size | η or α |
Generalisation
The real goal of ML is generalisation: doing well on data the model has never seen. Doing well on training data alone means little, because a model could just memorise it.
The maths
- Loss on one example
- Cost: the average over the data
1,000 samples, batch size 32: Updates per epoch = ⌈1000 / 32⌉ = 32 Training for 10 epochs = 320 updates
Common mistakes
- Mixing up batch and epoch. A batch is a group, and an epoch is a full pass.
- Thinking lower training loss always means a better model. Check validation data.
Check yourself
Summary
- Sample: row. Feature: input. Label: answer.
- Batch: a group. Epoch: one full pass.
- Generalisation is the real goal.
Finished reading?