Definition
Would you trust a model you never tested?
Testing a model means checking its predictions on new examples it did not see during training, and comparing them with the right answers. The examples kept aside for this are called the test set.
In simple words
If a teacher gave you the exact exam questions to practise, a perfect score wouldn't prove you understood. A fair exam uses new questions. In the same way, a model is tested only on examples it has never seen, so the result shows whether it really learned the pattern.
Why it matters
A model that is only checked on its training examples can look perfect and then fail in real use. Testing on new examples is how engineers find out whether a model is ready, and where it goes wrong, before people rely on it.
How it works
- Before training, put some examples aside. The model must never see them.
- Train the model on the rest.
- Ask the model about each test example.
- Mark each prediction right or wrong.
- Count the right ones and look closely at the wrong ones.
Try it yourself
Test your model with new animals
This model is already trained. Check its answer for each new animal, then test all nine and count how many it got right.
Common mistakes
- Testing on the training examples. That measures memory, not learning.
- Testing on too few examples to be sure.
- Looking only at the score and never at the mistakes.
Check yourself
Summary
- Test a model on new examples it has never seen.
- Keep the test set aside before training starts.
- Study the mistakes: they show what to improve.
Finished reading?
Sources: Google Machine Learning Crash Course