Definition
The model got 8 out of 10 test pictures right. How good is that?
Accuracy is the share of predictions a model gets right on a test set, usually given as a percentage. If a model is right on 8 of 10 test examples, its accuracy is 80 per cent.
In simple words
Accuracy is like your score in a test: count the right answers, divide by the number of questions, and turn it into a percentage. 8 out of 10 is 80 per cent; 45 out of 50 is 90 per cent. But just like a test, the score only means something if the questions were fair and new.
Why it matters
Accuracy is the first number people check to decide whether a model is good enough. It also lets you compare two models, or the same model before and after you improve it.
How it works
| Right answers | Test examples | Accuracy |
|---|---|---|
| 8 | 10 | 80 per cent |
| 45 | 50 | 90 per cent |
| 6 | 9 | About 67 per cent |
Try it yourself
Mark the model's test
Ten pictures the model has never seen. How many did it get right?
Score a model on ten test pictures
For each test picture, decide whether the model was right. Then count: what is its accuracy?
Picture of a dog. Model said: dog
Picture of a cat. Model said: cat (4th)
Picture of a cat. Model said: dog
Picture of a dog. Model said: dog (4th)
Picture of a cat. Model said: cat
Picture of a cat. Model said: cat (2nd)
Picture of a small dog. Model said: cat
Picture of a cat. Model said: cat (3rd)
Picture of a dog. Model said: dog (2nd)
Picture of a dog. Model said: dog (3rd)
Common mistakes
- Measuring accuracy on training examples instead of test examples.
- Trusting high accuracy when one answer is far more common than the other.
- Ignoring which mistakes were made. Some mistakes matter more than others.
Check yourself
Summary
- Accuracy is the percentage of correct predictions on a test set.
- Count the right answers and compare with the total.
- High accuracy can mislead when one answer is much more common.
Finished reading?
Sources: Google Machine Learning Crash Course