Precision, Recall and F1 Score

Definition

Precision is the share of positive predictions that were correct. Recall is the share of actual positives that were found. F1 is the harmonic mean of the two, so it's only high when both are high.

In simple words

A fisherman casts a net. Precision asks: of what's in the net, how much is fish, and not junk? Recall asks: of all the fish in the lake, how many did I catch? A tiny net gives high precision and low recall. A huge net gives high recall and low precision.

Why it matters

Choosing the wrong metric means optimising for the wrong thing. A search engine or spam filter needs precision, because false alarms annoy people. Disease screening needs recall, because missing cases is dangerous.

How it works

Use case	Priority	Why
Spam filter	Precision	Losing a real email hurts
Cancer screening	Recall	Missing a patient is worst
Fraud detection	Balance, by cost	Both errors cost money
Search results	Precision at top	People look at the first few

The trade-off

Lowering the threshold catches more positives, so recall goes up, but more false alarms slip in, so precision falls. You can't usually maximise both, and you choose the balance based on what each error costs.

The maths

Precision

Precision = \frac{TP}{TP + FP}

Recall (sensitivity)

Recall = \frac{TP}{TP + FN}

F1 score

F_{1} = 2 \cdot \frac{Precision \cdot Recall}{Precision + Recall}

Using TP = 8, FP = 5, FN = 2 from the screening example:

Precision = 8 / (8 + 5) = 0.615
Recall    = 8 / (8 + 2) = 0.800
F1 = 2 × (0.615 × 0.8) / (0.615 + 0.8) = 0.696

In code

metrics.py

from sklearn.metrics import classification_report, f1_score, precision_score, recall_score

actual =    [1] * 10 + [0] * 90
predicted = [1] * 8 + [0] * 2 + [1] * 5 + [0] * 85

print("Precision:", round(precision_score(actual, predicted), 3))
print("Recall:", round(recall_score(actual, predicted), 3))
print("F1:", round(f1_score(actual, predicted), 3))
print(classification_report(actual, predicted, target_names=["healthy", "sick"]))

Output

Precision: 0.615
Recall: 0.8
F1: 0.696
              precision    recall  f1-score   support

     healthy       0.98      0.94      0.96        90
        sick       0.62      0.80      0.70        10

    accuracy                           0.93       100
   macro avg       0.80      0.87      0.83       100
weighted avg       0.94      0.93      0.93       100

Common mistakes

Reporting one without the other. A model that flags everyone has perfect recall and is useless.
Using F1 when errors don't cost the same. F1 treats precision and recall as equally important.

Check yourself

1. TP = 30, FP = 10, FN = 20. What's the precision?

2. And the recall for those numbers?

3. Why does F1 use the harmonic mean?

Summary

Precision: of those flagged, how many were right?
Recall: of the real ones, how many did you find?
F1 combines them. Choose based on what errors cost.

Finished reading?

Sources: Google Machine Learning Crash Course