Definition
A test for a rare disease is 99% accurate, and you test positive. What is the chance you're actually sick? Most people, including many doctors, guess wrong.
Bayes' theorem tells you how to update the probability of a hypothesis after seeing evidence. The posterior equals the likelihood times the prior, divided by the total probability of the evidence.
In simple words
You start with a belief, then see a clue, and adjust. If a friend is late, you may first think traffic. If you then hear the metro has stopped, your belief changes. Bayes' theorem is the exact maths of that adjustment: how much to believe now, given what you believed before and how well the clue fits.
Why it matters
It powers Naive Bayes spam filters and Bayesian networks, and it's the base of MAP estimation and Bayesian machine learning. It also protects you from a very common error: ignoring how rare something is to begin with. That matters in fraud detection, medical AI and anywhere one class is rare.
How it works
| Word | Meaning | Disease example |
|---|---|---|
| Prior P(H) | What you believed before the evidence | 1 in 1,000 people have it |
| Likelihood P(E|H) | How likely the evidence is if the hypothesis is true | A sick person tests positive 99% of the time |
| Evidence P(E) | How common the evidence is overall | All positives, true and false |
| Posterior P(H|E) | What you should believe now | The chance you're sick, given a positive test |
The maths
- The hypothesis, such as "has the disease"
- The evidence, such as "tested positive"
- "Not H": the hypothesis is false
1 in 1,000 people have the disease. The test catches 99% of cases, and wrongly flags 1% of healthy people.
P(sick) = 0.001 P(healthy) = 0.999 P(+ | sick) = 0.99 P(+ | healthy) = 0.01 P(+) = 0.99×0.001 + 0.01×0.999 = 0.00099 + 0.00999 = 0.01098 P(sick | +) = 0.00099 / 0.01098 ≈ 0.09
Only about 9%, despite the "99% accurate" test. Out of 100,000 people, about 99 sick people test positive, but so do about 999 healthy people. Because the disease is rare, false alarms outnumber real cases.
In code
def posterior(prior, true_positive, false_positive):
evidence = true_positive * prior + false_positive * (1 - prior)
return true_positive * prior / evidence
print("Rare disease (1 in 1000):", round(posterior(0.001, 0.99, 0.01), 3))
print("Common disease (1 in 10):", round(posterior(0.1, 0.99, 0.01), 3))
# A second positive test: the first result becomes the new prior
first = posterior(0.001, 0.99, 0.01)
print("After a second positive test:", round(posterior(first, 0.99, 0.01), 3))Output
Rare disease (1 in 1000): 0.09 Common disease (1 in 10): 0.917 After a second positive test: 0.908
Common mistakes
- Base-rate neglect: ignoring the prior. This is the classic error, and the worked example shows how big it can be.
- Mixing up P(positive | sick) and P(sick | positive). They can be 99% and 9%.
Check yourself
Summary
- Posterior = likelihood × prior ÷ evidence.
- It updates a belief using new evidence.
- Never ignore the base rate: rare things stay fairly unlikely even after a good test.
Finished reading?
Sources: Deisenroth, Faisal & Ong, Mathematics for Machine Learning · Russell & Norvig, AI: A Modern Approach, ch. 12