Conditional Probability

Definition

In simple words

The chance a random person is carrying an umbrella may be 10%. The chance given that it's raining is far higher. New information changes the probability, because it shrinks the world you're looking at: now you count only rainy days.

Why it matters

Nearly every prediction is conditional: the chance of spam given these words, or the next word given the sentence so far. Classifiers estimate P(label | features) directly. Naive Bayes and Bayesian networks are built entirely on this idea.

How it works

Out of 100 emails, 30 are spam. The word "free" appears in 25 emails, and 20 of those are spam.

P(spam) is 30/100 = 0.30. But P(spam | free) looks only at the 25 emails with "free", so it is 20/25 = 0.80. Seeing that one word moved the chance from 30% to 80%.

The maths

P(spam ∩ free) = 20/100 = 0.20
P(free)        = 25/100 = 0.25
P(spam | free) = 0.20 / 0.25 = 0.80

In code

emails = [("spam", True)] * 20 + [("spam", False)] * 10 + [("safe", True)] * 5 + [("safe", False)] * 65

with_free = [label for label, has_free in emails if has_free]
print("P(spam):", sum(label == "spam" for label, _ in emails) / len(emails))
print("P(spam | free):", with_free.count("spam") / len(with_free))

P(spam): 0.3
P(spam | free): 0.8

Common mistakes

• Confusing P(A|B) with P(B|A).
• Dividing by the total. Divide by the count of the condition, because the condition shrinks the world.

Check yourself

Summary

• P(A|B) is the chance of A once you know B.
• Formula: P(A and B) divided by P(B).
• Independent means knowing one doesn't change the other.