Probability Basics | AI Pathshala

Definition

Probability is a number between 0 and 1 that measures how likely an event is, where 0 means impossible and 1 means certain. When all outcomes are equally likely, it's the number of favourable outcomes divided by the total number of outcomes.

In simple words

Roll a die. There are 6 faces and 3 of them are even, so the chance of an even number is 3/6 = 0.5. A weather app saying 70% rain is a probability too: of many days like today, about 7 in 10 had rain.

Why it matters

AI lives with uncertainty. A classifier doesn't say "spam", it says "spam with probability 0.93". LLMs choose each next word from a probability distribution. Loss functions, Naive Bayes and Bayesian networks are all built on these rules.

How it works

Term	Meaning	Die example
Experiment	Something with an uncertain result	Rolling a die
Sample space (S)	All possible outcomes	{1, 2, 3, 4, 5, 6}
Event	A set of outcomes you care about	Rolling an even number

The rules

Every probability is between 0 and 1, and all outcomes together add to 1.
Complement: the chance something does not happen is 1 minus the chance it does.
Or: add the two chances, then subtract the overlap so it isn't counted twice.
And: for independent events, multiply the two chances.

Counting outcomes

Use permutations when order matters, such as a PIN. Use combinations when it doesn't, such as choosing a team.

The maths

Equally likely outcomes

P (A) = \frac{favourable outcomes}{total outcomes}

Complement

P (not A) = 1 - P (A)

Addition rule

P (A \cup B) = P (A) + P (B) - P (A \cap B)

$\cup$: Or: either event happens
$\cap$: And: both events happen

Independent events

P (A \cap B) = P (A) P (B)

Permutations and combinations

P (n, r) = \frac{n !}{( n - r )!}, C (n, r) = (r n) = \frac{n !}{r ! ( n - r )!}

Pick one card from a 52-card deck. What is the chance of a heart or a king?

P(heart) = 13/52
P(king)  = 4/52
P(both)  = 1/52   (the king of hearts)
P(heart or king) = 13/52 + 4/52 − 1/52 = 16/52 ≈ 0.308

In code

Simulating 100,000 rolls gets very close to the exact answer of 0.5.

dice.py

import math
import random

random.seed(1)
rolls = [random.randint(1, 6) for _ in range(100_000)]
even = sum(1 for roll in rolls if roll % 2 == 0)
print("Simulated P(even):", round(even / len(rolls), 3))

print("Ways to pick 2 people from 5, order ignored:", math.comb(5, 2))
print("Ways to arrange 2 people from 5, order matters:", math.perm(5, 2))

Output

Simulated P(even): 0.501
Ways to pick 2 people from 5, order ignored: 10
Ways to arrange 2 people from 5, order matters: 20

Common mistakes

Adding probabilities without subtracting the overlap.
Multiplying probabilities of events that aren't independent.
The gambler's fallacy: after 5 heads, the next toss of a fair coin is still 50/50.

Check yourself

1. What is the chance of rolling a number above 4 on a fair die?

2. Toss two fair coins. What is the chance of two heads?

3. P(rain) = 0.3. What is P(no rain)?

Summary

Probability runs from 0 to 1, and all outcomes add to 1.
Or: add and subtract the overlap. And (independent): multiply.
Order matters: permutation. Order doesn't: combination.

Finished reading?

Sources: Deisenroth, Faisal & Ong, Mathematics for Machine Learning