Lecture 1: Axioms of Probability

EECS 126 (UC Berkeley) · Spring 2026 · January 20, 2026

A probability space provides the mathematical framework for modeling random experiments. In this lecture we define the axioms of probability, introduce sample spaces and events, work through several foundational examples, and derive the birthday paradox bound.

Probability Spaces

Definition 1.1 (Probability space)

A probability space is a triple $(\Omega, \FF, \PP)$ where:

$\Omega$ is the sample space: the set of all possible outcomes of the experiment.
$\FF \subseteq 2^{\Omega}$ is a $\sigma$ -algebra of subsets of $\Omega$ , called events.
$\PP : \FF \to [0,1]$ is a probability measure satisfying the Kolmogorov axioms.

The elements $\omega \in \Omega$ are called base outcomes (or samples). A $\sigma$ -algebra $\FF$ is a family of subsets of $\Omega$ that is closed under countable unions, countable intersections, and complements, and that contains $\Omega$ itself. In many discrete settings we simply take $\FF = 2^{\Omega}$ (the power set), so that every subset of $\Omega$ is an event.

Example 1.1 (Tossing two fair coins)

Toss two fair coins. The sample space is

\Omega = \{HH,\; HT,\; TH,\; TT\}.

Each base outcome is equally likely, so we assign $\PP(\{\omega\}) = \tfrac{1}{4}$ for each $\omega \in \Omega$ . The event space is the power set $\FF = 2^{\Omega}$ , which has $|\FF| = 2^{|\Omega|} = 2^4 = 16$ elements.

Example 1.2 (Toss until heads)

Toss a coin repeatedly until the first heads. Then

\Omega = \{H,\; TH,\; TTH,\; TTTH,\; \ldots\}

is a countably infinite sample space.

Example 1.3 (Waiting for a bus)

Wait at a bus stop for the next bus, which arrives at some time in $[0, T]$ . Now

\Omega = [0, T]

is an uncountably infinite (continuous) sample space. In this setting the choice of $\FF$ becomes nontrivial; one typically takes $\FF$ to be the Borel $\sigma$ -algebra on $[0,T]$ .

Kolmogorov Axioms

Definition 1.2 (Kolmogorov axioms)

A function $\PP : \FF \to \RR$ is a probability measure on $(\Omega, \FF)$ if it satisfies:

Nonnegativity. $\PP(A) \ge 0$ for all $A \in \FF$ .
Normalization. $\PP(\Omega) = 1$ .
Countable additivity. If $A_1, A_2, \ldots \in \FF$ are pairwise disjoint ( $A_i \cap A_j = \varnothing$ for $i \ne j$ ), then $\PP\!\Bigl(\bigcup_{i \ge 1} A_i\Bigr) = \sum_{i \ge 1} \PP(A_i).$

Why countable additivity? In a continuous sample space such as $\Omega = [0,1]$ , every individual point has probability zero: $\PP(\{x\}) = 0$ . If we only had finite additivity, we could not recover $\PP([0,1]) = 1$ from the probabilities of individual points. Countable additivity bridges this gap and is essential for the theory of continuous distributions.

Example 1.4 (Biased coin)

Flip a biased coin: heads with probability $p$ , tails with probability $1 - p$ . Set

\Omega = \{H, T\}, \qquad \FF = 2^{\Omega} = \bigl\{\varnothing,\; \{H\},\; \{T\},\; \{H,T\}\bigr\}.

Define

\PP(H) = p, \quad \PP(T) = 1-p, \quad \PP(\varnothing) = 0, \quad \PP(\{H,T\}) = 1.

Then $(\Omega, \FF, \PP)$ is a valid probability space.

Fundamental Probability Facts

The following basic identities are direct consequences of the Kolmogorov axioms.

Theorem 1.1 (Complement rule)

For any event $A \in \FF$ ,

\PP(A^c) = 1 - \PP(A).

Proof.

Since $A$ and $A^c$ are disjoint with $A \cup A^c = \Omega$ , countable additivity gives $\PP(A) + \PP(A^c) = \PP(\Omega) = 1$ .

Theorem 1.2 (Inclusion–exclusion)

For any two events $A, B \in \FF$ ,

\PP(A \cup B) = \PP(A) + \PP(B) - \PP(A \cap B).

More generally, for events $A_1, A_2, \ldots, A_n$ ,

\begin{aligned} \PP\!\Bigl(\bigcup_{i=1}^{n} A_i\Bigr) = \sum_{i} \PP(A_i) &- \sum_{i < j} \PP(A_i \cap A_j) + \sum_{i < j < k} \PP(A_i \cap A_j \cap A_k) \\ &- \cdots + (-1)^{n+1}\, \PP(A_1 \cap \cdots \cap A_n). \end{aligned}

Theorem 1.3 (Union bound)

For any events $A_1, \ldots, A_n$ ,

\PP\!\Bigl(\bigcup_{i=1}^{n} A_i\Bigr) \;\le\; \sum_{i=1}^{n} \PP(A_i).

Proof.

Define $B_1 = A_1$ and $B_k = A_k \setminus \bigcup_{i<k} A_i$ for $k \ge 2$ . The $B_k$ are disjoint with $\bigcup B_k = \bigcup A_k$ , and $\PP(B_k) \le \PP(A_k)$ , so $\PP(\bigcup A_k) = \sum \PP(B_k) \le \sum \PP(A_k)$ .

Discrete Probability and Counting

In a discrete sample space, the probability of an event is just the sum of the probabilities of its base outcomes:

\PP(A) = \sum_{\omega \in A} \PP(\{\omega\}).

Definition 1.3 (Uniform sample space)

When all outcomes are equally likely, we say $\Omega$ is a uniform sample space, and for any event $A$ ,

\PP(A) = \frac{|A|}{|\Omega|}.

The Birthday Paradox

Example 1.5 (Birthday paradox)

Let $n$ people be in a room. What is the probability that at least two share a birthday? Let $K = 365$ and assume birthdays are uniform. Let $A$ = “at least two people share a birthday.” Compute via the complement:

\PP(A^c) = \frac{K(K-1)(K-2)\cdots(K-n+1)}{K^n} = \prod_{j=0}^{n-1}\Bigl(1 - \frac{j}{K}\Bigr).

Using $1 - x \le e^{-x}$ :

\PP(A^c) \le \prod_{j=0}^{n-1} e^{-j/K} = e^{-\sum_{j=0}^{n-1} j/K} = e^{-n(n-1)/(2K)}.

Theorem 1.4 (Birthday bound)

$\PP(A) \ge 1 - e^{-n(n-1)/(2K)}$ . For $K = 365$ and $n = 23$ , this gives $\PP(A) \ge 50\%$ .

Balls-and-bins interpretation. The birthday problem is equivalent to throwing $n$ balls uniformly into $K$ bins and asking for the probability of a collision. This connection arises frequently in hashing, load balancing, and networking.

axioms-of-probability.html