Lecture 2: Conditional Probability and Independence

EECS 126 (UC Berkeley) · Spring 2026 · January 22, 2026

In this lecture we introduce conditional probability, derive the multiplication (chain) rule, state the law of total probability and Bayes’ rule, work through the classic false-positive medical testing example, and then develop the theory of independence and conditional independence.

Conditional Probability

Definition 2.1 (Conditional probability)

Let $A, B$ be events with $\PP(B) > 0$ . The conditional probability of $A$ given $B$ is

\PP(A \mid B) = \frac{\PP(A \cap B)}{\PP(B)}.

Intuitively, $\PP(A \mid B)$ is the probability that $A$ occurs when we know that $B$ has occurred. Conditioning on $B$ restricts the sample space from $\Omega$ to $B$ ; the event $B$ becomes the new “ $\Omega$ ” for $A$ .

In a uniform sample space, $\PP(A) = |A|/|\Omega|$ . Conditioning on $B$ gives $\PP(A \mid B) = |A \cap B|/|B|$ , which is simply the fraction of outcomes in $B$ that also lie in $A$ .

Example 2.1 (Two dice – conditioning on the sum)

Roll two standard dice. Let $A = \{\text{1st die is 6}\}$ , $B = \{\text{sum of two dice is 7}\}$ , and $C = \{\text{sum of two dice is 11}\}$ .

Computing $\PP(A \mid B)$ : The outcomes in $B$ (sum = 7) are $\{(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)\}$ , so $\PP(B) = 6/36$ . The only outcome in $A \cap B$ is $(6,1)$ , so $\PP(A \cap B) = 1/36$ . Therefore

\PP(A \mid B) = \frac{1/36}{6/36} = \frac{1}{6}.

Computing $\PP(A \mid C)$ : The outcomes in $C$ (sum = 11) are $\{(5,6),(6,5)\}$ , so $\PP(C) = 2/36$ . The only outcome in $A \cap C$ is $(6,5)$ , so

\PP(A \mid C) = \frac{1/36}{2/36} = \frac{1}{2}.

Multiplication Rule (Chain Rule)

Theorem 2.1 (Chain rule of probability)

For any events $A_1, A_2, \ldots, A_n$ with $\PP(A_1 \cap A_2 \cap \cdots \cap A_{n-1}) > 0$ ,

\PP(A_1 \cap A_2 \cap \cdots \cap A_n) = \PP(A_1)\,\PP(A_2 \mid A_1)\,\PP(A_3 \mid A_1 \cap A_2) \cdots \PP(A_n \mid A_1 \cap \cdots \cap A_{n-1}).

Proof.

By repeated application of the definition of conditional probability. Write $\PP(A_1 \cap A_2) = \PP(A_1)\,\PP(A_2 \mid A_1)$ , then $\PP(A_1 \cap A_2 \cap A_3) = \PP(A_1 \cap A_2)\,\PP(A_3 \mid A_1 \cap A_2)$ , and continue inductively. Each factor is well-defined because $\PP(A_1 \cap \cdots \cap A_{k-1}) > 0$ implies $\PP(A_1 \cap \cdots \cap A_k) \le \PP(A_1 \cap \cdots \cap A_{k-1})$ , and we are given the intersection has positive probability up to $n-1$ terms.

Birthday paradox via the chain rule. The chain rule gives an alternative derivation of the birthday bound. Let $A_i = \{\text{person } i \text{ doesn't share a birthday with persons } 1, \ldots, i-1\}$ . Then

\PP(A_1 \cap \cdots \cap A_n) = 1 \cdot \frac{K-1}{K} \cdot \frac{K-2}{K} \cdots \frac{K-(n-1)}{K} = \prod_{j=0}^{n-1}\Bigl(1 - \frac{j}{K}\Bigr) \approx e^{-n(n-1)/(2K)},

recovering the same bound from Lecture 1.

Law of Total Probability

Theorem 2.2 (Law of total probability)

Let $A_1, A_2, \ldots, A_N$ be a partition of $\Omega$ (i.e. the $A_i$ are pairwise disjoint — mutually exclusive — and their union is $\Omega$ — collectively exhaustive). Then for any event $B$ ,

\PP(B) = \sum_{i=1}^{N} \PP(A_i)\,\PP(B \mid A_i).

Proof.

Since $\{A_i\}$ partitions $\Omega$ , we can decompose $B$ as $B = \bigcup_{i=1}^N (A_i \cap B)$ , and these pieces are disjoint. By countable additivity, $\PP(B) = \sum_{i=1}^N \PP(A_i \cap B) = \sum_{i=1}^N \PP(A_i)\,\PP(B \mid A_i)$ , where the last step uses the definition of conditional probability.

Bayes’ Rule

Theorem 2.3 (Bayes’ rule)

Let $A_1, \ldots, A_N$ partition $\Omega$ . Given an event $B$ with $\PP(B) > 0$ ,

\PP(A_i \mid B) = \frac{\PP(B \mid A_i)\,\PP(A_i)}{\sum_{j=1}^{N} \PP(B \mid A_j)\,\PP(A_j)}.

The numerator uses the definition of conditional probability, and the denominator is $\PP(B)$ expanded by the law of total probability.

Example 2.2 (False positive quiz – testing for a rare disease)

A medical test for a rare disease has the following properties:

If a person has the disease, the test is positive with probability $0.95$ (sensitivity).
If a person does not have the disease, the test is negative with probability $0.95$ (specificity), i.e. false positive rate $= 0.05$ .
The prior prevalence is $\PP(A) = 0.001$ (1 in 1000 people have the disease).

A person tests positive. What is $\PP(\text{disease} \mid \text{positive})$ ?

Define $A = \{\text{has disease}\}$ , $B = \{\text{tests positive}\}$ . By Bayes’ rule:

\PP(A \mid B) = \frac{\PP(B \mid A)\,\PP(A)}{\PP(B \mid A)\,\PP(A) + \PP(B \mid A^c)\,\PP(A^c)} = \frac{(0.95)(0.001)}{(0.95)(0.001) + (0.05)(0.999)} \approx 0.0187.

Despite the test being 95% accurate, a positive result only indicates a $\approx 1.87\%$ chance of actually having the disease. This is because the disease is so rare that the false positives from healthy people vastly outnumber the true positives.

Sensitivity to the prior. If the prior were $\PP(A) = 0.01$ instead ( $10\times$ more common), then $\PP(A \mid B) \approx 16\%$ — still low, but dramatically higher. The base rate (prior) has an enormous effect on the posterior.

Independence

Definition 2.2 (Independence of two events)

Two events $A$ and $B$ are independent if the occurrence of one provides no information about the occurrence of the other. Formally:

\PP(A \mid B) = \PP(A), \qquad \text{or equivalently,} \qquad \PP(A \cap B) = \PP(A)\,\PP(B).

The second form says the joint probability equals the product of the marginals.

Key remarks on independence:

Disjoint $\ne$ independent. $A, B$ disjoint means $\PP(A \cap B) = 0$ . For this to equal $\PP(A)\PP(B)$ , we need $\PP(A) = 0$ or $\PP(B) = 0$ . So disjoint events with positive probability are dependent, not independent.
Base outcomes are always dependent. Individual outcomes $\{\omega\}$ in any finite $\Omega$ are always dependent (knowing one occurred rules out the others).

Example 2.3 (Dice: independence depends on the conditioning event)

Roll two fair dice. Let $A = \{\text{1st die is 6}\}$ , $B = \{\text{sum is 7}\}$ , $C = \{\text{sum is 11}\}$ .

Are $A$ and $B$ independent? $\PP(A \mid B) = 1/6 = \PP(A)$ , so yes, $A \perp B$ .

Are $A$ and $C$ independent? $\PP(A \mid C) = 1/2 \ne 1/6 = \PP(A)$ , so no, $A$ and $C$ are dependent. Intuitively, knowing the sum is 11 makes it more likely that the first die was a 6 (since $(6,5)$ is one of only two outcomes).

Conditional Independence

Definition 2.3 (Conditional independence)

Events $A$ and $B$ are conditionally independent given $C$ if

\PP(A \cap B \mid C) = \PP(A \mid C)\,\PP(B \mid C).

Independence and conditional independence are distinct concepts:

Dependent events can become conditionally independent.
Independent events can become conditionally dependent.

Example 2.4 (Two indistinguishable coins)

You have two coins: one is double-headed, the other is double-tailed. You pick one at random and flip it twice. Let $H_i = \{\text{flip } i \text{ is heads}\}$ , $i=1,2$ .

Q1: Are $H_1, H_2$ (unconditionally) independent? No. $\PP(H_1 \mid H_2) = 1 \ne 1/2 = \PP(H_1)$ . If the second flip is heads, you certainly picked the double-headed coin, so the first flip is heads with certainty.

Q2: Conditioning on the coin, are they independent? Let $A = \{\text{picked the double-headed coin}\}$ . Then $\PP(H_1 \cap H_2 \mid A) = \PP(H_1 \mid A)\,\PP(H_2 \mid H_1 \cap A) = 1 \cdot 1 = 1$ , and $\PP(H_1 \mid A)\,\PP(H_2 \mid A) = 1 \cdot 1 = 1$ . These are equal, so $H_1 \perp H_2 \mid A$ . Once we know which coin was picked, each flip is determined and independent.

Example 2.5 (Fair coin – independent but conditionally dependent)

Toss a fair coin twice. Let $H_1 = \{\text{1st toss is H}\}$ , $H_2 = \{\text{2nd toss is H}\}$ , and $D = \{\text{1st toss} \ne \text{2nd toss}\}$ .

Q1: $H_1 \perp H_2$ ? Yes, by independence of fair coin flips.

Q2: $H_1 \perp H_2 \mid D$ ? No. $\PP(H_1 \mid D)\,\PP(H_2 \mid D) = \tfrac{1}{2} \cdot \tfrac{1}{2} = \tfrac{1}{4}$ . But $\PP(H_1 \cap H_2 \mid D) = 0$ , since both flips being heads contradicts $D$ (the flips must differ). Since $0 \ne 1/4$ , the events are conditionally dependent given $D$ .

Joint Independence vs. Pairwise Independence

Definition 2.4 (Independence of a collection)

Events $\{A_i\}_{i \in S}$ are jointly (mutually) independent if for every subset $T \subseteq S$ ,

\PP\!\Bigl(\bigcap_{i \in T} A_i\Bigr) = \prod_{i \in T} \PP(A_i).

Pairwise independence only requires this for all pairs $|T| = 2$ .

Pairwise independence $\nRightarrow$ joint independence. Consider the fair coin example: $H_1, H_2, D$ are pairwise independent (check all three pairs), but they are not jointly independent because $\PP(H_1 \cap H_2 \cap D) = 0 \ne \PP(H_1)\PP(H_2)\PP(D) = 1/8$ .

Renewal: Rolling a 6 Before a 7

Example 2.6 (Roll a sum of 6 before a sum of 7)

Roll two fair dice independently and repeatedly. Let $E = \{\text{a sum of 6 appears before a sum of 7}\}$ .

The outcomes that sum to 6 are $\{(1,5),(2,4),(3,3),(4,2),(5,1)\}$ , so $\PP(S) = 5/36$ where $S = \{\text{1st roll sums to 6}\}$ . Similarly, $\PP(T) = 6/36$ where $T = \{\text{1st roll sums to 7}\}$ .

By the law of total probability, conditioning on the first roll:

\PP(E) = \PP(E \mid S)\,\PP(S) + \PP(E \mid T)\,\PP(T) + \PP(E \mid (S \cup T)^c)\,\PP((S \cup T)^c).

We have $\PP(E \mid S) = 1$ , $\PP(E \mid T) = 0$ , and $\PP(E \mid (S \cup T)^c) = \PP(E)$ by the renewal/memoryless property (each roll is independent, so if neither 6 nor 7 appears, we restart). Thus

\PP(E) = 1 \cdot \frac{5}{36} + 0 \cdot \frac{6}{36} + \PP(E) \cdot \frac{25}{36}.

Solving: $\PP(E)(1 - 25/36) = 5/36$ , so $\PP(E)(11/36) = 5/36$ , giving

\PP(E) = \frac{5}{11}.

conditional-probability-and-independence.html