discrete-random-variables-and-expectation.html

Lecture 3: Discrete Random Variables and Expectation

EECS 126 (UC Berkeley) · Spring 2026 · January 27, 2026

We transition from events to random variables — functions that assign numerical values to outcomes — and develop the key tools of expectation, linearity, LOTUS, and variance.

Random Variables and PMFs

Definition 3.1 (Random variable)

A (discrete) random variable is a function X:Ω→RX : \Omega \to \RR that assigns a real number to each outcome ω∈Ω\omega \in \Omega. We write P(X=x)\PP(X = x) for the probability of the event {ω∈Ω:X(ω)=x}\{\omega \in \Omega : X(\omega) = x\}.

Definition 3.2 (Probability mass function)

The probability mass function (PMF) of a discrete random variable XX is the function pX:R→[0,1]p_X : \RR \to [0,1] defined by

pX(x)=P(X=x).p_X(x) = \PP(X = x).

The PMF satisfies ∑x∈XpX(x)=1\sum_{x \in \mathcal{X}} p_X(x) = 1, where X\mathcal{X} denotes the set of all values taken by XX (the support).

Example 3.1 (Functions of random variables)

Let XX be a random variable with pX(x)=1/5p_X(x) = 1/5 for x∈{−2,−1,0,1,2}x \in \{-2, -1, 0, 1, 2\}. Define Y=∣X∣Y = |X|. Then YY takes values in {0,1,2}\{0, 1, 2\} and

pY(y)=∑{x : ∣x∣=y}pX(x)={1/5if y=0,2/5if y=1,2/5if y=2.p_Y(y) = \sum_{\{x\,:\, |x| = y\}} p_X(x) = \begin{cases} 1/5 & \text{if } y = 0, \\ 2/5 & \text{if } y = 1, \\ 2/5 & \text{if } y = 2. \end{cases}

In general, if Y=g(X)Y = g(X), then pY(y)=∑{x:g(x)=y}pX(x)p_Y(y) = \sum_{\{x : g(x) = y\}} p_X(x).

Definition 3.3 (Bernoulli distribution)

X∼Bernoulli⁡(p)X \sim \Bern(p): a single coin toss.

P(X=k)={1−pk=0,pk=1.\PP(X = k) = \begin{cases} 1-p & k = 0, \\ p & k = 1. \end{cases}
Definition 3.4 (Binomial distribution)

X∼Binomial⁡(n,p)X \sim \Bin(n, p): number of successes in nn independent Bernoulli(p)(p) trials.

P(X=k)=(nk)pk(1−p)n−k,k=0,1,…,n.\PP(X = k) = \binom{n}{k} p^k (1-p)^{n-k}, \qquad k = 0, 1, \ldots, n.

Note: ∑k=0n(nk)pk(1−p)n−k=(p+(1−p))n=1\sum_{k=0}^n \binom{n}{k} p^k(1-p)^{n-k} = (p + (1-p))^n = 1 by the binomial theorem, confirming this is a valid PMF.

Definition 3.5 (Geometric distribution)

X∼Geometric⁡(p)X \sim \Geom(p): number of trials until the first success.

P(X=k)=(1−p)k−1p,k=1,2,3,…\PP(X = k) = (1-p)^{k-1} p, \qquad k = 1, 2, 3, \ldots

The geometric distribution is the discrete analog of “waiting time.”

Definition 3.6 (Poisson distribution)

X∼Poisson⁡(λ)X \sim \Pois(\lambda): models the number of arrivals in a fixed interval.

P(X=k)=λke−λk!,k=0,1,2,…\PP(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}, \qquad k = 0, 1, 2, \ldots

The Poisson is the limiting distribution of Binomial⁡(n,λ/n)\Bin(n, \lambda/n) as n→∞n \to \infty, and arises naturally in modeling rare events and arrival processes.

Example 3.2 (Chess match – a PMF computation)*

Alice and Bob play a match: the first to win a game wins the match. If there are 10 consecutive draws, the match is declared drawn. Each game independently: P(A wins)=0.3\PP(\text{A wins}) = 0.3, P(B wins)=0.4\PP(\text{B wins}) = 0.4, P(draw)=0.3\PP(\text{draw}) = 0.3.

Let LL be the duration (number of games). Then for ℓ=1,2,…,9\ell = 1, 2, \ldots, 9:

pL(ℓ)=(0.3)ℓ−1(1−0.3)=(0.3)ℓ−1(0.7),p_L(\ell) = (0.3)^{\ell - 1}(1 - 0.3) = (0.3)^{\ell-1}(0.7),

since the first ℓ−1\ell - 1 games must be draws and the ℓ\ell-th decisive. For ℓ=10\ell = 10: pL(10)=(0.3)9p_L(10) = (0.3)^9 (all 9 games drawn, match ends).

P(Alice wins match)=∑ℓ=19(0.3)ℓ−1(0.3)=0.3∑k=08(0.3)k\PP(\text{Alice wins match}) = \sum_{\ell=1}^{9}(0.3)^{\ell-1}(0.3) = 0.3 \sum_{k=0}^{8}(0.3)^k. Using ∑k=0∞rk=1/(1−r)\sum_{k=0}^{\infty} r^k = 1/(1-r) for ∣r∣<1|r|<1, in the limit of no cap on draws: P(Alice wins)=0.3/(1−0.3)=3/7\PP(\text{Alice wins}) = 0.3/(1-0.3) = 3/7.

Expectation

Definition 3.7 (Expected value)

The expected value (or mean) of a discrete random variable XX is

E[X]=∑ω∈ΩX(ω) P({ω})=∑x∈Xx⋅P(X=x),\EE[X] = \sum_{\omega \in \Omega} X(\omega)\,\PP(\{\omega\}) = \sum_{x \in \mathcal{X}} x \cdot \PP(X = x),

provided the sum converges absolutely (i.e. ∑∣x∣ P(X=x)<∞\sum |x|\,\PP(X=x) < \infty).

Theorem 3.1 (Linearity of expectation)

For any random variables X,YX, Y (not necessarily independent) and constant cc:

E[X+Y]=E[X]+E[Y],E[cX]=c E[X].\EE[X + Y] = \EE[X] + \EE[Y], \qquad \EE[cX] = c\,\EE[X].

No independence assumption is needed. This is the single most useful property of expectation.

Definition 3.8 (Indicator random variable)

For an event AA, the indicator 1A\mathbf{1}_A is

1A(ω)={1ω∈A,0ω∉A.\mathbf{1}_A(\omega) = \begin{cases} 1 & \omega \in A, \\ 0 & \omega \notin A. \end{cases}

Observe that 1A∼Bernoulli⁡(P(A))\mathbf{1}_A \sim \Bern(\PP(A)), so E[1A]=P(A)\EE[\mathbf{1}_A] = \PP(A).

Example 3.3 (Seats on a plane)

A full flight has nn passengers with assigned seats, but every passenger ignores their assignment and sits in a uniformly random seat. What is E[number who sit in their own seat]\EE[\text{number who sit in their own seat}]?

Let Xi=1{passenger i sits in their own seat}X_i = \mathbf{1}\{\text{passenger } i \text{ sits in their own seat}\}. Then X=X1+X2+⋯+XnX = X_1 + X_2 + \cdots + X_n. The XiX_i are not independent (if n=2n = 2 and passenger 1 is in their seat, passenger 2 must be too). But linearity of expectation does not require independence:

E[X]=∑i=1nE[Xi]=∑i=1nP(passenger i is in their seat)=n⋅1n=1.\EE[X] = \sum_{i=1}^n \EE[X_i] = \sum_{i=1}^n \PP(\text{passenger } i \text{ is in their seat}) = n \cdot \frac{1}{n} = 1.

Regardless of the number of passengers, on average exactly 1 person sits in their own seat.

LOTUS (Law of the Unconscious Statistician)

Theorem 3.2 (LOTUS)

If XX is a discrete random variable and g:R→Rg : \RR \to \RR is any function, then

E[g(X)]=∑x∈Xg(x) P(X=x).\EE[g(X)] = \sum_{x \in \mathcal{X}} g(x)\,\PP(X = x).

More generally, for a function of several random variables:

E[f(X1,…,Xn)]=∑x1,…,xnf(x1,…,xn) pX1,…,Xn(x1,…,xn).\EE[f(X_1, \ldots, X_n)] = \sum_{x_1, \ldots, x_n} f(x_1, \ldots, x_n)\, p_{X_1, \ldots, X_n}(x_1, \ldots, x_n).

The name reflects that one can compute E[g(X)]\EE[g(X)] directly from the distribution of XX, without first finding the distribution of Y=g(X)Y = g(X).

Independence of Random Variables

Definition 3.9 (Independent random variables)

Random variables XX and YY are independent (written X⊥YX \perp Y) if their joint distribution factors into the product of the marginals:

P(X=x, Y=y)=P(X=x) P(Y=y)for all x,y.\PP(X = x,\, Y = y) = \PP(X = x)\,\PP(Y = y) \quad \text{for all } x, y.
Theorem 3.3 (Expectation of a product of independent RVs)

If X⊥YX \perp Y, then

E[XY]=E[X] E[Y].\EE[XY] = \EE[X]\,\EE[Y].
Proof.

By LOTUS and independence:

E[XY]=∑x,yxy pX,Y(x,y)=∑x,yxy pX(x) pY(y)=(∑xx pX(x))(∑yy pY(y))=E[X] E[Y].\begin{aligned} \EE[XY] &= \sum_{x,y} xy\, p_{X,Y}(x,y) = \sum_{x,y} xy\, p_X(x)\, p_Y(y) \\ &= \Bigl(\sum_x x\, p_X(x)\Bigr)\Bigl(\sum_y y\, p_Y(y)\Bigr) = \EE[X]\,\EE[Y]. \end{aligned}

The key step is factoring the joint PMF into marginals (by independence), which allows separating the double sum into a product of single sums.

The converse is false. E[XY]=E[X]E[Y]\EE[XY] = \EE[X]\EE[Y] does not imply X⊥YX \perp Y. The condition E[XY]=E[X]E[Y]\EE[XY] = \EE[X]\EE[Y] is called uncorrelation, which is strictly weaker than independence. As an exercise, construct dependent random variables that are uncorrelated.

Tail Sum Formula

Theorem 3.4 (Tail sum formula)

Let XX be a random variable taking values in N={0,1,2,…}\NN = \{0, 1, 2, \ldots\}. Then

E[X]=∑k=1∞P(X≥k).\EE[X] = \sum_{k=1}^{\infty} \PP(X \ge k).
Proof.

Starting from the definition of expectation:

E[X]=∑k=1∞k P(X=k)=∑k=1∞∑j=1k1⋅P(X=k)=∑j=1∞∑k=j∞P(X=k)=∑j=1∞P(X≥j).\begin{aligned} \EE[X] &= \sum_{k=1}^{\infty} k\,\PP(X = k) = \sum_{k=1}^{\infty} \sum_{j=1}^{k} 1 \cdot \PP(X = k) \\ &= \sum_{j=1}^{\infty} \sum_{k=j}^{\infty} \PP(X = k) = \sum_{j=1}^{\infty} \PP(X \ge j). \end{aligned}

The key step is writing k=∑j=1k1k = \sum_{j=1}^k 1 to introduce a double sum, then swapping the order of summation (Tonelli’s theorem ensures this is valid for nonnegative terms). Geometrically, this corresponds to “counting vertically then horizontally”: instead of weighting each column kk by its height kk, we sum across rows j=1,2,…j = 1, 2, \ldots, each contributing P(X≥j)\PP(X \ge j).

Intuition for the tail sum. Think of E[X]\EE[X] as the total area under the “staircase” of the survival function P(X≥k)\PP(X \ge k). Each unit step at height kk contributes P(X≥k)\PP(X \ge k) to the total. This is the same idea as computing the area under a curve by integrating horizontally rather than vertically.

Variance

Definition 3.10 (Variance and standard deviation)

The variance of XX measures the spread of XX around its mean:

Var⁡(X)≜σX2≜E[(X−E[X])2].\Var(X) \triangleq \sigma_X^2 \triangleq \EE\bigl[(X - \EE[X])^2\bigr].

The standard deviation is σX=Var⁡(X)\sigma_X = \sqrt{\Var(X)}.

An equivalent computational formula (often easier to use):

Var⁡(X)=E[X2]−(E[X])2.\Var(X) = \EE[X^2] - (\EE[X])^2.
Derivation of the computational formula.

Expand the square using linearity:

Var⁡(X)=E[(X−E[X])2]=E[X2−2X E[X]+(E[X])2]=E[X2]−2 E[X] E[X]+(E[X])2=E[X2]−(E[X])2.\begin{aligned} \Var(X) &= \EE[(X - \EE[X])^2] = \EE[X^2 - 2X\,\EE[X] + (\EE[X])^2] \\ &= \EE[X^2] - 2\,\EE[X]\,\EE[X] + (\EE[X])^2 = \EE[X^2] - (\EE[X])^2. \end{aligned}

Note that E[X]\EE[X] is a constant, so E[2X⋅E[X]]=2 E[X]⋅E[X]\EE[2X \cdot \EE[X]] = 2\,\EE[X] \cdot \EE[X] and E[(E[X])2]=(E[X])2\EE[(\EE[X])^2] = (\EE[X])^2.

Theorem 3.5 (Scaling property of variance)

For constants a,c∈Ra, c \in \RR:

Var⁡(aX+c)=a2 Var⁡(X).\Var(aX + c) = a^2\,\Var(X).

Adding a constant shifts the mean but does not change the spread. Scaling by aa scales the variance by a2a^2 (and the standard deviation by ∣a∣|a|).

Proof.

Let μ=E[X]\mu = \EE[X]. Then E[aX+c]=aμ+c\EE[aX + c] = a\mu + c, so

Var⁡(aX+c)=E[(aX+c−aμ−c)2]=E[(a(X−μ))2]=a2 E[(X−μ)2]=a2 Var⁡(X).\Var(aX + c) = \EE[(aX + c - a\mu - c)^2] = \EE[(a(X - \mu))^2] = a^2\,\EE[(X - \mu)^2] = a^2\,\Var(X).

Coming up next lecture: covariance Cov⁡(X,Y)\Cov(X, Y), variance of sums, and the beginning of continuous random variables.

1436 words