The Leibniz rule (differentiation under the integral sign) justifies moving dαd outside the integral. Finally,
Var(Z)=E[Z2]−(E[Z])2=1−02=1.
Why only α=1? I was confused why we set α=1, and whether it could be something else. The identity I(α)=α−1/2 holds for everyα>0, so I is a function of α that we can differentiate, and differentiating in α is what produces the factor x2. We set α=1 only after differentiating, because the standard normal has e−x2/2, i.e. α=1. Setting α=1 first would leave the constant I(1)=1, whose derivative is 0.
Cross-check by integration by parts. With u=x and dv=2π1xe−x2/2dx, so v=−2π1e−x2/2:
Is fZ(z)⋅σ1 already the answer? I was unsure whether the line fX(x)=2π1e−z2/2⋅σ1 was correct. It is: substituting z=σx−μ gives exactly the N(μ,σ2) density. The factor σ1 in front comes from dxdz.
Should μ and σ have a subscript Z? I was also confused about this. No: Z always has mean 0 and variance 1. Here μ and σ are the shift and scale applied to Z, and they become the mean and standard deviation of X: E[X]=σE[Z]+μ=μ and Var(X)=σ2Var(Z)=σ2. If anything, they are μX and σX.
Corollary 7.1 (Standardization)
If X∼N(μ,σ2), then Z=σX−μ∼N(0,1), and
FX(x)=Φ(σx−μ).
Symmetry of Φ and Standardization
Theorem 7.4 (Symmetry of the standard normal CDF)
For every y∈R,
Φ(−y)=1−Φ(y).
So a table only needs Φ(y) for y≥0.
Proof.
fZ is even: fZ(−v)=fZ(v). Substitute u=−v, du=−dv:
Why 1−Φ(1)? I was confused why the answer is 1−Φ(1) rather than Φ(1). Φ(1) is the area to the left of 1; P(Z>1) is the area to the right, which is its complement.
Sums of Independent Gaussians
Theorem 7.5 (Sum of independent Gaussians)
If X∼N(μX,σX2) and Y∼N(μY,σY2) are independent, then
X+Y∼N(μX+μY,σX2+σY2).
The parameters follow from Lecture 4: E[X+Y]=E[X]+E[Y]=μX+μY, and by independence Var(X+Y)=Var(X)+Var(Y)=σX2+σY2. The content of the theorem is that X+Y is again Gaussian. This can be shown with a convolution, or in one line with moment generating functions.
Proof: MGF proof TBD in a later lecture.
Theorem 7.6 (Converse)
If X and Y are independent and X+Y is Gaussian, then X and Y are each Gaussian (allowing a constant, i.e. variance 0).
This direction is much harder; its proof is beyond the scope of this course.
Derived Distributions
Given X with known density fX and Y=g(X), find fY.
If g is strictly increasing, {x:g(x)≤y}={x≤g−1(y)}, so FY(y)=FX(g−1(y)). If g is strictly decreasing, the inequality flips: {x:g(x)≤y}={x≥g−1(y)}, so FY(y)=1−FX(g−1(y)).
Corollary 7.2 (Monotone change of variables)
If g is strictly monotone and differentiable, then
fY(y)=fX(g−1(y))dydg−1(y).(7.21)
Proof.
Increasing g: dydFX(g−1(y))=fX(g−1(y))dydg−1(y), with dydg−1(y)>0. Decreasing g: dyd[1−FX(g−1(y))]=−fX(g−1(y))dydg−1(y), with dydg−1(y)<0. In both cases the result is (7.21).
For X=σZ+μ: g−1(x)=σx−μ and dxdg−1(x)=σ1, which is the factor σ1 found in Section 7.3.
Example 7.2 (Square root of a uniform)
Let X∼Uniform[0,1] and Y=X. For 0≤y≤1, since x is increasing on [0,1] and both sides are nonnegative,
Why only 1 way for the third group? I was confused whether the right-hand group contributes (n−k⋅) more choices. It does not: once the middle sample and the k−1 left samples are chosen, the other n−k samples are forced to the right. (Two or more samples landing in (x,x+dx) has probability of order (dx)2, which is negligible.)
Why y? I was confused why the formula was written in y rather than x. The argument is a dummy variable: fX(k)(y) and fX(k)(x) are the same function. What matters is that one variable is used throughout, and that FX and fX are the CDF and PDF of a single sample, not of X(k).
Check, k=n (the maximum).P(X(n)≤x)=P(all Xi≤x)=[FX(x)]n, and dxd[FX(x)]n=n[FX(x)]n−1fX(x), matching (7.28) with k=n.
Corollary 7.3 (Uniform samples)
If Xi∼Uniform[0,1], then fX(x)=1 and FX(x)=x on [0,1], so
Do the gaps really sum to 1? I was unsure about this. They do: the gaps tile [0,1] exactly, which is the telescoping sum above.
Why are the gaps symmetric? Place 11 independent uniform points on a circle of circumference 1 and cut the circle at one of them. The other 10 points are i.i.d. uniform on the resulting interval [0,1], and the 11 arcs between consecutive points are exactly the gaps Δ1,…,Δ11. On the circle no arc is special, so all gaps have the same distribution.
Cross-check with (7.30). With k=1: fX(1)(x)=0!9!10!(1−x)9=10(1−x)9. Substituting u=1−x,