Chapter 7 · Subchapter 2

Laws of Large Numbers: Averages Converge, When They May

The theorem practitioners quote most and prove least. This subchapter proves it twice — weak, then strong — with every hypothesis bought and paid for, and then shows what happens to the most-used theorem in finance when its hypotheses fail.

In this subchapter

The statement, and the two proof strategies
Chebyshev's weak law: three lines
The strong law: subsequences and Borel–Cantelli
Kolmogorov's 0–1 law
The Cauchy failure
Reading the fine print of practice

7.2.1The statement, and the two proof strategies

Setup — iid averages

Throughout: X₁, X₂, … independent and identically distributed (iid) on some probability space — the countable product construction of 6.1 Example 3 guarantees such spaces exist with any desired law μ — with mean m = E[X_i], when it exists. The sample average is X̄_n = (X₁ + ⋯ + X_n)/n. The weak law claims X̄_n → m in probability; the strong law claims X̄_n → m almost surely. Both are convergence statements in the modes of 7.1, and the strong law implies the weak.

Two strategies, in increasing depth. Chebyshev: control the variance of the average and invoke Markov — weak law, minimal machinery, hypothesis: finite variance. Borel–Cantelli on subsequences: prove a.s. convergence along a sparse subsequence, then fill in the gaps with a maximal inequality — strong law, and the honest version here assumes a finite fourth moment to make every step executable with the course's tools. (Kolmogorov's 1933 proof needs only E|X| < ∞ but uses martingale/truncation technology beyond our current toolkit; we flag the gap rather than fake it.)

7.2.2Chebyshev's weak law: three lines

Theorem 1 (weak law, Chebyshev version)

If the X_i are iid (in fact pairwise-uncorrelated suffices) with common mean m and finite variance σ², then X̄_n → m in probability, at the quantified rate P(|X̄_n − m| > ε) ≤ σ²/(nε²).

Proof — the three-line budget, fully spent

Line 1: E[X̄_n] = m (linearity, 5.1 Theorem 2). Line 2: Var(X̄_n) = E[(X̄_n−m)²] = (1/n²)ΣΣ E[(X_i−m)(X_j−m)] = σ²/n — the cross terms vanish by independence (expectations factor, 6.1 Theorem 2; uncorrelated suffices), and the n diagonal terms each contribute σ²/n². Line 3: Markov's inequality (7.1 Theorem 1's proof) on (X̄_n − m)²: P(|X̄_n−m| > ε) ≤ E[(X̄_n−m)²]/ε² = σ²/(nε²) → 0. ∎

Example 1 · What the weak law does not say

The weak law permits the deviant set to wander (the typewriter of 7.1 Example 2 converges in probability to 0 while every ω oscillates forever). For averages the phenomenon is milder but real: the weak law gives no statement about when — or whether — a given realization settles. A backtest whose running Sharpe ratio "is consistent with convergence in probability" may still, in principle, never settle on your data's path. The strong law exists because practitioners need the pathwise guarantee, and the next section pays for it with real machinery. Also note the hypothesis ledger: variance. Every Cauchy-adjacent law (5.5) exits here — the fine print that §5 weaponizes.

7.2.3The strong law: subsequences and Borel–Cantelli

Theorem 2 (strong law, fourth-moment version)

If the X_i are iid with mean m and E[X_i⁴] < ∞, then X̄_n → m almost surely: with probability 1, the realized average converges to the theoretical mean.

Proof — the full argument, in four moves

Center: Y_i = X_i − m, so E[Y_i] = 0 and we show Σ_i≤nY_i/n → 0 a.s.

Move 1 — fourth moments of sums. Expand E[(Σ_i≤nY_i)⁴]: the multinomial expansion has terms E[Y_i⁴] (n of them), E[Y_i²Y_j²] (3n(n−1) of them — the pair-choosing cross terms), and terms containing a variable to the first or third power alone, which vanish by independence (factoring, 6.1 Theorem 2) since E[Y_i] = 0. So E[(ΣY_i)⁴] = n·E[Y⁴] + 3n(n−1)·σ⁴ ≤ C n² for a constant C — the fourth-moment hypothesis is exactly what makes the count work.

Move 2 — a.s. along powers of 2. At n = 2^k: by Markov on the fourth power, P( |Σ_{i≤2^kY_i| > ε·2^k ) ≤ C·2^2k/(ε⁴·2^4k) = C/(ε⁴4^k)} — summable in k. Borel–Cantelli I (6.3 Theorem 1): a.s., for all large k, |Σ_i≤2^kY_i| ≤ ε·2^k — i.e. the dyadic-subsequence averages converge to 0 a.s. Let ε run over 1/m: intersection of countably many a.s. events is a.s. (2.1 continuity), so the dyadic averages converge to 0 a.s.

Move 3 — filling the gaps between dyadic points. For 2^k ≤ n < 2^k+1, the sum S_n is sandwiched between dyadic sums only at endpoints — the real argument bounds the maximum fluctuation within the block: max_{2^k≤n<2^k+1}|S_n − S_2^k|, and shows via a fourth-moment bound (Move 1 applied to block increments, plus Markov) that the probability of a fluctuation exceeding ε·2^k is ≤ C'/4^k — summable, Borel–Cantelli again. So a.s. the within-block fluctuation is eventually small relative to 2^k, hence relative to n ≥ 2^k.

Move 4 — assemble. On the a.s. event where both dyadic averages converge and block fluctuations are negligible: |S_n|/n ≤ (|S_2^k| + fluctuation)/n → 0. ∎

The two reusable patterns inside the proof

Subsequence-plus-fluctuation: prove convergence where sums are summable (here: geometrically spaced indices make the tail probabilities summable), then control the maximum deviation in between. This is the standard route to a.s. results and it reappears in 8.3 for martingales. Moment escalation: the fourth moment was not aesthetic — the count in Move 1 needs a power above 2 to beat the n² normalization (with only second moments, the dyadic bound is C 2^2k/2^2k = constant — not summable; the ε-room vanishes). Kolmogorov's E|X| < ∞ version replaces brute fourth moments with the Doob maximal inequality for martingales — the honest statement of what we borrowed.

7.2.4Kolmogorov's 0–1 law

Theorem 3 (Kolmogorov's 0–1 law)

For independent X₁, X₂, …, every tail event (6.3 §6: an event in 𝒯 = ⋂_nσ(X_n+1, …)) has probability 0 or 1.

Proof — the tail σ-algebra is independent of itself

Step 1: for each fixed N, 𝒯 ⊆ σ(X_N+1, X_N+2, …), and σ(X₁, …, X_N) is independent of that future σ-algebra — independence of the sequence extends to the two σ-algebras generated by disjoint index blocks (a good-sets argument on rectangles, each of which multiplies by 6.1; the pattern is by now an old friend). So: 𝒯 is independent of σ(X₁,…,X_N), for every N. Step 2: let F₀ = σ(X₁, X₂, …) = ⋃_Nσ(X₁,…,X_N)-generated σ-algebra; a σ-algebra generated by a union of σ-algebras each independent of 𝒯 is itself independent of 𝒯 (good-sets once more, on the π-system of finite intersections from the union). But 𝒯 ⊆ F₀ — the tail is part of the whole. A σ-algebra independent of itself: P(A) = P(A ∩ A) = P(A)² for A ∈ 𝒯, so P(A) ∈ {0, 1}. ∎

Example 2 · Tail questions, answered before they are asked

"Does the average converge" is a tail question (finitely many X_i cannot decide it) — so under independence its probability is 0 or 1 before any analysis. The strong law then identifies which. The same holds for "does Σ 1/n·X_n converge", "is the sequence eventually monotone" (not tail — flag the exercise of checking which questions are tail questions; the boundary is instructive). The law's moral: for independent processes, asymptotic events carry no genuine randomness — the randomness is all in the finite prefixes; the infinite run is decided. This is why a.s. results feel "deterministic": they are.

7.2.5The Cauchy failure

Example 3 · iid Cauchy: no law of large numbers, provably

By 5.5 Theorem 2, averages of iid standard Cauchy variables are exactly standard Cauchy for every n — X̄_n has the same law as X₁. So X̄_n converges in distribution (trivially, to Cauchy) but not to a constant; it does not converge in probability (Cauchy mass does not concentrate); a fortiori not a.s. Every hypothesis of the strong law fails at once — no mean — and the conclusion fails exactly as the theorem predicts. The practitioner's version, sharpened by proof: averaging does not rescue you from heavy tails. Diversification across iid heavy-tailed bets at α ≤ 1 has no asymptotic effect on the average's spread; for 1 < α < 2 (mean exists, variance does not) the strong law holds but the CLT does not (7.3), and the rate of convergence in the LLN is governed by stable laws — the boundary regime where "more data" helps ever more slowly.

7.2.6Reading the fine print of practice

The ledger for the most-quoted theorem in finance

"The average converges" — to what? The mean exists only under 5.5's tail conditions; checking is a tail-index exercise, not an option.
"Consistency of the sample mean" (the estimator's justification): a strong-law statement — pathwise, with hypotheses.
Dependence: the strong law survives some dependence (ergodic/mixing conditions, beyond the course) — but the honest default is that correlated data effectively shrinks n, and naive standard errors are optimistic by a factor computable only within a dependence model. The iid fine print is the single most-exploited gap in applied statistics.
Rate: the LLN says nothing about how fast; the quantified Chebyshev bound σ²/(nε²) is crude, and the true rate is the CLT's √n (7.3) under second moments.

Averages converge almost surely — when the tails permit, the draws are honest, and the mean exists. All three conditions are theorems' fine print, not nature's guarantees.

Where we are, and what is next

The averages converge; the next question is at what rate and to what shape. The central limit theorem answers: √n scaling, Gaussian limit — under second moments (and fails for heavy tails, with the stable-law regime taking over). We state the CLT, prove the binomial-to-normal case honestly (the method of moments, one computation at a time), and quantify the gap with Berry–Esseen.