Chapter 5 · Subchapter 4

Change of Variables and LOTUS

Expectation was defined on the outcome space, but computed quantities should depend only on the law. This subchapter proves that independence — and builds the computational tools: pushforward measures, LOTUS, the simulation inverse, and the Jacobian rule.

In this subchapter

The pushforward: a law is a measure
Change of variables: expectation via the law
LOTUS: the density formula, earned
Simulating any law: the generalized inverse
The Jacobian rule, and its trap
The change-of-variables map of the course

5.4.1The pushforward: a law is a measure

Definition — law (distribution) of X, pushforward

For a random variable X: Ω → ℝ, its law is the probability measure on ℬ(ℝ) defined by

μ_X(B) = P(X ∈ B) = P(X⁻¹(B))

Called the pushforward (or image measure): X transports probability from Ω onto the real line. It is a genuine probability measure: μ(∅) = 0, μ(ℝ) = 1, and countable additivity holds because inverse images preserve disjoint unions — the same commutation facts as 4.1 Theorem 1. Its CDF (4.2) is F(t) = μ_X((−∞, t]), and Theorem 3 of 4.2 said exactly this: the law determines the law — a tautology that becomes content now that laws are measures.

Example 1 · Pushforwards you already own

Uniform on [0,1]: μ(B) = λ(B ∩ [0,1]). A fair die: μ = (1/6)Σ δ_k — a sum of Dirac measures, where δ_a(B) = 1 if a ∈ B, else 0: the law concentrated at a single point. The Cantor law (4.2 §5): the measure with CDF the devil's staircase — neither a density nor a sum of Diracs, its own object. These are the three species: absolutely continuous, discrete, singular; mixed laws are convex combinations, and the classification theorem (Lebesgue decomposition, previewed in 8.2) says this is essentially the whole zoo.

5.4.2Change of variables: expectation via the law

Theorem 1 — change of variables

For any random variable X and any measurable g: ℝ → ℝ such that either side is defined,

E[g(X)] = ∫_ℝ g dμ_X — g integrated against the pushforward measure

the right side being the Lebesgue integral of g on the measure space (ℝ, ℬ, μ_X), defined exactly as in 5.1 (the construction needs only a measure, not a probability). The left side is an integral over Ω; the right, over ℝ. The theorem says they are the same number — expectation depends only on the law.

Proof — climb the ladder, once more, once for all

Both sides are linear in g, so it suffices to handle g = 1_B and extend by the standard machine: sums (linearity), nonnegative limits (MCT, 5.2), differences (splitting). For indicators: E[1_B(X)] = P(X ∈ B) = μ_X(B) = ∫ 1_B dμ_X — true by definition of the pushforward. For simple g: linearity on both sides, matching term by term. For g ≥ 0: band g into simple g_n ↑ g; then g_n(X) ↑ g(X) (composition preserves monotone pointwise limits), MCT applies on both sides, and the limits agree because each rung agreed. General g: split ± parts. The "standard machine" — indicators → simple → nonnegative → general — is the last full deployment of the good-sets pattern in the course; from here on it is cited by name. ∎

What the theorem licenses

Two desks, two probability spaces, one fitted law: all expectations of all payoffs agree — now a theorem, not an assumption (this is the formal content behind the "same CDF, same pricing" intuition of 4.2 Example 7). It also relocates computation: to price E[g(X)] you never need the outcome space — you need μ_X, i.e. the law, i.e. (by 4.2's Theorem 3) the CDF. Everything in applied probability is computed on the real line; the outcome space retires from active duty.

5.4.3LOTUS: the density formula, earned

Definition — density (absolutely continuous law)

A law μ has a density f (with respect to Lebesgue measure λ) if μ(B) = ∫_B f dλ for all Borel B, with f ≥ 0, ∫ f dλ = 1. The CDF relation: F(t) = ∫_(−∞,t] f dλ, so f is (up to null-set changes) the derivative of F where that makes sense.

Theorem 2 — LOTUS (law of the unconscious statistician)

If X has density f, then for measurable g:

E[g(X)] = ∫_ℝ g(x) f(x) dx

an ordinary (Lebesgue, and by 5.3 often Riemann) integral over the real line. The name jokes that practitioners apply this formula "unconsciously" — it is in fact a theorem: density plus change of variables.

Proof — one line, given the machinery

Theorem 1 gives E[g(X)] = ∫ g dμ_X; the density hypothesis says μ_X(B) = ∫_B f dλ; and the definition of integration against a measure built from a density gives ∫ g dμ_X = ∫ g f dλ — itself another standard-machine run (check indicators: μ_X(B) = ∫ 1_B f dλ, which is the hypothesis). Chain the two. ∎

Example 2 · LOTUS misapplied: a law with no density

Let X have the Cantor law (4.2 §5) and let a script compute E[X] = ∫ x f(x) dx with "the density" f read off as the derivative of the Cantor staircase. The derivative exists and equals 0 almost everywhere, so the script returns 0 — but E[X] = 1/2, by the symmetry of the Cantor construction (replacing every ternary digit 2 by 0 maps X to 1 − X and preserves the law, so the mean, if it exists, is pinned at 1/2; existence follows by boundedness — X lives in [0,1]). The error is not arithmetic; it is applying a theorem whose hypothesis (existence of a density) silently fails. The correct computation runs on the law itself: E[g(X)] = ∫ g dμ_X, and for the Cantor law that integral is computable by the self-similar construction (each ternary digit contributes independently — a preview of 6.1).

5.4.4Simulating any law: the generalized inverse

Theorem 3 — inverse transform sampling

Let F be any CDF (the three properties of 4.2 Theorem 1), and define the generalized inverse F^←(u) = inf{ t : F(t) ≥ u } for u ∈ (0,1). If U is uniform on (0,1), then X = F^←(U) has CDF F. So every probability law on ℝ is the pushforward of the uniform — one uniform variable generates the entire zoo.

Proof — unwinding the definitions

Key equivalence, from monotonicity of F: F^←(u) ≤ t ⟺ u ≤ F(t). (⇐: if u ≤ F(t) then t belongs to the set whose inf is F^←(u); ⇒: if the inf is ≤ t then F(t) ≥ F(inf) ≥ u by right-continuity and monotonicity — the ε-fine point is that F(inf set) can jump past u, which only helps the inequality.) Then P(X ≤ t) = P(F^←(U) ≤ t) = P(U ≤ F(t)) = F(t), the last step because U is uniform: P(U ≤ s) = s on (0,1). That "P(U ≤ s) = s" is the entire content of uniformity — everything else in the proof is bookkeeping of monotone functions. ∎

Example 3 · The quant's daily inversion, certified

Exponential law, F(t) = 1 − e^−t (t ≥ 0): F^←(u) = −ln(1−u) — the formula every simulation library uses, now with a proof and, importantly, with the jump case certified: for a discrete law (e.g. die), F^← is a staircase-valued function and F^←(U) lands on atoms with exactly the right probabilities — check: P(F^←(U) = 3) = P(F(2.5) < U ≤ F(3)) = F(3) − F(2.5) = the atom's mass. Mixed laws — jumps plus continuous stretches, e.g. a loss distribution with an atom at zero plus a continuous spread of losses — are handled by the same proof without modification; that robustness is why the generalized inverse, not the naive inverse, is the industry tool.

5.4.5The Jacobian rule, and its trap

Theorem 4 — multivariate change of variables (statement)

If X = (X₁,…,X_n) has density f on ℝⁿ and T: ℝⁿ → ℝⁿ is a continuously differentiable bijection (on the support) with det DT ≠ 0, then Y = T(X) has density

f_Y(y) = f(T⁻¹(y)) · |det DT(T⁻¹(y))|⁻¹

The Jacobian determinant |det DT| is the local volume stretch of T; the proof is the n-dimensional analog of the 1-D chain f_Y(y) = f_X(T⁻¹(y))·|(T⁻¹)′(y)|, itself provable from Theorem 1 by running the standard machine on sets (we spare the ε-management; the 1-D case for monotone T is worth doing by hand once — integrate the CDF of Y, differentiate back).

Example 4 · Log-returns from prices

Price S > 0 with density; log-return R = ln S. T(x) = ln x, T⁻¹(r) = e^r, derivative e^r: f_R(r) = f_S(e^r) · e^r. The factor e^r is the volume distortion — probabilities per unit of price become probabilities per unit of return, and the two densities' shapes genuinely differ. Standard normal-to-uniform polar coordinate tricks (Box–Muller) are this theorem exercised twice.

Example 5 · The trap: forgetting the inverse's stretch, or the sign

(a) From R back to S = e^R: the rule gives f_S(s) = f_R(ln s)/s — the division by s, not multiplication; applied mechanically in the wrong direction, the "density" integrates to something other than 1, a check that catches the error (always check ∫ f = 1 — the cheapest diagnostic in the subject). (b) The decreasing-map trap: Y = 1 − X for X uniform on [0,1]. T(x) = 1 − x has derivative −1, and the correct density of Y is f_X(1 − y)·|−1| = 1 — uniform, as symmetry demands. A derivation that drops the absolute value writes the density as −1 and produces a "negative probability". Both traps are the same moral: the Jacobian rule is a volume bookkeeping theorem, and volumes are positive. Where does the positivity come from? Orientation-reversing maps (odd-dimensional mirrors, negative derivatives) flip signs that the absolute value restores — and the honest reason the formula needs the absolute value is that probabilities of sets, not signed lengths, are what pushforward measures track. Every term in a change-of-variables formula answers to a set probability; the calculus symbols are bookkeeping for that.

5.4.6The change-of-variables map of the course

The computational toolkit, assembled

Law as measure (μ_X) → all one-variable questions are real-line questions.
Change of variables (Theorem 1) → E[g(X)] needs only μ_X.
Density + LOTUS (Theorem 2) → the familiar ∫ g f dx, valid exactly when a density exists.
Generalized inverse (Theorem 3) → every law simulatable from uniforms; also the quantile function of 4.2.
Jacobian rule (Theorem 4) → coordinate changes in several variables, absolute value mandatory.

What is still missing: judgment about which laws behave — the question "when do averages exist, and when do tails destroy them" that the moving-bump examples have been circling. The next subchapter confronts it head-on: the Cauchy law, infinite means, and the heavy-tail lessons every risk manager should be able to prove.

A law is a measure, an expectation is an integral against it, and the outcome space can retire: probability now lives on the real line.

Where we are, and what is next

Chapter 5 closes with the cautionary tales: a distribution whose mean does not exist, averages of iid variables that refuse to concentrate, and the precise sense in which heavy tails break the intuition that "averaging washes everything out". The Cauchy lesson is the payload — it is the first time the course's tools prove a practical, checkable no-go result rather than a construction.