Conditional Expectation: Updating Without a Formula
"Given what we know, what is the average?" — the most operational question in probability, and the one where formulas lie fastest. This subchapter builds the honest object: not a number, but a random variable — measurable with respect to the information you hold.
In this subchapter
8.1.1The partition case: computed by hand
Start where every intuition starts: information as a finite list of answers. The observer of 4.3 holds a finite partition A₁, …, Ak (disjoint events covering Ω — the atoms of 2.3 doing information duty), and asks: given the answer cell, what is the average of X?
For an integrable X and a partition with P(Aj) > 0 for all j:
A random variable: on each cell it reports the cell's own average. This is the elementary conditional mean E[X|A] (itself a definition via averaging, or — equivalently when P(A) > 0 — the ratio of expectations), assembled into a function of the outcome.
A die roll's outcome X ∈ {1,…,6}, partition = {even, odd}. Cell averages: E[X|even] = (2+4+6)/3 = 4, E[X|odd] = 3. The conditional expectation is the variable Y with Y = 4 on evens, Y = 3 on odds: a function of the parity — exactly the shape Doob–Dynkin (4.3) demands for a σ(parity)-measurable variable. Check the accounting identity: E[Y] = 4·(1/2) + 3·(1/2) = 3.5 = E[X] — the conditional expectation averages back to the unconditional one (the "tower" property, first sighting).
8.1.2What breaks when information is not a partition
Now the harder information: ([0,1], ℒ, λ), X the identity, and the information is the full σ(X) itself — "the observer knows the exact outcome". No partition into finitely many positive-probability cells describes this: the atoms are singletons, probability zero, and the formula E[X·1{ω}]/P({ω}) = ω/0 divides by zero everywhere. The elementary route is dead — and the answer obviously exists: knowing the outcome exactly, the conditional expectation of X is X itself. The definition-to-come must reproduce this ("E[X|σ(X)] = X") and everything else besides, without dividing by zero anywhere. The trick, as so often in this course: state the property, not the computation.
8.1.3The definition: averaging, not dividing
For X integrable (X ∈ L¹) and a sub-σ-algebra G ⊆ F: the conditional expectation E[X | G] is the random variable Z characterized by two properties:
- Z is G-measurable — Z is computable from the information (4.3's reading);
- E[Z·1A] = E[X·1A] for every A ∈ G — Z and X have the same average over every G-question.
Not a formula: a specification. "The G-measurable variable whose bookkeeping matches X's on every event the observer can name." A variable that passes property 2 is called a version; uniqueness is a.s. (Theorem 2 below).
For the partition case, check that the two-property definition reproduces §1's formula: G-measurability forces Z constant on each cell; taking A = Aj in property 2 forces each constant to be the cell average. The general definition is the partition one with the partitions removed — the same claim ("right average on every answer"), stated as an equality of integrals over all G-events, which makes sense even when every atom is null. The cost is abstractness; the payoff is existence in every case, including Example 2's — where property 1 forces Z to be a function of X (Doob–Dynkin, 4.3), and property 2 applied to A = {X ∈ B} pins that function to the identity: Z = X. Sanity anchors: E[X | {∅,Ω}] = E[X] (the constant: conditioning on knowing nothing returns the unconditional average — the trivial σ-algebra's one tenant, 4.3 Example 5, was preparing for this); E[X | F] = X (knowing everything).
8.1.4Existence and uniqueness
For X ∈ L¹ and any sub-σ-algebra G, a version Z exists. Any two versions agree a.s. (so "E[X|G]" denotes an equivalence class of variables, equal a.s. — the license of 4.1 §4, used freely).
Let Z, Z' be versions. Then W = Z − Z' is G-measurable with E[W·1A] = 0 for all A ∈ G. Let A+ = {W > 0} — in G (measurability); then E[W·1A+] = 0 ≥ ... the ε-move: consider An = {W > 1/n}, also in G: 0 = E[W 1An] ≥ (1/n)P(An), so P(An) = 0; union over n (2.1 continuity): P(W > 0) = 0, symmetrically P(W < 0) = 0. W = 0 a.s.
Existence is the Radon–Nikodym theorem: the measure Q(A) = E[X·1A] on G is absolutely continuous with respect to P|G, and Z is its Radon–Nikodym derivative — the next subchapter proves the theorem (in the L² case, with the full case flagged) precisely because conditioning needs it. This is the course's architecture paying forward: Chapter 8's second subchapter exists to service its first. ∎-structure
8.1.5The projection picture
For X ∈ L² (finite variance), there is a geometric reading that practitioners should keep for life: the square-integrable variables form a vector space with inner product ⟨U, V⟩ = E[UV] (positive-definite: E[U²] = 0 ⟹ U = 0 a.s.); the G-measurable variables form a subspace; and E[X|G] is the orthogonal projection of X onto that subspace — property 2 says exactly ⟨X − Z, V⟩ = 0 for all G-measurable V (check V = 1A, then simple, then MCT up). Consequences, all standard geometry: the projection is the best mean-square approximation of X by any function of the information — conditional expectation is the optimum of every forecast-loss problem with squared error; and the residual X − E[X|G] is the "surprise", uncorrelated with the entire information set. Every Kalman filter, regression, and Gauss–Markov argument in the wild is this projection run under notation changes. The abstraction of the definition was not ornament: it made conditioning a right angle.
8.1.6The properties, and their honest limits
- Linearity: E[aX + bY | G] = aE[X|G] + bE[Y|G].
- Taking out what is known: if Y is G-measurable (and integrability permits), E[XY | G] = Y·E[X|G] — the known factor passes through.
- Tower property: for H ⊆ G: E[ E[X|G] | H ] = E[X|H] — coarsening a forecast equals forecasting from the coarser information. Iterated information collapses.
- Independence: if X is independent of G, E[X|G] = E[X] — irrelevant information is ignored.
- Monotonicity, Jensen: X ≤ Y ⟹ E[X|G] ≤ E[Y|G]; for convex φ, φ(E[X|G]) ≤ E[φ(X)|G].
- Doob–Dynkin form: conditioning on a variable is clean: E[Y|X] is σ(X)-measurable, hence (by 4.3) equals g(X) for a Borel g — the "conditional mean function" of regression, existing in full generality.
Multiplication is not unconditional: in general E[XY|G] ≠ E[X|G]E[Y|G] — the identity requires Y G-measurable (take Y = X on the trivial G: E[X²] ≠ (E[X])², the variance itself, the canonical counterexample). Expectation outside, not inside: E[E[X|G]] = E[X] always (tower to the trivial σ-algebra), but E[X|G] cannot be replaced by a number anywhere else — the object is a variable, and treating it as one is the most common professional slip in applied conditioning. Independence is of σ-algebras: "X independent of Y" licenses E[X|σ(Y)] = E[X], but "X independent of σ(G, some other information)" is a different, stronger claim — conditional independence is a course of its own, and every Bayesian-network shortcut is a hypothesis about exactly this structure, checkable only by naming the σ-algebras (6.4's discipline, final form).
Conditional expectation is the best approximation of the world by the information you hold — a variable, a projection, and the object every "update" in practice should mean.
Existence of the conditional expectation rests on the Radon–Nikodym theorem — the last pillar of the course still unproved. The next subchapter proves it (the L² case honestly, by projection and completeness), reads its content as "densities relative to any measure", and exhibits the negative example that keeps it honest: the Cantor law's singular measure, absolutely continuous in no direction, for which no density exists and conditioning takes a genuinely non-elementary form.