Motivation: Where Probability 101 Breaks
You can flip coins, shuffle cards, and pass any exam built on urns. This chapter is about the problems your toolkit cannot even phrase — and the list of requirements those problems write for us.
In this chapter
0.1What Probability 101 actually assumes
Start by being honest about what the classical framework says, because the failure cases only make sense once the assumptions are on the table. Every 101 argument you have ever made — cards, dice, urns, coin flips — runs on the following engine.
A probability model in the 101 style is: a sample space S — the set of all possible outcomes — together with the rule that each outcome is equally likely, and
P(A) = (number of outcomes in A) / (number of outcomes in S),
for any subset A ⊆ S. Read the symbols: P(A) is "the probability that the outcome lands in A"; |A| is the number of elements of the set A; ⊆ means "is a subset of". The sets A we assign probabilities to are called events.
Unstated in that single line are four assumptions, and this chapter is the story of all four failing:
- The space is finite. |S| is a number you can write down. Counting only works on finite sets.
- Every subset is an event. We silently assigned a probability to every A ⊆ S.
- Outcomes are uniform. "Equally likely" is doing enormous work: the whole probability assignment comes from symmetry.
- Adding probabilities is a finite operation. If A and B are disjoint, P(A ∪ B) = P(A) + P(B) — for two events, or finitely many. Nothing was said about infinite collections.
Roll two fair dice. Then S has 6 × 6 = 36 equally likely ordered outcomes. The event "sum is 8" contains the outcomes (2,6), (3,5), (4,4), (5,3), (6,2), so P = 5/36. Ten seconds, correct, and no framework on earth would improve the answer. When the 101 assumptions hold, use the 101 machinery — this course will not take it away from you.
Throw a dart at a circular board and record which region it hits: bullseye, triple ring, double ring, miss. Four outcomes — but wildly unequal in size. A model that assigns each 1/4 is a valid 101 model of a terrible experiment. "Finite" is not the problem; "equally likely" is. Probability needs to survive non-uniformity — and 101 handles that only by ad-hoc patching (weights that sum to 1), never explaining where weights come from or what happens to them when we take limits.
The rest of this chapter takes aim at the assumptions one at a time, with concrete experiments, not vague warnings. Each failure adds one item to a shopping list. In 0.7 we read the list back and it is — spoiler — the definition of a probability space.
0.2Infinitely many outcomes: a uniform random real
Here is the simplest experiment 101 cannot handle. Spin the pointer below. When it stops, record its angle as a number X between 0 and 1: the fraction of a full turn, measured clockwise from the mark. The shaded half is the winning zone we will use later.
The outcome space is the entire interval [0, 1] — every real number in it, uncountably many. Already assumption 1 is dead: there is no |S| to divide by. And the situation is worse than "uncountable": there is no uniform way to pick outcomes that counting could imitate.
Try to salvage it anyway. Symmetry says no point is special — every x has the same probability p of being the outcome. But p cannot be a positive number: if p > 0, then outcomes 0, 0.9, 0.99 alone would already carry probability 3p, and enough points would push the total past 1. So p = 0. Every single outcome has probability zero.
Yet the pointer stops somewhere. So P([0,1]) = 1 even though P({x}) = 0 for every single point x.
Probability zero does not mean impossible. The event {x} — "the pointer stops exactly at x" — is perfectly possible; it just has probability 0. And its converse is coming in 0.5: probability one does not mean certain. In 101 these phrases coincided, because finite spaces let you add up all the little probabilities and recover the whole. Infinity breaks the identification, permanently.
Suppose we could "pick a natural number uniformly at random" — a counting-style distribution on S = {1, 2, 3, …} where every number has the same probability p. Either p = 0, and then the probability that some number is picked is 0 + 0 + 0 + … = 0 (an infinite sum of zeros is zero) — the experiment never happens; or p > 0, and the first ⌈1/p⌉ + 1 outcomes already carry total probability above 1. Both options contradict the experiment. No such distribution exists. Counting intuitions about "uniform over an infinite set" are not merely hard — they are self-contradictory.
For the spinner, the honest assignment is by length: the probability of landing in an interval [a, b] should be b − a — the fraction of the circle it covers. Probability lives on sets, not on points. Already you can see what the new framework must do: assign numbers to sets, in a way that behaves well when sets are combined — and "behaves well" will have to include infinite combinations. That word, length, generalized ruthlessly, is the Lebesgue measure of Chapter 3.
0.3Events that don't exist: a first look at Vitali
Fine — probability is a function that takes a set and returns a number. In 101 the input could be any subset of S. For the spinner, can every subset of [0, 1] get a probability, consistent with length? It would be lovely. It is also false, and the reason is one of the most beautiful short arguments in mathematics. We will build it slowly, because every step is elementary — the conclusion only feels like magic if a step stays vague.
Is there an assignment P defined on every subset of [0, 1] such that: (i) P([0,1]) = 1; (ii) P is translation-invariant — shifting a set along the circle doesn't change its probability; and (iii) P is additive over disjoint sets, including countably many at once? Intuition screams yes: this is just "length" extended everywhere.
The plan: build one specific set V that any such assignment must be able to price, and show that every possible price for V leads to a contradiction. The construction has three moves: a way of sorting numbers into families, a way of picking one delegate per family, and a way of mass-producing disjoint copies.
Move 1 — sorting numbers into families
Declare two numbers x, y ∈ [0,1] to be equivalent, written x ~ y, when their difference x − y is a rational number — a ratio of integers, like 1/2, −7/3, or 0. Check it behaves like a family relation: x ~ x (difference 0, rational); if x ~ y then y ~ x (negatives of rationals are rational); and if x ~ y and y ~ z, then x − z = (x − y) + (y − z) is a sum of rationals, hence rational, so x ~ z. A relation with these three properties is called an equivalence relation, and it automatically partitions [0,1] into non-overlapping equivalence classes (the families) that together cover everything.
Take the family of 0: it contains exactly the rationals in [0,1], since x − 0 = x must be rational. So 0, 1/3, 1/2, 2/3, 1 all sit in one family (gold dots below). Now take an irrational, say √2/2 ≈ 0.7071…. Its family contains √2/2 itself, √2/2 − 1/2 ≈ 0.2071…, √2/2 − 2/3 ≈ 0.0405…, √2/2 + 1/6 ≈ 0.8738…, and so on — irrational numbers that share the same irrational "core" and differ from each other by rationals (teal dots below). Two members of different families can never have a rational difference: that is what "different families" means.
How many families are there? Every rational is in the family of 0, so different families consist of different irrational cores. There are uncountably many irrationals but only countably many rationals to glue them together: the families are in one-to-one correspondence with the irrationals (made precise in 3.4), so there are uncountably many families, each countable (each is a shifted copy of the rationals, clipped to the interval). The interval is a union of uncountably many countable families — like a deck of uncountably many thin countable cards.
Move 2 — one delegate per family
From each family, choose exactly one member — call it the delegate. (For the family of 0, say we choose 0; for the family of √2/2, the number √2/2 itself; the choices for the other families are made once and fixed.) The set of all delegates is the Vitali set V. Two properties are guaranteed by construction, and the whole argument runs on them:
- No two delegates are related. If v, w ∈ V with v ≠ w, then v − w is not rational — otherwise v and w would be in the same family, which was only allowed one delegate.
- Every number has its family's delegate at rational distance. For any x ∈ [0,1], its family's delegate v ∈ V satisfies x ~ v, i.e. q = x − v is rational, and since both live in [0,1] we have q ∈ [−1, 1].
Move 3 — mass-producing copies
The rationals in [−1, 1] form a countable set, so they can be listed: q₁ = 0, q₂ = 1, q₃ = −1, q₄ = 1/2, q₅ = −1/2, q₆ = 1/3, … — a single infinite list, exhausting every rational in the range. For each n, shift the entire delegate set: V + qn means {v + qn : v ∈ V}, every delegate nudged by the same rational. Translation-invariance will force every copy to have the same probability as V itself — the copies are congruent, so they must weigh the same.
The copies are disjoint. Suppose x lay in both V + qm and V + qn. Then x = v + qm = w + qn for two delegates v, w ∈ V, so v − w = qn − qm, a rational. By property 1, delegates at rational distance must be the same delegate: v = w. But then qn = qm, so m = n — it was the same copy after all. Distinct copies never touch.
The copies cover everything. Take any x ∈ [0,1]. By property 2, x = v + q for its family's delegate v and some rational q ∈ [−1,1] — and every such q is somewhere on the list, q = qn. So x ∈ V + qn. The countable pile of copies blankets the whole interval (and spills beyond its ends, since a copy can stick out past 0 or 1).
The trap closes
Now price the pile. Let p = P(V) — the number the hypothetical all-subsets assignment gives to the delegate set. Translation-invariance gives every strip weight p; disjointness plus countable additivity gives the union of the strips weight p + p + p + …. The covering fact squeezes this union from both sides:
Read the inequality in the middle twice. If p = 0: the pile weighs nothing, yet it covers something of weight 1 — 1 ≤ 0, absurd. If p > 0: the pile weighs more than any finite number of strips' worth, so infinitely much — yet it sits inside a window of weight 3 — ∞ ≤ 3, absurd. Every possible price for V contradicts the three demands. Conclusion: the demands cannot all be met. There is no length-like probability defined on all subsets of [0,1].
Not "we don't know how"; there isn't one. If we want length-like probability, some sets must be denied membership in the club of events. The club has a name — a σ-algebra, Chapter 1 — and choosing it is not bureaucratic pedantry; it is the only way the spinner gets a model at all. Notice, too, where the contradiction needed help: it used countable additivity on the pile of strips. A merely finitely additive assignment would slip this particular trap — and walk into others instead; we meet those monsters in 2.4.
Intervals, open sets, closed sets, countable unions and intersections of them, sets built from any formula of inequalities — all of these can consistently carry length. In Chapter 3 we will see exactly how far the club extends (the Borel sets, then a completion), and the argument above will return in full dress in 3.4, where we verify that the delegate set itself is not in the club. The point here: the delegates are a deliberately pathological object no finite experiment ever produces — and mathematics still has to account for it.
0.4From finite to countable: the price of infinity
Assumption 4 said probabilities add over disjoint events — for two events, or any finite family. The moment outcomes form an infinite sequence, the experiments themselves demand infinitely many additions at once.
Flip a fair coin until the first head appears. The outcome space is {1, 2, 3, …}: the toss number on which the first head lands. The event "first head at toss n" is one specific sequence of n − 1 tails then a head, with probability (1/2)ⁿ. Now the sentence "the coin eventually shows a head" is the union of all these disjoint events, and its probability is
That is an infinite sum of probabilities of disjoint events. If probability were only finitely additive, "eventually a head" would be one of the infinitely many events that get no probability at all: no finite partial union ever contains it. The experiment makes perfect sense; a framework that cannot assign it a number has failed the experiment, not vice versa. So we promote finite additivity to countable additivity: probabilities add over any sequence of disjoint events.
Countable additivity is a genuine strengthening, and it has a price: the collection of events must contain the results of all these infinite unions and intersections, or the bookkeeping collapses. That closure requirement is precisely what a σ-algebra enforces — and it is why the formalism of Chapter 1 is not optional decoration.
On the natural numbers define P(A) = 0 if A is finite, and P(A) = 1 if A is infinite. This is consistent for finite unions of disjoint sets: finitely many disjoint sets cannot all be infinite, so the sums work out. It even matches a naive "almost all naturals" intuition. But it is not countably additive: each singleton {n} has probability 0, yet their union {1, 2, 3, …} has probability 1, and 0 + 0 + 0 + … ≠ 1. A more careful construction (via "natural density" on the naturals, and its cousins via Banach limits) repairs finite additivity but still fails countable additivity; we meet these monsters in 2.4. The lesson: without countable additivity, "the probability that infinitely many things happen" is not merely unknown — it is unstatable, and limit statements about experiments cannot be written down.
Some of the most important statements in probability are about what happens eventually or infinitely often. "The coin shows heads infinitely often" is literally ⋂n ⋃k≥n of "head on toss k" — countably many unions inside countably many intersections. The famous Borel–Cantelli lemmas (Chapter 6) compute probabilities of exactly such sets. A theory whose events are not closed under countable unions cannot even write these sentences. This, more than any technicality, is why the σ in σ-algebra earns its place.
0.5Conditioning on probability zero
Conditional probability is the workhorse of applied probability — for a practitioner, Bayes' rule is practically an identity card. The 101 definition:
Roll a die; given that the outcome is even, what is the probability it is a 6? P(6 | even) = (1/6)/(1/2) = 1/3. Clean, correct, and the backbone of everything from spam filters to credit scoring. Note the whole computation divides by P(A) > 0 — the probability of the thing we observed.
Now condition on the spinner from 0.2. A casino pays if the spinner lands in the winning zone [0, ½]. A sensor reads the exact value and tells you: X = 0.3713…, a specific real number. What is the probability of winning, given this reading?
Division by zero. Not a small number, not a rounding issue: the conditioning event has probability exactly zero. And this is not an edge case of applied work — it is the typical case. Every continuous observation in finance, physics, and engineering is a point of probability zero. The 101 formula for conditioning — the formula daily practice depends on — has, in the continuous world, no domain of validity at all.
Suppose the sensor is coarse: it reports only the first decimal digit. Given the reading "0.3", the value lies in an interval of length 1/10, and the answer is a ratio of lengths — well-defined. Shrink the sensor's resolution: intervals of length 1/100, 1/1000, … The answers converge, and for well-behaved models the limit is exactly what intuition wants. The measure-theoretic fix — conditional expectation given a random variable, Chapter 8 — makes this limiting procedure precise instead of hand-wavy: one takes a limit of coarse observations, and a theorem (not a hope) says the limit exists. The naive formula survives as the motivation, not the definition.
"Draw a random chord of a circle. What is the probability it is longer than the side of the inscribed equilateral triangle?" Three classical constructions of "random chord" give three different answers: 1/3 (random endpoints on the circumference), 1/2 (random midpoint along a radius), 1/4 (random midpoint in the disk). All three are "uniform" in a natural sense; all three are different probability models attached to the same sentence of English. In 101 the sample space came with probabilities attached, so this could not happen. In the continuous world, "at random" is not a specification until you exhibit the measure — and Bertrand's trio are exhibits A, B, and C. (Full resolution in 6.4.)
0.6Frequencies need a foundation
Every practitioner believes in one theorem above all others: sample averages converge to expectations. 101 asserts it (in its finite-variance folklore form) and applies it daily. But look at what the assertion involves once outcomes are infinite.
Flip a fair coin forever. A single outcome is an infinite sequence like HTTHTHH…. The sample space is the set of all such sequences — uncountable, with the same "uniform point probability zero" problem as the spinner. The statement "the fraction of heads converges to 1/2" concerns the behavior of averages along an infinite sequence — a limit of random quantities, defined on an uncountable space. To even say it, we need: events closed under countable operations (for "converges"), a measure on uncountable spaces (for the model to exist), and an integral (for "average" to mean something when outcomes cannot be summed by hand). All three are the subjects of Chapters 1–5.
Here is why the theorem needs hypotheses, and why a practitioner should care. The Cauchy distribution — the law of X = tan(Θ) for a uniform angle Θ — is symmetric, bell-shaped, and treacherous: its tails are so heavy that its expectation does not exist (the defining integral diverges, Chapter 5). Sample from it and average: the sample mean of n Cauchy observations has the same distribution as one single draw, for every n. Average a thousand samples and you have learned nothing at all — the average does not settle down as you collect more data, because one wild draw can seize the whole sum. The Law of Large Numbers fails here not because the theorem is fragile but because a hypothesis fails — and stating that hypothesis requires the integral we build in Chapter 5. A quant who averages Cauchy-ish returns without knowing this is reading a signal that is pure noise.
Even the folklore theorems — frequencies converge, averages learn — are statements about limits on infinite spaces, and they are true only under hypotheses formulated with integrals and measures. 101 gets to use them the way a tenant gets to live in a house: the building exists, and this course is the tour of the foundations.
0.7The shopping list: anatomy of (Ω, F, P)
Collect the requirements each failure imposed, and read them back. This is the definition the rest of the course unfolds — arrived at, not announced.
A probability space is a triple (Ω, F, P) where:
- Ω (capital omega) is the sample space: the set of all possible outcomes — finite, countable, or uncountable. [0,1] for the spinner; infinite sequences for the coin. Repairs assumption 1.
- F is a σ-algebra on Ω: a collection of subsets of Ω containing Ω itself, closed under complements, and closed under countable unions. Its members are the events: the only sets to which we assign probabilities. Repairs assumptions 2 and 4: not every set is an event (Vitali), and events must survive the countable operations that limit statements require.
- P : F → [0,1] is a probability measure: P(Ω) = 1, and for disjoint events A₁, A₂, …, P(⋃ Aₙ) = ∑ P(Aₙ) — countable additivity. Repairs assumption 3: weights may be uniform, weighted, or spread continuously by a measure — but they must add up, even infinitely.
Ω is a capital omega, the last letter of the Greek alphabet — mathematicians reach for the end of the alphabet for "the space of everything". F is a script F, for family of sets; some books write 𝒜 or 𝓕. P is a function whose inputs are sets and whose outputs are numbers — get comfortable with that: from now on, probabilities are measures of sets.
Map each failure to its repair, and you have the course's table of contents:
Measure-theoretic probability is not a luxury added to the 101 theory. It is the smallest structure in which the questions you already ask — a fair spin, a long-run average, a conditional forecast — are questions at all.
Next: Chapter 1.1 builds the first item on the list, the σ-algebra, the same way — through examples, with the proofs kept in.