Chapter 2 · Subchapter 1

Axioms and Immediate Consequences

We have a sample space and a σ-algebra of events. Now the numbers: what a probability measure is, and everything that follows from the definition by pure bookkeeping — each rule proved, each rule tested on both sides.

In this subchapter

  1. The definition: a function that eats events
  2. First consequences: complement, monotonicity, inclusion–exclusion
  3. The union bound and how loose it is
  4. Continuity of probability: limits of events
  5. Where we are, and what is next

2.1.1The definition: a function that eats events

Chapters 0 and 1 built the stage: outcomes Ω, events F. The shopping list of 0.7 demanded that probabilities (a) total 1, (b) add over disjoint events — including countably many, and (c) live on the sets in F and no others. The definition is exactly that list, nothing more:

Definition — probability measure

Let (Ω, F) be a measurable space (1.1). A probability measure is a function P whose inputs are events and whose outputs are numbers, P : F → [0, 1], satisfying:

  1. Normalization: P(Ω) = 1 — something happens.
  2. Countable additivity: for every sequence A₁, A₂, A₃, … of disjoint events (no two share an outcome), P(⋃n An) = ∑n P(An).

The triple (Ω, F, P) is the probability space promised in 0.7. Read the arrow: P is defined only on F — asking for P(A) of a non-event is a category error, like asking for the age of a color. The sum on the right is an infinite series of numbers in [0,1]; its total is automatically well-defined (possibly infinite a priori, but clause 1 caps it — see 2.1.2).

Example 1 · Fair dice, formally

Two fair dice: Ω = {(i,j) : 1 ≤ i, j ≤ 6}, F = 2Ω (all subsets — safe on finite spaces, 1.1), and P(A) = |A|/36. Check the axioms: P(Ω) = 36/36 = 1; and if A₁, A₂, … are disjoint, the sizes add (only finitely many are non-empty here), so the counting measure is countably additive. The 101 engine is a special case of the axioms — not a rival of them.

Example 2 · A loaded die — non-uniformity is free of charge

One die, biased so that 6 comes up twice as often as each other face. Assign P({1}) = … = P({5}) = 1/7, P({6}) = 2/7, and for a general event A, P(A) = the sum of the weights of the faces in A. Normalization: 5·(1/7) + 2/7 = 1. Countable additivity: disjoint events have disjoint face sets, and sums of weights add. This is a fully legal probability measure, no symmetry anywhere — the axioms never demanded uniformity; that was a 101 accident.

Example 3 · The first-head measure on the naturals

Flip a fair coin until the first head, and record the toss number: Ω = {1, 2, 3, …}, F = 2Ω (countable space, all subsets are events). Assign P(A) = ∑n∈A 2⁻ⁿ — add up the probabilities of the toss numbers in A. Normalization is the geometric series 1/2 + 1/4 + 1/8 + … = 1. Countable additivity is the statement that a double sum of non-negative numbers can be regrouped — this is where a 101 course would wave its hands; we will have the theorem for it from 5.2 and note it as a debt here. This P puts positive mass on every outcome, yet every single outcome has probability less than 1/2, and the "eventually a head" sentence of 0.4 now has an exact price: P(Ω) = 1.

Example 4 · A measure on the countable–cocountable σ-algebra

On Ω = ℝ with the σ-algebra of 1.1 (events = countable sets and cocountable sets), define P(A) = 0 if A is countable and P(A) = 1 if A is cocountable. Check: P(ℝ) = 1 (cocountable). Disjoint sequences: at most one member of a disjoint family can be cocountable (two cocountable sets must overlap — their complements are countable, and two countable sets cannot cover ℝ), so the union's probability is either 0 = 0 + 0 + … (all countable; the union of countably many countable sets is countable) or 1 = 1 + 0 + 0 + …. A perfectly legal probability on an uncountable space — with no connection whatsoever to length. Different σ-algebra, different measure, different probability theory on the same Ω: the triple, not the sample space, is the model.

Example 5 · Not every proposed P survives the axioms

On the naturals try P(A) = 1 if A is infinite, else P(A) = |A|/10. Normalization holds: Ω is infinite, so P(Ω) = 1. Countable additivity dies immediately: Ω is the disjoint union of the singletons, each of weight 1/10, and 1/10 + 1/10 + … diverges while P(Ω) = 1. Also P({n}) = 1/10 > 0 for eleven singletons already totals above 1 — the axioms are not a rubber stamp; they reject assignments that no coin could realize. The failed candidate of 0.2 ("uniform on the naturals") is exactly an attempt that the axioms reject before you even build it.

2.1.2First consequences: complement, monotonicity, inclusion–exclusion

Everything in this section follows from the two axioms by bookkeeping. That is not a dismissal — these six rules are the load-bearing walls of daily practice, and seeing them proved shows how little the axioms actually assume.

Proposition 1 — the basics

Let (Ω, F, P) be a probability space and A, B ∈ F. Then:

  1. P(∅) = 0;
  2. P(Ac) = 1 − P(A);
  3. (Monotonicity) if A ⊆ B then P(A) ≤ P(B);
  4. P(A) ≤ 1 for every event;
  5. (Finite additivity) for disjoint A, B: P(A ∪ B) = P(A) + P(B);
  6. (Inclusion–exclusion) in general: P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
Proof

(1) The sets ∅, ∅, ∅, … are disjoint and their union is ∅, so P(∅) = P(∅) + P(∅) + …; the only finite number equal to an infinite sum of copies of itself is 0. (2) A and Ac are disjoint with union Ω, so 1 = P(Ω) = P(A) + P(Ac). (3) Write B as the disjoint union A ∪ (B ∖ A) (the set difference is an event: 1.1 clause 2 + clause 3); then P(B) = P(A) + P(B ∖ A) ≥ P(A), since probabilities are non-negative. (4) A ⊆ Ω, so P(A) ≤ P(Ω) = 1. (5) Take the disjoint sequence A, B, ∅, ∅, …: countable additivity gives P(A ∪ B) = P(A) + P(B) + 0 + 0 + … — padding with empty sets is how finite sums ride for free on countable additivity. (6) Split A ∪ B = (A ∖ B) ∪ (B ∖ A) ∪ (A ∩ B), three disjoint pieces; apply (5) twice and collect: P(A ∪ B) = [P(A) − P(A ∩ B)] + [P(B) − P(A ∩ B)] + P(A ∩ B) = P(A) + P(B) − P(A ∩ B).

Example 6 · Inclusion–exclusion on the dice grid

Two fair dice. Let A = "first die is 6" and B = "second die is 6". Counting cells: P(A) = 6/36, P(B) = 6/36, and the intersection is the single cell (6,6): P(A ∩ B) = 1/36. So the probability of at least one six is 6/36 + 6/36 − 1/36 = 11/36 — not 12/36, because the double-six cell was counted twice. The formula is bookkeeping that respects geometry.

die 1 → die 2 ↓ A ∩ B — counted twice by naive addition A : 6 cells B : 6 cells P(A ∪ B) = 6/36 + 6/36 − 1/36 = 11/36
The two-dice sample space, 36 cells. Column = A (first die 6), row = B (second die 6); the bright cell is the overlap that inclusion–exclusion must subtract.

2.1.3The union bound and how loose it is

Proposition 2 — the union bound (Boole's inequality)

For any events A₁, …, An (no disjointness assumed): P(A₁ ∪ … ∪ An) ≤ P(A₁) + … + P(An).

Proof (by disjointification)

Set B₁ = A₁ and Bk = Ak ∖ (A₁ ∪ … ∪ Ak−1): what is new in Ak. The Bk are disjoint, their union is the same as the union of the Ak, and Bk ⊆ Ak so P(Bk) ≤ P(Ak) by monotonicity. Then

P(⋃k Ak) = P(⋃k Bk) = ∑k P(Bk) ≤ ∑k P(Ak).

The pattern "disjointify, then add" is the single most used trick in the subject — remember it as a move, not a formula.

Example 7 · The bound earning its keep

A portfolio has 10 independent positions, each defaulting with probability 0.01. What is the chance at least one defaults? Exact answer: 1 − 0.99¹⁰ ≈ 0.0956. Union bound: ≤ 10 × 0.01 = 0.1. Here the bound is nearly tight, because the events are rare and overlap little — and it costs no independence assumption, no distributional assumptions, nothing. That is the practical appeal: it is assumption-free and usually good enough as an upper bound.

Example 8 · The bound telling you nothing

Now three events, each of probability 0.5, all equal to the same event A ("the single coin lands heads", written three ways). The union is just A, of probability 0.5; the bound reports P ≤ 1.5 — true, and worthless, and not even better than the trivial bound P ≤ 1. The union bound degrades exactly as events overlap: it is tight for near-disjoint rare events and vacuous for heavily overlapping ones. Any time you see "≤ sum of probabilities" used to certify safety, ask how much the events share — the bound cannot tell.

2.1.4Continuity of probability: limits of events

Here is the payoff of countable additivity — a property no finitely additive assignment can deliver, and the precise tool that lets probabilities commute with limits of events.

Definition — increasing and decreasing sequences of events

A sequence of events A₁ ⊆ A₂ ⊆ A₃ ⊆ … is increasing; write An ↑ A where A = ⋃n An ("each event adds outcomes; A is everything eventually included"). A sequence A₁ ⊇ A₂ ⊇ A₃ ⊇ … is decreasing; write An ↓ A where A = ⋂n An ("each event removes outcomes; A is what survives all removals").

A₁ A₂ A₃ A₄ Aₙ ↑ ⋃ₙ Aₙ — each ring Bₖ = Aₖ ∖ Aₖ₋₁ adds fresh outcomes A₁ A₂ A₃ Aₙ ↓ ⋂ₙ Aₙ — nested removals
Left: an increasing sequence as nested sets — probability arrives ring by ring. Right: a decreasing sequence — probability drains layer by layer.
Theorem 3 — continuity of probability

(From below) If An ↑ A, then P(An) ↑ P(A): the probabilities converge upward to the probability of the limit event. (From above) If An ↓ A, then P(An) ↓ P(A).

Proof — from below, by disjointification

Slice the union into disjoint rings: B₁ = A₁, Bn = An ∖ An−1 for n ≥ 2. The Bn are disjoint (each ring is used exactly once), each Bn is an event (1.1: differences and countable unions of events are events), ⋃n Bn = A, and An = B₁ ∪ … ∪ Bn — the n-th set is exactly the first n rings. So

P(A) = ∑n=1∞ P(Bn) = limn→∞ ∑k=1n P(Bk) = limn→∞ P(An)

by countable additivity plus the definition of an infinite series as the limit of its partial sums. From above: apply the from-below case to the complements Anc ↑ Ac, then convert back with P(An) = 1 − P(Anc). This is where the probability bound P ≤ 1 quietly carries weight — the conversion needs the complements' probabilities to stay finite, and clause 1 guarantees it.

Example 9 · The first head, priced by continuity

Back to the coin. Let AN = "a head appears by toss N" — the finite, perfectly 101-computable event with P(AN) = 1 − 2⁻ᴺ. The sequence A₁ ⊆ A₂ ⊆ … is increasing, and its limit is "a head eventually appears" — the event of 0.4, which is a countable union and so lives only in the measure-theoretic world. Continuity from below hands us the price without any new computation:

P(eventually a head) = P(⋃N AN) = limN P(AN) = limN (1 − 2⁻ᴺ) = 1.

This is the working pattern for all "eventually" statements: price the finite approximations, then let continuity transport the limit. The Borel–Cantelli lemmas of 6.3 are two more instances of the same transport.

Example 10 · Continuity from above is a probability privilege

The from-above clause can fail for measures that are allowed to take the value ∞. Take Lebesgue measure λ on the whole real line (length, from 3.2, now on all of ℝ) and the decreasing sequence An = [n, ∞): tails marching to the right. Each λ(An) = ∞, but the limit event ⋂n An = ∅ — no number is larger than every n — with length 0. So λ(An) ↛ λ(∅): ∞ never descends to 0. The from-above proof needed P(A₁) < ∞, and probability measures have it free (clause 1). The mismatch between "mass escapes to infinity" and "mass disappears" is exactly why probability theory, with its normalization, is cleaner than general measure theory — and why renormalizing tricks (turning tails into probability 1 by conditioning) will keep reappearing from Chapter 8 onward.

Where we are, and what is next

The axioms plus six bookkeeping rules plus continuity: with just this, an enormous amount of practice is already rigorous — every finite computation you ever did is now a theorem. In 2.2 we exploit continuity systematically (limits of events, lim sup and lim inf of event sequences — the grammar of "infinitely often"), in 2.3 we classify the measures that concentrate on points (atoms) and see why discrete models are exactly the countable-space ones, and in 2.4 we meet the finitely additive monsters that the countable axiom keeps out — and see precisely what goes wrong when they get in.

Powered by Webloyt | Report abuse