Law of Large Numbers

As the number of trials grows, the sample mean converges to the true population mean — the mathematical foundation of why probability works.

Short runs wobble; long runs settle near the true probability

Current run

20 flips: 11 heads, 9 tails

55.0%

target for a fair coin: 50%

flips 20

The law does not promise every short run looks fair. It says the running proportion settles down as evidence accumulates.

Definition

The Law of Large Numbers (LLN) states that as you repeat an experiment more and more times, the sample average converges to the true expected value.

If $X_1, X_2, \ldots$ are independent and identically distributed with mean $\mu = E[X]$ , then:

$\bar{X}_n = \frac{X_1 + X_2 + \cdots + X_n}{n} \to \mu \quad \text{as } n \to \infty$

In plain terms: run it enough times, and the average stabilises.

Coin flips converging to 0.5

Flip a fair coin. The probability of heads is $\mu = 0.5$ .

After 10 flips: you might get 7 heads (70%).
After 100 flips: probably somewhere between 44%–56%.
After 10,000 flips: almost certainly within 1% of 50%.

The LLN says the running proportion of heads will converge to $0.5$ — but it makes no guarantee about any single finite run.

Try it

A die is rolled $n$ times and the average of the outcomes is recorded. The true mean is $\mu = 3.5$ . What does the LLN predict happens to the running average as $n$ grows?

Solution

The LLN predicts that the running average $\bar{X}_n = \frac{X_1 + \cdots + X_n}{n}$ converges to $\mu = 3.5$ .

For small $n$ , wide fluctuations are possible. As $n$ grows into the hundreds and thousands, the average will get progressively closer to $3.5$ and stay there. You will rarely see the average stray far from $3.5$ for large $n$ .

Related concepts

Needs first

Mean Probability Variance

Central Limit Theorem Markov Chains Markov Chain Monte Carlo Random Walks Sampling

View in full concept graph →