Central Limit Theorem

The sample mean of a large number of independent random variables is approximately normally distributed — regardless of the population's shape.

The population can be skewed, while the distribution of sample means becomes bell-shaped

Definition

The Central Limit Theorem (CLT) states that if you take a large enough random sample from any population — regardless of the population's shape — the distribution of sample means will be approximately normal (bell-shaped).

Formally: if $X_1, X_2, \ldots, X_n$ are independent and identically distributed (i.i.d.) with mean $\mu$ and variance $\sigma^2$ , then as $n \to \infty$ :

$\bar{X} = \frac{X_1 + X_2 + \cdots + X_n}{n} \sim N\!\left(\mu,\, \frac{\sigma^2}{n}\right)$

The standard error of the mean is $\text{SE} = \dfrac{\sigma}{\sqrt{n}}$ .

This is one of the most important theorems in all of statistics.

Rolling a single die vs. averaging many rolls

A fair six-sided die has mean $\mu = 3.5$ and standard deviation $\sigma \approx 1.71$ . A single roll is uniformly distributed — not bell-shaped at all.

But average $n = 30$ rolls: by the CLT, $\bar{X} \sim N(3.5,\, 1.71^2/30) = N(3.5,\, 0.098)$ , with standard error $\approx 0.31$ .

The distribution of averages is approximately normal, tightly clustered around $3.5$ .

Try it

A population has mean $\mu = 50$ and standard deviation $\sigma = 10$ . You take a sample of $n = 25$ . What is the standard error of the mean? What distribution does $\bar{X}$ follow?

Solution

$\text{SE} = \frac{\sigma}{\sqrt{n}} = \frac{10}{\sqrt{25}} = \frac{10}{5} = 2$

By the CLT, $\bar{X} \sim N(50, 4)$ — i.e., approximately normal with mean $50$ and variance $4$ (standard deviation $2$ ).

Related concepts

Needs first

Normal Distribution Sampling Variance

Hypothesis Testing Law of Large Numbers

View in full concept graph →