Kolmogorov-Smirnov Test

Testing whether a sample comes from a specified distribution, or whether two samples come from the same distribution, using the empirical CDF.

Diagram · Empirical CDF vs Theoretical

n =

The KS statistic D = 0.195 is the largest vertical gap between the empirical CDF (step function) and the theoretical CDF (smooth curve). Larger n → closer agreement.

Definition

The Kolmogorov-Smirnov (KS) test tests whether a sample comes from a specified distribution (one-sample) or whether two samples come from the same distribution (two-sample).

Empirical CDF: for data $x_1, \ldots, x_n$ , the empirical CDF is $F_n(x) = \frac{1}{n}\#\{i: x_i \leq x\}$ .

One-sample test: compare $F_n(x)$ to the hypothesized CDF $F_0(x)$ : $D_n = \sup_x |F_n(x) - F_0(x)|$

Two-sample test: compare empirical CDFs from two samples: $D_{n,m} = \sup_x |F_n(x) - G_m(x)|$

Large $D$ is evidence against $H_0$ (same distribution).

Testing normality

Data: $\{0.5, 1.2, 2.1, 2.8, 3.5\}$ ( $n=5$ ). Test $H_0$ : standard normal.

The empirical CDF is a step function. We find the largest vertical gap between this step function and $\Phi$ (the standard normal CDF). If $D_5 = 0.32$ , the critical value at $\alpha = 0.05$ for $n=5$ is 0.565. Since $0.32 < 0.565$ , fail to reject normality.

Try it

Why should you generally not use the one-sample KS test to test normality when the mean and variance were estimated from the same data?

Solution

The KS test assumes the null distribution $F_0$ is completely specified in advance — not estimated from the data. If you estimate $\mu$ and $\sigma$ from the data, then fit $F_0 = \mathcal{N}(\hat{\mu}, \hat{\sigma}^2)$ , the estimated distribution will always fit better than a fixed one would. The KS critical values are too conservative (too large), meaning you'll fail to reject normality too often (p-values are too large).

Use the Lilliefors test instead — it's the KS test with corrected critical values for the case where parameters are estimated from the data. Or use the Shapiro-Wilk test, which is generally more powerful for testing normality.

Related concepts

Needs first

Hypothesis Testing Normal Distribution

Sampling

View in full concept graph →