Hypothesis Testing
A formal framework for deciding whether data provides enough evidence to reject a default assumption.
Hypothesis testing is a framework for using data to decide between two competing claims about the world.
- The null hypothesis is the default, conservative claim — usually "no effect" or "no difference."
- The alternative hypothesis is what you're trying to find evidence for.
The logic: assume is true, then ask "how surprising is the data I collected?" If the data would be very unlikely under , that's evidence against it.
You suspect a coin is biased. You flip it 20 times and get 15 heads.
: the coin is fair (). : the coin is biased ().
If the coin were fair, getting 15 or more heads in 20 flips would happen about 2% of the time. That's unlikely. So the data provides some evidence against the fair coin hypothesis.
A new drug is tested. The null hypothesis is that the drug has no effect on blood pressure. You observe that patients' blood pressure dropped by an average of 8 points. Is this enough information to reject ? What else would you need?
Solution
No, it's not enough on its own. You also need to know how variable blood pressure is (to judge whether 8 points is a lot or a little), how many patients were in the study (more patients → more reliable estimate), and the size of the typical placebo effect. Without knowing how surprising 8 points is relative to the noise, you can't assess the evidence against .