What Is a Significance Level and a p-Value?

A significance level is the threshold chosen for deciding whether to reject the null hypothesis, while a p-value measures how likely the observed result, or something more extreme, would be if the null hypothesis were true. Compare them: if $p<\alpha$, reject the null hypothesis.

How the Decision Works

In a hypothesis test, you first state a null hypothesis, $H_0$, and an alternative hypothesis, $H_1$. The significance level, written $\alpha$, is selected before examining the test result; common values are $0.01$, $0.05$, and $0.10$.

The significance level represents the accepted probability of making a Type I error, rejecting $H_0$ when it is actually true. A significance level of $\alpha=0.05$ therefore allows a 5% risk of this error.

The p-value is then calculated using a GDC and compared with $\alpha$.

A smaller p-value indicates stronger evidence against $H_0$, but it does not measure the size or practical importance of an effect. The significance level is a decision threshold, not evidence produced by the sample. Changing $\alpha$ can therefore change the decision even though the p-value remains unchanged for that sample.

For example, suppose a two-sample t-test investigates whether two populations have different means:

$$H_0:\mu_1=\mu_2,\qquad H_1:\mu_1\neq\mu_2$$

If $\alpha=0.05$ and the GDC gives $p=0.032$, then $0.032<0.05$. Therefore, reject $H_0$. There is significant evidence that the two population means differ.

A common misconception is that the p-value is the probability that $H_0$ is true. It is not: the p-value is calculated assuming that $H_0$ is true.

IB Exam Technique

This is SL 4.11 hypothesis testing and may appear on either GDC-active paper. For an “interpret” or “justify” question, explicitly compare $p$ with $\alpha$, give the correct decision, and conclude in context. Never write “accept $H_0$”; write “insufficient evidence to reject $H_0$.”