Type I and Type II Errors

The null and alternative hypotheses are competing statements about the population. Either the null hypothesis H₀ is true or the alternative hypothesis H_a is true, but not both. Ideally the hypothesis testing procedure should lead to the acceptance of H₀ when H₀ is true and the rejection of H₀ when H_a is true. Unfortunately, the correct conclusions are not always possible. Because hypothesis tests are based on sample information, we must allow for the possibility of errors. Table 9.1 illustrates the two kinds of errors that can be made in hypothesis testing.

The first row of Table 9.1 shows what can happen if the conclusion is to accept H₀. If H₀ is true, this conclusion is correct. However, if H_a is true, we make a Type II error; that is, we accept H₀ when it is false. The second row of Table 9.1 shows what can happen if the conclusion is to reject H₀. If H₀ is true, we make a Type I error; that is, we reject H₀ when it is true. However, if H_a is true, rejecting H₀ is correct.

Recall the hypothesis testing illustration discussed in Section 9.1 in which an automobile product research group developed a new fuel injection system designed to increase the miles-per-gallon rating of a particular automobile. With the current model obtaining an average of 24 miles per gallon, the hypothesis test was formulated as follows.

The alternative hypothesis, H_a: μ > 24, indicates that the researchers are looking for sample evidence to support the conclusion that the population mean miles per gallon with the new fuel injection system is greater than 24.

In this application, the Type I error of rejecting H₀ when it is true corresponds to the researchers claiming that the new system improves the miles-per-gallon rating (μ > 24) when in fact the new system is not any better than the current system. In contrast, the Type II error of accepting H₀ when it is false corresponds to the researchers concluding that the new system is not any better than the current system (μ — 24) when in fact the new system improves miles-per-gallon performance.

For the miles-per-gallon rating hypothesis test, the null hypothesis is H₀: μ — 24. Suppose the null hypothesis is true as an equality; that is, μ = 24. The probability of making a Type I error when the null hypothesis is true as an equality is called the level of significance. Thus, for the miles-per-gallon rating hypothesis test, the level of significance is the probability of rejecting H₀: μ ≤ 24 when μ = 24. Because of the importance of this concept, we now restate the definition of level of significance.

The Greek symbol a (alpha) is used to denote the level of significance, and common choices for a are .05 and .01.

In practice, the person responsible for the hypothesis test specifies the level of signific­ance. By selecting a, that person is controlling the probability of making a Type I error.

If the cost of making a Type I error is high, small values of a are preferred. If the cost of making a Type I error is not too high, larger values of a are typically used. Applications of hypothesis testing that only control for the Type I error are called significance tests. Many applications of hypothesis testing are of this type.

Although most applications of hypothesis testing control for the probability of making a Type I error, they do not always control for the probability of making a Type II error. Hence, if we decide to accept H₀, we cannot determine how confident we can be with that decision. Because of the uncertainty associated with making a Type II error when conducting signi­ficance tests, statisticians usually recommend that we use the statement “do not reject H₀” instead of “accept H₀.” Using the statement “do not reject H₀” carries the recommendation to withhold both judgment and action. In effect, by not directly accepting H₀, the statistician avoids the risk of making a Type II error. Whenever the probability of making a Type II error has not been determined and controlled, we will not make the statement “accept H₀.” In such cases, only two conclusions are possible: do not reject H₀ or reject H₀.

Although controlling for a Type II error in hypothesis testing is not common, it can be done. In Sections 9.7 and 9.8 we will illustrate procedures for determining and controlling the probability of making a Type II error. If proper controls have been established for this error, action based on the “accept H₀” conclusion can be appropriate.

Source:  Anderson David R., Sweeney Dennis J., Williams Thomas A. (2019), Statistics for Business & Economics, Cengage Learning; 14th edition.