The null and alternative hypotheses are competing statements about the population. Either the null hypothesis H0 is true or the alternative hypothesis Ha is true, but not both. Ideally the hypothesis testing procedure should lead to the acceptance of H0 when H0 is true and the rejection of H0 when Ha is true. Unfortunately, the correct conclusions are not always possible. Because hypothesis tests are based on sample information, we must allow for the possibility of errors. Table 9.1 illustrates the two kinds of errors that can be made in hypothesis testing.
The first row of Table 9.1 shows what can happen if the conclusion is to accept H0. If H0 is true, this conclusion is correct. However, if Ha is true, we make a Type II error; that is, we accept H0 when it is false. The second row of Table 9.1 shows what can happen if the conclusion is to reject H0. If H0 is true, we make a Type I error; that is, we reject H0 when it is true. However, if Ha is true, rejecting H0 is correct.
Recall the hypothesis testing illustration discussed in Section 9.1 in which an automobile product research group developed a new fuel injection system designed to increase the miles-per-gallon rating of a particular automobile. With the current model obtaining an average of 24 miles per gallon, the hypothesis test was formulated as follows.
The alternative hypothesis, Ha: μ > 24, indicates that the researchers are looking for sample evidence to support the conclusion that the population mean miles per gallon with the new fuel injection system is greater than 24.
In this application, the Type I error of rejecting H0 when it is true corresponds to the researchers claiming that the new system improves the miles-per-gallon rating (μ > 24) when in fact the new system is not any better than the current system. In contrast, the Type II error of accepting H0 when it is false corresponds to the researchers concluding that the new system is not any better than the current system (μ — 24) when in fact the new system improves miles-per-gallon performance.
For the miles-per-gallon rating hypothesis test, the null hypothesis is H0: μ — 24. Suppose the null hypothesis is true as an equality; that is, μ = 24. The probability of making a Type I error when the null hypothesis is true as an equality is called the level of significance. Thus, for the miles-per-gallon rating hypothesis test, the level of significance is the probability of rejecting H0: μ ≤ 24 when μ = 24. Because of the importance of this concept, we now restate the definition of level of significance.
The Greek symbol a (alpha) is used to denote the level of significance, and common choices for a are .05 and .01.
In practice, the person responsible for the hypothesis test specifies the level of significance. By selecting a, that person is controlling the probability of making a Type I error.
If the cost of making a Type I error is high, small values of a are preferred. If the cost of making a Type I error is not too high, larger values of a are typically used. Applications of hypothesis testing that only control for the Type I error are called significance tests. Many applications of hypothesis testing are of this type.
Although most applications of hypothesis testing control for the probability of making a Type I error, they do not always control for the probability of making a Type II error. Hence, if we decide to accept H0, we cannot determine how confident we can be with that decision. Because of the uncertainty associated with making a Type II error when conducting significance tests, statisticians usually recommend that we use the statement “do not reject H0” instead of “accept H0.” Using the statement “do not reject H0” carries the recommendation to withhold both judgment and action. In effect, by not directly accepting H0, the statistician avoids the risk of making a Type II error. Whenever the probability of making a Type II error has not been determined and controlled, we will not make the statement “accept H0.” In such cases, only two conclusions are possible: do not reject H0 or reject H0.
Although controlling for a Type II error in hypothesis testing is not common, it can be done. In Sections 9.7 and 9.8 we will illustrate procedures for determining and controlling the probability of making a Type II error. If proper controls have been established for this error, action based on the “accept H0” conclusion can be appropriate.
Source: Anderson David R., Sweeney Dennis J., Williams Thomas A. (2019), Statistics for Business & Economics, Cengage Learning; 14th edition.