As a review of what you read in Chapter 1 and this chapter, we now provide an extended example based on the HSB data. We will walk you through the process of identifying the variables, research questions, and approach, and then show how we selected appropriate statistics and interpreted the results.
Research problem. Suppose your research problem was to investigate the relation of gender and math courses taken to math achievement.
Identification of the variables and their measurement. The research problem specifies three variables: gender, math courses taken, and math achievement. The latter appears to be the outcome or dependent variable, and gender and math courses taken are the independent or predictor variables because they occurred before the math exam. As such, they are presumed to have an effect on math achievement scores.
What is the level of measurement for these three variables? Gender is clearly dichotomous (male or female). Math courses taken has six ordered values, from zero to five courses. These are scale data because there should be an approximately normal distribution of scores: most students took some but not all of the math courses. Likewise, the math achievement test has many levels, with more scores somewhere in the middle than at the high and low ends. It is desirable to confirm that math courses taken and math achievement are at least approximately normally distributed by determining the skewness of each.
Research questions. There are a number of possible research questions that could be asked and statistics that could be used with these three variables, including all of the types of questions in Appendix B, the descriptive statistics discussed in Chapter 3, and several of the inferential statistics presented in this chapter. However, we focus on three research questions and three inferential statistics because they answer this research problem and fit our earlier recommendations for good choices. First, we discuss two basic research questions, given the previous specification of the variables and their measurement. Then, we discuss a complex research question that could be asked instead of research questions 1 and 2.
- Is there a difference between males and females (the two levels of the variable, gender) on their average math achievement scores?
Type of research question. Using the text, Fig. 6.1, and Table 6.1, you should see that the first question is phrased as a basic difference question because there are only two variables and the focus is a group difference (the difference between the male group and the female group).
Selection of an appropriate statistic. If you examine Table 6.1, you will see that the first question should be answered with an independent samples t test because (a) the independent variable has only two values (male and female), (b) the design is between groups (males and females form two independent groups), and (c) the dependent variable (math achievement) is normal/scale data. We would also check other assumptions of the t test to be sure that they are not markedly violated.
Interpretation of the results for question 1. Let’s assume that about 50 students participated in the study and that t = 2.05. The output will give you the exact Sig. In this case, p < .05 and thus t is statistically significant. However, if you had 25 participants, this t would not have been significant (because the t value necessary for statistical significance is influenced strongly by sample size. Small samples require a larger t to be significant.).
Deciding whether the statistic is significant only means the result is unlikely to be due to chance. You still have to state the direction of the result and the effect size and/or the confidence interval (see Fig. 6.4). To determine the direction, we need to know the mean (average) math achievement scores for males and females. In the HSB data, males have the higher mean, as you will see in later chapters. Given that the difference is significant, you can be quite confident that males in the population are at least a little better at math achievement, on average, than females. So you should state that males scored higher than females. If the difference was not statistically significant, it is best not to make any comment about which mean was higher because the difference could be due to chance. Likewise, if the difference was not significant, we recommend that you report but do not discuss or interpret the effect size. [1] You should also provide the means and standard deviations so that the effect size can be better understood.
Because the t was statistically significant, we would calculate d and discuss the effect size, as shown earlier. In this situation, we would compute the pooled (weighted average) standard deviation for male and female math achievement scores. Let’s say that the difference between the means was 2.0 and the pooled standard deviation was 6.0; then d would be .33, a small to medium size effect. This means that the difference is less than typical of the statistically significant findings in the behavioral sciences. A d of .33 may or may not be a large enough difference to use for recommending programmatic changes (i.e., may or may not be practically significant).
Confidence intervals might help you decide if the difference in math achievement scores was large enough to have practical significance. For example, say you found (from the lower bound of the confidence interval) that you could only be confident that there was a 1/2 point difference between males and females. Then you could decide whether that was a big enough difference to justify, for example, a programmatic change.
- Is there an association between math courses taken and math achievement?
Type of research question. This second question is phrased as a basic associational question because there are only two variables and both have many ordered levels. Thus, use Table 6.2 for the second question.
Selection of an appropriate statistic. As you can see from Table 6.2, the second research question should be answered with a Pearson correlation because both math courses taken and math achievement are normally distributed variables.
Interpretation of the results for research question 2. The interpretation of r is based on decisions similar to those made above for t. If r = .30 (with 50 subjects), it would be statistically significant at the p < .05 level. If the r is statistically significant, you still need to discuss the direction of the correlation and effect size. Because the correlation is positive, we would say that students with a relatively high number of math courses taken tend to perform at the high end on the math achievement test and those with few math courses taken tend to perform poorly on the math achievement test. The effect size of r = .30 is medium or typical.
Note that if N were 25, the r of .30 would not be significant. On the other hand, if N were 500 and r = .30, p would be <.0001. With N = 500, even r = .10 would be statistically significant, indicating that you could be quite sure the association was not zero, but the effect size would be small, or less than typical.
Complex research question and statistics. As you will see in later chapters, there are advantages to considering the two independent variables (gender and math courses taken) together rather than separately as in questions 1 and 2. There are several statistics that you will compute that could be used to consider gender and math courses taken together. A research question which subsumes both questions 1 and 2 could be:
- Is there a combination of gender and math courses that predicts math achievement?
Selection of an appropriate statistic. Multiple regression could be used to answer this question. As you can see in Table 6.4, multiple regression is appropriate because we are trying to predict a normally distributed/scale variable (math achievement) from two independent variables, which are math courses taken (normal or scale) and gender (a dichotomous variable).
Based on our discussion of the general linear model (GLM), a two-way factorial ANOVA would be another statistic that could be used to consider both gender and math courses taken simultaneously. However, to use ANOVA, the many levels of math courses taken would have to be recoded into a few categories or levels (perhaps high, medium, and low). Because information is lost when you do such a recode, we would not recommend factorial ANOVA for this example. Another possible statistic to use for this example is analysis of covariance (ANCOVA) using gender as the independent variable and math courses taken as the covariate; ANCOVA is discussed in Leech et al. (2005).
Interpretation of the results for research question 3. We provide an introduction to multiple regression in Chapter 8 and to factorial ANOVA in Chapter 10, but extended treatment is beyond the scope of this book (see Leech et al., 2005). For now, let’s just say that we would obtain more information about the relationships among these three variables by doing these complex statistics than by doing only the t test and correlation described earlier and in the next section.
[1] At times when one has a small sample, one would present the effect size and talk about the need to replicate the study with a larger sample.
Source: Morgan George A, Leech Nancy L., Gloeckner Gene W., Barrett Karen C.
(2012), IBM SPSS for Introductory Statistics: Use and Interpretation, Routledge; 5th edition; download Datasets and Materials.
29 Mar 2023
14 Sep 2022
28 Mar 2023
14 Sep 2022
30 Mar 2023
19 Sep 2022