Measures of Central Tendency in Effect Sizes

1. Choices of Indices of Central Tendency

Turning momentarily away from the topic of weighting, I now consider the ways in which you can represent the central tendency of effect sizes from a series of studies in your meta-analysis. As with representing central tendency within a primary data analysis, you can consider the mode, median, and mean as possible indices.

The mode (the most commonly occurring value) is not a good choice for representing typical effect sizes in a meta-analysis. The problem is that effect sizes computed from multiple studies are likely to fall along such a fine-grained continuum that identifying the most commonly occurring value is meaningless. Although grouping effect sizes into categories might allevi­ate the problem, such categorizations are arbitrary and likely lead to a loss of information. In short, I view the mode as a poor choice for describing typical effect sizes in a meta-analytic review.

The median (the middle value of a rank-ordered list of values, or the 50th percentile) is a better choice for representing typical effect sizes in a meta-analysis. This value is easy to determine (e.g., in the example data of Table 8.1, the median effect size is r = .35) and is a valuable supplement in many situations because it is less influenced by skew in effect sizes than is the mean. At the same time, the disadvantage of the median, as typically computed, is that it does not differentially weight studies by the precision of their point estimates of effect sizes (see previous section). For this reason, I view the median as at best being a supplement to the mean.

The mean effect size is generally the most important index of central tendency in your meta-analyses. It is widely used in primary research, and therefore well understood by readers. Importantly, it is also possible to dif­ferentially weight effect sizes, and therefore give more weight to some studies than to others, by computing a weighted mean effect size. This weighted mean effect size (or the random-effects variant I describe in Chapter 10) represents critical information that will be reported in your meta-analytic reviews.

2. Computing the weighted Mean Effect Size

The weighted mean effect size across studies is computed from the weights (wj) and effect sizes (ESj) from each of the studies i using the following equa­tion:

In other words, the mean effect size is calculated by computing the prod­uct of each study’s effect size by its weight (creating a separate variable repre­senting this product in your database), summing these products across stud­ies, and dividing this value by the sum of weights across studies. The logic of this equation is more obvious if you consider using w = 1 for all studies, or giving equal weight to all studies. Here, the mean is simply the sum of effect sizes divided by the number of effect sizes, the traditional formula for the (unweighted) mean.

Equation 8.2 is generic in that it can be used with any of the effect sizes I have described in this book. With those effect sizes that are typically trans­formed (e.g., r is typically transformed to Z_r), this formula is applied to the transformed effect size from each study, and the average effect size is then back-transformed to the more interpretable effect size for reporting.

To illustrate the calculation of this weighted mean effect size, consider again the data in Table 8.1. In this table (just to the right of w), I have added a column showing the product of w and the effect size (here, Z_r) for each study. I then summed these values across the 22 studies (easily done within any spreadsheet or basic statistical software package) to obtain the value 2764.36, which is the numerator of Equation 8.2. Also shown at the bottom of Table 8.1 is the sum of weights (w) across the 22 studies, 7152.21, which comprises the denominator of Equation 8.2. I then compute the weighted mean effect size as Z = 2764.36/7152.21 = .387. For reporting, I would transform this mean Z_r into mean r using Equation 5.3, yielding r = .368.

This average effect size is a crucially important result of your meta­analytic review (and it wasn’t nearly as difficult to compute as you might have thought!). After you compute this average effect size, you have valuable infor­mation to describe the typical effect sizes in the area of your meta-analysis. However, it is also important to consider the effect size in terms of its statisti­cal significance and confidence intervals, the topic to which I turn next.

Source: Card Noel A. (2015), Applied Meta-Analysis for Social Science Research, The Guilford Press; Annotated edition.