An Alternative SEM Approach for Moderation in meta-analysis

Cheung (2008) described an approach to meta-analysis within an SEM frame­work that can be used for moderator analyses as described in this chapter, as well as estimating fixed-effects means as described in Chapter 8 and more complex models (random- and mixed-effects models) described in Chapter 10. You should be aware that this is not

Practical Matters: The Limits of Interpreting Moderators in Meta-Analysis

Notwithstanding the considerable flexibility of a regression framework and the SEM approach for moderator analysis in meta-analysis, you should con­sider three potential limits when drawing conclusions from moderator analy­ses. 1. Empirically confounded Moderators Just as you want to avoid highly correlated predictors in a multiple regression analysis of primary data, it is important to

1 Comments

Differences among fixed-, random-, and Mixed-Effects Models

It is easiest to begin with the simple case in which you are interested only in the mean effect size among a set of studies, both in identifying the mean effect size and in computing its standard errors for inferential testing or for computing of confidence intervals. Even in this simple case, there are

Analyses of Random-Effects Models

A random-effects model in meta-analysis can be estimated in four general steps: (1) estimating the heterogeneity among effect sizes, (2) estimating pop­ulation variability in effect sizes, (3) using this estimate of population vari­ability to provide random-effects weights of study effect sizes, and (4) using these random-effects weights to estimate a random-effects mean effect size

Mixed-Effects Models

Mixed-effects models, sometimes called conditionally random models, com­bine the (fixed-effects) moderator analyses of Chapter 9 with the estimation of variance in population effect sizes (random-effects) described earlier in this chapter. These models are useful when you want to evaluate moderators in meta-analysis, and you (1) either want the generalizability provided by random-effects models, or

A Structural Equation Modeling Approach to Random- and Mixed-Effects Models

In Chapter 9, I introduced an alternative approach to meta-analysis based on Cheung’s (2008) description of meta-analysis within the context of struc­tural equation modeling. Here, I extend the logic of this approach to describe how it can be used to estimate random- and mixed-effects models (follow­ing closely the presentation by Cheung, 2008). As when

Practical Matters: Which Model Should I Use?

In Sections 10.1 and 10.2, I have presented the random-effects model for estimating mean effect sizes, which can be contrasted with the fixed-effects model I described in Chapter 8. I have also described (Section 10.3) mixed- effects models, in which (fixed) moderators are evaluated in the context of conditional random heterogeneity; this section can

The Problem of Publication Bias of Meta-Analysis

Publication bias refers to the possibility that studies finding null (absence of statistically significant effect) or negative (statistically significant effect in opposite direction expected) results are less likely to be published than studies finding positive effects (statistically significant effects in expected direction).1 This bias is likely due both to researchers being less motivated to

Managing Publication Bias of Meta-Analysis

In this section, I describe six approaches to managing publication bias within meta-analysis. I also illustrate some of these approaches through the example meta-analysis I have used throughout this book: a review of 22 studies report­ing associations between relational aggression and peer rejection among chil­dren and adolescents. In Chapter 8, I presented results of

Practical Matters: What Impact Do Sampling Biases Have on Meta-Analytic Conclusions?

The short answer to the question, “What impact do sampling biases have on the conclusions of a meta-analysis?” is “I don’t know.” As a meta-analyst you do not know, your readers do not know, and it is not possible to know unless you could obtain every study that has ever been conducted on the

Meta-Analysis to Obtain Sufficient Statistics

1. Sufficient Statistics for Multivariate Analyses As you may recall (fondly or not) from your multivariate statistics courses, nearly all multivariate analyses do not require the raw data. Instead, you can perform these analyses using sufficient statistics—summary information from your data that can be inserted into matrix equations to provide esti­mates of multivariate parameters.

Two Approaches to Multivariate Meta-Analysis

Given the challenges I described in the previous section, multivariate meta­analysis is considerably more complex than simply synthesizing several corre­lations to serve as input for a multivariate analysis. The development of models that can manage these challenges is an active area of research, and the field has currently not resolved which approach is best.

Practical Matters: The Interplay between Meta-Analytic Models and Theory

As with any data-analytic approach, meta-analytic techniques are most valu­able when applied in the service of theories relevant to the content of your review. I place this discussion on the interplay between meta-analysis and theory in this chapter on multivariate meta-analysis because many of our theories are multivariate and therefore benefit from multivariate analyses.

Dimensions of Literature Reviews, Revisited in writing Meta-Analytic Results

Before I turn to specific recommendations for writing the results of your meta-analysis, it is important for you to recognize that there is no single “right” way to write these results. As I described in Chapter 1 (see also Coo­per, 1988), literature reviews vary along several dimensions. Before you begin to write the results

What to Report and Where to Report It

In this section, I discuss the basic structural sections of a manuscript and special considerations in reporting meta-analytic results within these sec­tions. Two caveats are in order here. First, I expect that you are aware of the ways that manuscripts (whether primary studies or meta-analyses) are struc­tured within your field, in terms of what

Using Tables and Figures in Reporting Meta-Analyses

Tables and figures, if used effectively, can provide a large amount of data in an informative way, as well as reduce the burden of describing all of this information within the text (though you should not omit key findings from the text just because they are also displayed in tables or figures). In this

Practical Matters: Avoiding Common Problems in Reporting Results of Meta-Analyses

In this section, I identify 10 problems that I perceive to be common in report­ing results of meta-analytic reviews. More importantly, I offer concrete sug­gestions for how you can avoid each. Although following these suggestions will not guarantee that your meta-analytic report will be successful (whether defined by publication in a top-outlet, high-impact, or