Literature reviews need to be assessed and evaluated as strictly as empirical articles, but is this always the case? Palmatier et al. (2018) suggest that a quality literature review must have both depth and rigor, that is, it needs to demonstrate an appropriate strategy for selecting articles and capturing data and insights and to offer something beyond a recitation of previous research. In addition, they state that a quality literature review needs to be replicable, that is, the method must be described such that an external reader could replicate the study and reach similar findings. Lastly, they state that a literature review must be useful for scholars and practitioners. However, evaluating different types of literature reviews can be challenging. Therefore, some guidelines for eventuating literature review articles across approaches are suggested as a starting point to help editors, reviewers, authors, and readers evaluating literature reviews (summarized in Table 4). These depart from the different stages of conducting a literature review and should be broad enough to encompass most types of literature reviews. However, of importance is that when evaluating an individual review, specific standards for the type of review must be examined to assess whether the review meets the criteria for rigor and depth. Depending on if, it is a systematic, semi-systematic or integrative review, different standards can be valid. However, independent of type of review, pay close attention to what studies have been included and for what reasons as these decisions make all the differences in terms of what type of conclusions the authors reach. Ignoring a relevant field of research, some journals or years can have major consequences for the results and conclusions of the studies. In addition, its contribution should always be evaluated against the topic or field to which it adds. What constitutes a useful contribution in one area may be insufficient in another.
Guidelines to assess the quality of a literature review.
Phase 1: design
- In relationship to the overall research field, is this literature review needed and does it make a substantial, practical, or theoretical contribution?
- Are the motivation, the purpose, and the research question(s) clearly stated and motivated?
- Does the review account for the previous literature review and other relevant literature?
- Is the approach/methodology for the literature review clearly stated?
- Is this the most appropriate approach to address the research problem?
- Are the methodology and the search strategy clearly and transparently described and motivated (including search terms, databases used, and explicit inclusion and exclusion criteria)?
Phase 2: conduct
- Is the search process appropriate for this type of review?
- Is the practical search process accurately described and accounted for?
- Is the process of the inclusion and exclusion of articles transparent?
- Have proper measures been taken to ensure research quality?
- Can it be trusted that the final sample is appropriate and in concordance with the overall purpose of the review?
Phase 3: data abstraction and analysis
- Is the data abstracted from the article appropriate in concordance with the overall purpose of the review?
- Is the process for abstracting data accurately described?
- Have proper measures been taken to ensure quality data abstraction?
- Is the chosen data analysis technique appropriate in relation to the overall research question and the data abstracted?
- Is the analysis process properly described and transparent?
Phase 4: structuring and writing the review
- Is the review article organized coherently in relation to the overall approach and research question?
- Is the overall method of conducting the literature review sufficiently described? Can the study be replicated?
- Is the result of the review reported in an appropriate and clear way?
- Does the article synthesize the findings of the literature review into a clear and valuable contribution to the topic?
- Are questions or directions for further research included? Are the results from the review useable?
Source: Hannah Snyder (2019), “Literature review as a research methodology: An overview and guidelines”, Journal of Business Research, Volume 104, Pages 333-339, https://doi.org/10.1016/j.jbusres.2019.07.039