Skip to Main Content

Evidence Synthesis + Systematic Reviews

Systematic Reviews, advanced search + retrieval strategies

Due Diligence

Look for existing systematic reviews within your topic to make sure you aren't duplicating existing work. Reviewing existing systematic reviews also gives you the opportunity to review approaches and methodologies within your field.

  1. Search for reviews in scholarly databases. Many databases, such as Web of Science, allow users to filter by "Review Articles" under the "Document Type" filter. Try searching for phrases such as systematic review or systematic literature review in the article title field.

Screenshot from Web of Science with arrow pointing to Review Article filter

  1. Search for systematic reviews and protocols in registries. PROSPERO is an open access online database housing systematic review protocols. CEEDER is an open access database that houses evidence syntheses related to environmental management. Campbell Collaboration houses evidence syntheses related to the effects of social interventions. If your research is interdisciplinary and crosses over with health sciences, the Cochrane Library is a good place to find systematic reviews in human health care and health policy. The EPPI Centre also houses systematic reviews in their evidence library primarily on health, education, welfare, and other public policy sectors.

Assembling Your Team

Systematic reviews cannot be performed alone. One investigator is not sufficient to reduce the risk of bias in the review process.

It is essential that Cochrane reviews be undertaken by more than one person. This ensures that tasks such as selection of studies for eligibility and data extraction can be performed by at least two people independently, increasing the likelihood that errors are detected.

Cochrane Handbook version 5.1, 2011, section 2.3.4.1

The objective of organizing the review team is to pull together a group of researchers as well as key users and stakeholders who have the necessary skills and clinical content knowledge to produce a high-quality SR.

Standard 2.1 Establish a team with appropriate expertise and experience to conduct the systematic review

Required elements:

  • Include expertise in the pertinent clinical content areas
  • Include expertise in systematic review methods
  • Include expertise in searching for relevant evidence
  • Include expertise in quantitative methods
  • Include other expertise as appropriate

- National Academies of Sciences, Engineering, and Medicine, Finding What Works in Health Care: Standards for Systematic Reviews, chapter 2, 2011.

Steps of a Systematic Review

See the further resources page for links to more in-depth resources on these steps.

  1. Planning your review
    • Choosing a topic
      • Usually this means deciding on an answerable question. The PICO framework can help you formulate a question that can be answered in the literature. PICO stands for: Patient or population, Intervention, Comparison or control, and Outcome.
    • Assembling your team
      • It is important to include team members who have clinical expertise related to the research topic. You also want to have at least one team member with expertise systematic review methodology, one team member with expertise in evidence searching (e.g. a medical librarian), and a biostatistician if you intend to perform a meta-analysis on your findings.
    • Writing a protocol (see below)
      • A protocol is critical for your process. It spells out your search plan and your inclusion and exclusion criteria for the evidence you will discover. Sticking to your previously published protocol increases transparency and reduces bias in the process of gathering evidence.
  2. Searching the evidence
    • Preliminary literature searching
      • You may need to do some scoping searches as you develop your protocol, in order to help refine your research question.
    • Final literature searching
      • Once your protocol is finalized, you can work with a medical librarian on search strategies for multiple literature databases.
    • Gray literature searching
      • Evidence may exist beyond the published literature. Gray literature searching is necessary to correct for publication bias.
  3. Analyzing your results
    • Screening
      • At least two independent screeners review titles and abstracts first, then full text.
    • Assessing quality
      • Various quality checklists (especially for RCTs) exist. You may also want to read about Cochrane's methods and risk of bias tool.
    • Extracting data
      • Data must be extracted in a structured, documented way for included studies.
  4. Synthesizing your results
    • Quantitative synthesis (i.e. meta-analysis)
      • Meta-analyses statistically combine results from multiple studies to gain more power, potentially detecting a different effect, through a larger sample size than the individual studies. A biostatistician should be part of the research team if a meta-analysis is conducted.
    • Qualitative or narrative synthesis
      • It may not be possible to perform a meta-analysis on the existing evidence. In this case, evidence can be synthesized narratively.
  5. Reporting your results
    • PRISMA
      • PRISMA is a popular reporting standard required by many journals.
    • Field-specific reporting standards (e.g. MOOSEROSES)
      • Check to see if a specialized reporting standard exists for your subfield.

Writing and Publishing Your Protocol

What is a protocol?

  • A protocol lays out your plan for the systematic review. It specifies the systematic review authors, the rationale and objectives for the review, the inclusion and exclusion criteria for study eligibility, the databases to be searched along with the search strategy, and the process for managing, screening, analyzing, and synthesizing the results.

Why write a protocol?

  • As with any other study, a systematic review needs a plan. The protocol provides the team with a road map for completion.

Why publish a protocol?

  • A published protocol makes your plan public. This accountability mitigates bias that can result from changing the research topic, or study eligibility criteria, based on results that were discovered during the study.
  • It also informs other researchers of your ongoing work, preventing possible duplication of efforts.
  • For more, read this article: Why prospective registration of systematic reviews makes sense

Guidance on writing a protocol

  • PRISMA-P is an extension of the PRISMA reporting standard for protocols
  • The Cochrane Handbook part 1, chapter 4 has information on writing Cochrane protocols

Sharing/publishing protocols

Reporting Standards

There are some reporting standards for evidence syntheses. These can serve as guidelines for protocol and manuscript preparation and journals may require that these standards are followed for the review type that is being employed (e.g. systematic review, scoping review, etc).