1.2.2  What is a systematic review?

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question.  It  uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993) . The key characteristics of a systematic review are:

a clearly stated set of objectives with pre-defined eligibility criteria for studies;

an explicit, reproducible methodology;

a systematic search that attempts to identify all studies that would meet the eligibility criteria;

an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and

a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

Many systematic reviews contain meta-analyses. Meta-analysis is the use of statistical methods to summarize the results of independent studies (Glass 1976). By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review (see Chapter 9, Section 9.1.3 ). They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies.

Jump to navigation


Cochrane Training

Chapter 1: starting a review.

Toby J Lasserson, James Thomas, Julian PT Higgins

Key Points:

Cite this chapter as: Lasserson TJ, Thomas J, Higgins JPT. Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook .

1.1 Why do a systematic review?

Systematic reviews were developed out of a need to ensure that decisions affecting people’s lives can be informed by an up-to-date and complete understanding of the relevant research evidence. With the volume of research literature growing at an ever-increasing rate, it is impossible for individual decision makers to assess this vast quantity of primary research to enable them to make the most appropriate healthcare decisions that do more good than harm. By systematically assessing this primary research, systematic reviews aim to provide an up-to-date summary of the state of research knowledge on an intervention, diagnostic test, prognostic factor or other health or healthcare topic. Systematic reviews address the main problem with ad hoc searching and selection of research, namely that of bias. Just as primary research studies use methods to avoid bias, so should summaries and syntheses of that research.

A systematic review attempts to collate all the empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman et al 1992, Oxman and Guyatt 1993). Systematic review methodology, pioneered and developed by Cochrane, sets out a highly structured, transparent and reproducible methodology (Chandler and Hopewell 2013). This involves: the a priori specification of a research question; clarity on the scope of the review and which studies are eligible for inclusion; making every effort to find all relevant research and to ensure that issues of bias in included studies are accounted for; and analysing the included studies in order to draw conclusions based on all the identified research in an impartial and objective way.

This Handbook is about systematic reviews on the effects of interventions, and specifically about methods used by Cochrane to undertake them. Cochrane Reviews use primary research to generate new knowledge about the effects of an intervention (or interventions) used in clinical, public health or policy settings. They aim to provide users with a balanced summary of the potential benefits and harms of interventions and give an indication of how certain they can be of the findings. They can also compare the effectiveness of different interventions with one another and so help users to choose the most appropriate intervention in particular situations. The primary purpose of Cochrane Reviews is therefore to inform people making decisions about health or health care.

Systematic reviews are important for other reasons. New research should be designed or commissioned only if it does not unnecessarily duplicate existing research (Chalmers et al 2014). Therefore, a systematic review should typically be undertaken before embarking on new primary research. Such a review will identify current and ongoing studies, as well as indicate where specific gaps in knowledge exist, or evidence is lacking; for example, where existing studies have not used outcomes that are important to users of research (Macleod et al 2014). A systematic review may also reveal limitations in the conduct of previous studies that might be addressed in the new study or studies.

Systematic reviews are important, often rewarding and, at times, exciting research projects. They offer the opportunity for authors to make authoritative statements about the extent of human knowledge in important areas and to identify priorities for further research. They sometimes cover issues high on the political agenda and receive attention from the media. Conducting research with these impacts is not without its challenges, however, and completing a high-quality systematic review is often demanding and time-consuming. In this chapter we introduce some of the key considerations for potential review authors who are about to start a systematic review.

1.2 What is the review question?

Getting the research question right is critical for the success of a systematic review. Review authors should ensure that the review addresses an important question to those who are expected to use and act upon its conclusions.

We discuss the formulation of questions in detail in Chapter 2 . For a question about the effects of an intervention, the PICO approach is usually used, which is an acronym for Population, Intervention, Comparison(s) and Outcome. Reviews may have additional questions, for example about how interventions were implemented, economic issues, equity issues or patient experience.

To ensure that the review addresses a relevant question in a way that benefits users, it is important to ensure wide input. In most cases, question formulation should therefore be informed by people with various relevant – but potentially different – perspectives (see Chapter 2, Section 2.4 ).

1.3 Who should do a systematic review?

Systematic reviews should be undertaken by a team. Indeed, Cochrane will not publish a review that is proposed to be undertaken by a single person. Working as a team not only spreads the effort, but ensures that tasks such as the selection of studies for eligibility, data extraction and rating the certainty of the evidence will be performed by at least two people independently, minimizing the likelihood of errors. First-time review authors are encouraged to work with others who are experienced in the process of systematic reviews and to attend relevant training.

Review teams must include expertise in the topic area under review. Topic expertise should not be overly narrow, to ensure that all relevant perspectives are considered. Perspectives from different disciplines can help to avoid assumptions or terminology stemming from an over-reliance on a single discipline. Review teams should also include expertise in systematic review methodology, including statistical expertise.

Arguments have been made that methodological expertise is sufficient to perform a review, and that content expertise should be avoided because of the risk of preconceptions about the effects of interventions (Gøtzsche and Ioannidis 2012). However, it is important that both topic and methodological expertise is present to ensure a good mix of skills, knowledge and objectivity, because topic expertise provides important insight into the implementation of the intervention(s), the nature of the condition being treated or prevented, the relationships between outcomes measured, and other factors that may have an impact on decision making.

A Cochrane Review should represent an independent assessment of the evidence and avoiding financial and non-financial conflicts of interest often requires careful management. It will be important to consider if there are any relevant interests that may constitute a conflict of interest. There are situations where employment, holding of patents and other financial support should prevent people joining an author team. Funding of Cochrane Reviews by commercial organizations with an interest in the outcome of the review is not permitted. To ensure that any issues are identified early in the process, authors planning Cochrane Reviews should consult the Conflict of Interest Policy . Authors should make complete declarations of interest before registration of the review, and refresh these annually thereafter until publication and just prior to publication of the protocol and the review. For authors of review updates, this must be done at the time of the decision to update the review, annually thereafter until publication, and just prior to publication. Authors should also update declarations of interest at any point when their circumstances change.

1.3.1 Involving consumers and other stakeholders

Because the priorities of decision makers and consumers may be different from those of researchers, it is important that review authors consider carefully what questions are important to these different stakeholders. Systematic reviews are more likely to be relevant to a broad range of end users if they are informed by the involvement of people with a range of experiences, in terms of both the topic and the methodology (Thomas et al 2004, Rees and Oliver 2017). Engaging consumers and other stakeholders, such as policy makers, research funders and healthcare professionals, increases relevance, promotes mutual learning, improved uptake and decreases research waste.

Mapping out all potential stakeholders specific to the review question is a helpful first step to considering who might be invited to be involved in a review. Stakeholders typically include: patients and consumers; consumer advocates; policy makers and other public officials; guideline developers; professional organizations; researchers; funders of health services and research; healthcare practitioners, and, on occasion, journalists and other media professionals. Balancing seniority, credibility within the given field, and diversity should be considered. Review authors should also take account of the needs of resource-poor countries and regions in the review process (see Chapter 16 ) and invite appropriate input on the scope of the review and the questions it will address.

It is established good practice to ensure that consumers are involved and engaged in health research, including systematic reviews. Cochrane uses the term ‘consumers’ to refer to a wide range of people, including patients or people with personal experience of a healthcare condition, carers and family members, representatives of patients and carers, service users and members of the public. In 2017, a Statement of Principles for consumer involvement in Cochrane was agreed. This seeks to change the culture of research practice to one where both consumers and other stakeholders are joint partners in research from planning, conduct, and reporting to dissemination. Systematic reviews that have had consumer involvement should be more directly applicable to decision makers than those that have not (see online Chapter II ).

1.3.2 Working with consumers and other stakeholders

Methods for working with consumers and other stakeholders include surveys, workshops, focus groups and involvement in advisory groups. Decisions about what methods to use will typically be based on resource availability, but review teams should be aware of the merits and limitations of such methods. Authors will need to decide who to involve and how to provide adequate support for their involvement. This can include financial reimbursement, the provision of training, and stating clearly expectations of involvement, possibly in the form of terms of reference.

While a small number of consumers or other stakeholders may be part of the review team and become co-authors of the subsequent review, it is sometimes important to bring in a wider range of perspectives and to recognize that not everyone has the capacity or interest in becoming an author. Advisory groups offer a convenient approach to involving consumers and other relevant stakeholders, especially for topics in which opinions differ. Important points to ensure successful involvement include the following.

Templates such as terms of reference, job descriptions, or person specifications for an advisory group help to ensure clarity about the task(s) required and are available from INVOLVE . The website also gives further information on setting and organizing advisory groups. See also the Cochrane training website for further resources to support consumer involvement.

1.4 The importance of reliability

Systematic reviews aim to be an accurate representation of the current state of knowledge about a given issue. As understanding improves, the review can be updated. Nevertheless, it is important that the review itself is accurate at the time of publication. There are two main reasons for this imperative for accuracy. First, health decisions that affect people’s lives are increasingly taken based on systematic review findings. Current knowledge may be imperfect, but decisions will be better informed when taken in the light of the best of current knowledge. Second, systematic reviews form a critical component of legal and regulatory frameworks; for example, drug licensing or insurance coverage. Here, systematic reviews also need to hold up as auditable processes for legal examination. As systematic reviews need to be both correct, and be seen to be correct, detailed evidence-based methods have been developed to guide review authors as to the most appropriate procedures to follow, and what information to include in their reports to aid auditability.

1.4.1 Expectations for the conduct and reporting of Cochrane Reviews

Cochrane has developed methodological expectations for the conduct, reporting and updating of systematic reviews of interventions (MECIR) and their plain language summaries ( Plain Language Expectations for Authors of Cochrane Summaries ; PLEACS). Developed collaboratively by methodologists and Cochrane editors, they are intended to describe the desirable attributes of a Cochrane Review. The expectations are not all relevant at the same stage of review conduct, so care should be taken to identify those that are relevant at specific points during the review. Different methods should be used at different stages of the review in terms of the planning, conduct, reporting and updating of the review.

Each expectation has a title, a rationale and an elaboration. For the purposes of publication of a review with Cochrane, each has the status of either ‘mandatory’ or ‘highly desirable’. Items described as mandatory are expected to be applied, and if they are not then an appropriate justification should be provided; failure to implement such items may be used as a basis for deciding not to publish a review in the Cochrane Database of Systematic Reviews (CDSR). Items described as highly desirable should generally be implemented, but there are reasonable exceptions and justifications are not required.

All MECIR expectations for the conduct of a review are presented in the relevant chapters of this Handbook . Expectations for reporting of completed reviews (including PLEACS) are described in online Chapter III . The recommendations provided in the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement have been incorporated into the Cochrane reporting expectations, ensuring compliance with the PRISMA recommendations and summarizing attributes of reporting that should allow a full assessment of the methods and findings of the review (Moher et al 2009).

1.5 Protocol development

Preparing a systematic review is complex and involves many judgements. To minimize the potential for bias in the review process, these judgements should be made as far as possible in ways that do not depend on the findings of the studies included in the review. Review authors’ prior knowledge of the evidence may, for example, influence the definition of a systematic review question, the choice of criteria for study eligibility, or the pre-specification of intervention comparisons and outcomes to analyse. It is important that the methods to be used should be established and documented in advance (see MECIR Box 1.5.a , MECIR Box 1.5.b and MECIR Box 1.5.c ).

Publication of a protocol for a review that is written without knowledge of the available studies reduces the impact of review authors’ biases, promotes transparency of methods and processes, reduces the potential for duplication, allows peer review of the planned methods before they have been completed, and offers an opportunity for the review team to plan resources and logistics for undertaking the review itself. All chapters in the Handbook should be consulted when drafting the protocol. Since systematic reviews are by their nature retrospective, an element of knowledge of the evidence is often inevitable. This is one reason why non-content experts such as methodologists should be part of the review team (see Section 1.3 ). Two exceptions to the retrospective nature of a systematic review are a meta-analysis of a prospectively planned series of trials and some living systematic reviews, as described in Chapter 22 .

The review question should determine the methods used in the review, and not vice versa. The question may concern a relatively straightforward comparison of one treatment with another; or it may necessitate plans to compare different treatments as part of a network meta-analysis, or assess differential effects of an intervention in different populations or delivered in different ways.

The protocol sets out the context in which the review is being conducted. It presents an opportunity to develop ideas that are foundational for the review. This concerns, most explicitly, definition of the eligibility criteria such as the study participants and the choice of comparators and outcomes. The eligibility criteria may also be defined following the development of a logic model (or an articulation of the aspects of an extent logic model that the review is addressing) to explain how the intervention might work (see Chapter 2, Section 2.5.1 ).

MECIR Box 1.5.a Relevant expectations for conduct of intervention reviews

A key purpose of the protocol is to make plans to minimize bias in the eventual findings of the review. Reliable synthesis of available evidence requires a planned, systematic approach. Threats to the validity of systematic reviews can come from the studies they include or the process by which reviews are conducted. Biases within the studies can arise from the method by which participants are allocated to the intervention groups, awareness of intervention group assignment, and the collection, analysis and reporting of data. Methods for examining these issues should be specified in the protocol. Review processes can generate bias through a failure to identify an unbiased (and preferably complete) set of studies, and poor quality assurance throughout the review. The availability of research may be influenced by the nature of the results (i.e. reporting bias). To reduce the impact of this form of bias, searching may need to include unpublished sources of evidence (Dwan et al 2013) ( MECIR Box 1.5.b ).

MECIR Box 1.5.b Relevant expectations for the conduct of intervention reviews

Developing a protocol for a systematic review has benefits beyond reducing bias. Investing effort in designing a systematic review will make the process more manageable and help to inform key priorities for the review. Defining the question, referring to it throughout, and using appropriate methods to address the question focuses the analysis and reporting, ensuring the review is most likely to inform treatment decisions for funders, policy makers, healthcare professionals and consumers. Details of the planned analyses, including investigations of variability across studies, should be specified in the protocol, along with methods for interpreting the results through the systematic consideration of factors that affect confidence in estimates of intervention effect ( MECIR Box 1.5.c ).

MECIR Box 1.5.c Relevant expectations for conduct of intervention reviews

While the intention should be that a review will adhere to the published protocol, changes in a review protocol are sometimes necessary. This is also the case for a protocol for a randomized trial, which must sometimes be changed to adapt to unanticipated circumstances such as problems with participant recruitment, data collection or event rates. While every effort should be made to adhere to a predetermined protocol, this is not always possible or appropriate. It is important, however, that changes in the protocol should not be made based on how they affect the outcome of the research study, whether it is a randomized trial or a systematic review. Post hoc decisions made when the impact on the results of the research is known, such as excluding selected studies from a systematic review, or changing the statistical analysis, are highly susceptible to bias and should therefore be avoided unless there are reasonable grounds for doing this.

Enabling access to a protocol through publication (all Cochrane Protocols are published in the CDSR ) and registration on the PROSPERO register of systematic reviews reduces duplication of effort, research waste, and promotes accountability. Changes to the methods outlined in the protocol should be transparently declared.

This Handbook provides details of the systematic review methods developed or selected by Cochrane. They are intended to address the need for rigour, comprehensiveness and transparency in preparing a Cochrane systematic review. All relevant chapters – including those describing procedures to be followed in the later stages of the review – should be consulted during the preparation of the protocol. A more specific description of the structure of Cochrane Protocols is provide in online Chapter II .

1.6 Data management and quality assurance

Systematic reviews should be replicable, and retaining a record of the inclusion decisions, data collection, transformations or adjustment of data will help to establish a secure and retrievable audit trail. They can be operationally complex projects, often involving large research teams operating in different sites across the world. Good data management processes are essential to ensure that data are not inadvertently lost, facilitating the identification and correction of errors and supporting future efforts to update and maintain the review. Transparent reporting of review decisions enables readers to assess the reliability of the review for themselves.

Review management software, such as Covidence and EPPI-Reviewer , can be used to assist data management and maintain consistent and standardized records of decisions made throughout the review. These tools offer a central repository for review data that can be accessed remotely throughout the world by members of the review team. They record independent assessment of studies for inclusion, risk of bias and extraction of data, enabling checks to be made later in the process if needed. Research has shown that even experienced reviewers make mistakes and disagree with one another on risk-of-bias assessments, so it is particularly important to maintain quality assurance here, despite its cost in terms of author time. As more sophisticated information technology tools begin to be deployed in reviews (see Chapter 4, Section and Chapter 22, Section 22.2.4 ), it is increasingly apparent that all review data – including the initial decisions about study eligibility – have value beyond the scope of the individual review. For example, review updates can be made more efficient through (semi-) automation when data from the original review are available for machine learning.

1.7 Chapter information

Authors: Toby J Lasserson, James Thomas, Julian PT Higgins

Acknowledgements: This chapter builds on earlier versions of the Handbook . We would like to thank Ruth Foxlee, Richard Morley, Soumyadeep Bhaumik, Mona Nasser, Dan Fox and Sally Crowe for their contributions to Section 1.3 .

Funding: JT is supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. JPTH is a member of the NIHR Biomedical Research Centre at University Hospitals Bristol NHS Foundation Trust and the University of Bristol. JPTH received funding from National Institute for Health Research Senior Investigator award NF-SI-0617-10145. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

1.8 References

Antman E, Lau J, Kupelnick B, Mosteller F, Chalmers T. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: treatment for myocardial infarction. JAMA 1992; 268 : 240–248.

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, Howells DW, Ioannidis JP, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet 2014; 383 : 156–165.

Chandler J, Hopewell S. Cochrane methods – twenty years experience in developing systematic review methods. Systematic Reviews 2013; 2 : 76.

Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias: an updated review. PloS One 2013; 8 : e66844.

Gøtzsche PC, Ioannidis JPA. Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses? BMJ 2012; 345 .

Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, Al-Shahi Salman R, Chan AW, Glasziou P. Biomedical research: increasing value, reducing waste. Lancet 2014; 383 : 101–104.

Moher D, Liberati A, Tetzlaff J, Altman D, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine 2009; 6 : e1000097.

Oxman A, Guyatt G. The science of reviewing research. Annals of the New York Academy of Sciences 1993; 703 : 125–133.

Rees R, Oliver S. Stakeholder perspectives and participation in reviews. In: Gough D, Oliver S, Thomas J, editors. An Introduction to Systematic Reviews . 2nd ed. London: Sage; 2017. p. 17–34.

Steel R. Involving marginalised and vulnerable people in research: a consultation document (2nd revision). INVOLVE; 2004.

Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton G, Kavanagh J. Integrating qualitative research with trials in systematic reviews. BMJ 2004; 328 : 1010–1012.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Save citation to file

Email citation, add to collections.

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

What is a Cochrane review?


Systematic reviews carried out by Cochrane Collaboration (an international network of researchers belonging to this independent, not-for-profit organization) are recognized worldwide as the highest standard in evidence-based healthcare. The main reason is that Cochrane reviews follow a common and specific methodology to limit bias and random error. In this issue, we highlight the most important methodological features of Cochrane reviews, also reporting details on the editorial process to publish the review in the Cochrane Library.

Similar articles

Publication types

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on December 7, 2022.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

See an example

definition systematic review cochrane

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

A systematic review has many pros .

Systematic reviews also have a few cons .

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

You can rearrange these four components to write your research question:

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

Their research question was:

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2022, December 07). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved March 17, 2023, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.

Cochrane Consumer Network

Cochrane and systematic reviews, about the cochrane library, systematic reviews.


  Top of page       

The Cochrane Library is an electronic collection of databases published on the internet and also available on CD-Rom. It is updated quarterly in an effort to add to and keep the information current. The Library is made up of a number of parts.

The Cochrane Database of Systematic Reviews (CDSR) contains the published Cochrane reviews and protocols.

The Cochrane Central Register of Controlled Trials (CENTRAL) collates references to controlled trials in health care. These healthcare trial references are entered by Cochrane groups. The main way of finding health care studies is by looking in electronic databases (such as MEDLINE, EMBASE, CINAHL) using special search terms. Other ways are by asking experts in a particular health field and through hand searching journals.

The Database of Abstracts of Reviews of Effects (DARE) is a collection of structured abstracts and bibliographic references of systematic reviews of the effects of health care. It is developed by the Centre for Research and Dissemination, University of York, UK.

Methodological reviews and articles are also presented in The Cochrane Library.

In addition, each Cochrane group (termed an entity) has a section (module) in the Library that gives information on the group’s organisation, contact details, function, reviews, and other general information.

Abstracts of reviews are readily accessible at www.cochrane.org/reviews . In countries such as Australia, Denmark, Finland, Ireland, Latin America, Norway and UK the full reviews are freely available as the governments of these countries have subscriptions to The Cochrane Library . Consumers who live in other countries and who wish to read a full review may need to access The Cochrane Library through a university, hospital or large public library.

A Cochrane Library Users Guide is available ( https://www.cochrane.org.au/libraryguide/ ) to help you find the information you want from The Cochrane Library.

Brief summaries (plain language summaries) of Cochrane reviews are written for consumers and others to highlight the information in a review. A What’s New Digest summarises the newest reviews.

If you would like to make comments on any existing review in The Cochrane Library, you will find a special section for 'Comments and Criticisms' with the review.

If someone decides to look critically at articles that have appeared in the medical or health literature on a particular topic they are said to be ‘reviewing the literature’. The authors may review, say, all the drug treatments available for one type of heart disease. A review is very clearly defined and sets out to find what evidence there is for prescribing one particular intervention or drug in a specific health condition, often in a certain group of people.

Examples of review topics are: Single dose celecoxib for acute postoperative pain; Artichoke leaf extract for treating hypercholesterolaemia; Chocolate avoidance for preventing migraine; Etidronate for treating and preventing postmenopausal osteoporosis.

A systematic review summarises the results of available carefully designed healthcare studies (controlled trials) and provides a high level of evidence on the effectiveness of healthcare interventions.

The review authors set about their task very methodically following, step by step, an advance plan that covers:

A protocol is the plan or set of steps to be followed in preparing a review. A protocol for a systematic review clearly describes why the review is needed (the review question), what the review is about (the healthcare context of the review), and how the reviewer authors will go about developing the review. It details how they will seek, select as relevant, critically appraise studies, and collect and analyse data (combine data and check for significance to the healthcare situation) from the included studies.

Cochrane protocols are published in the Cochrane Database of Systematic Reviews so that people can comment on them before the actual review has been carried out.

How do I know a healthcare intervention works?

The aim of a systematic review is to thoroughly assess, by means of a set procedure, the best possible evidence about the effects of a healthcare intervention or treatment in a particular healthcare situation.

Healthcare studies are generally designed to assess the benefits, rather than the harms, of an intervention. Studies generally have a relatively short designated time period. Any possible harms of an intervention may be expected to occur less frequently and over a longer period of time than the studies cover.

The process of a review is clearly defined, before starting the actual review of the literature, to minimise associations of expectations of effects and other sources of bias. Bias is a systematic ‘error’ or mistake in the judgments and decisions made that influence the results of a study or a review. Bias differs from a ‘placebo effect’, which is where participants of a study (or assessors of the outcomes) perceive a beneficial effect, or harm, with an inactive treatment.

The specific methods used in a review are carefully set out by The Cochrane Collaboration and are described in each review.

A Cochrane review is prepared and maintained using specific methodologies described in the Cochrane Handbook .

Systematic reviews of randomised controlled trials provide the clearest evidence for the benefits of a healthcare intervention.

This is because the best way to assess the effects of a health care treatment is to use procedures that reduce the influence of chance effects and associations of cause and effect. Individual expectations on the part of a service provider, assessor and the person receiving an intervention can all contribute to modifying observed findings from a healthcare study. Randomised controlled trials where none of these people know the exact intervention a study participant is receiving (intervention under investigation, a placebo, or a comparator) may be expected to provide the best evidence.

By assessing the health of the two comparative groups in a study after their treatments, we can tell which intervention is more successful – but only if the two groups of people were very similar before treatment began. Otherwise we might be misled. For instance, one group may become healthier not because their treatment was better but because they were younger, not so ill, at less risk of ill health before treatment began, or even self selected to a particular intervention because of a particular personality trait, for example, people who chose to take a hormone may have wanted to stay younger and be more active.

Randomised controlled trials are studies that are rigorously designed. People are allocated to intervention groups in a way that minimises the chances of predicting which treatment group a study participant is in. The intervention under investigation is compared against a well-known intervention or an inactive treatment (placebo). Studies are controlled so that participants have similar associated care in all ways other than the intervention. Ideally, depending on the type of intervention, the service provider is unaware of which group a participant is in and those assessing outcomes are also unaware – this is termed ’ blinding ’.

The strength of evidence for a particular intervention can be increased further by systematically looking at (reviewing) all available randomized controlled trials that have been reported relevant to a particular healthcare situation.

Many people are needed to properly test an intervention. This is more than can be recruited into a single trial; it is also important to investigate the intervention in different populations. Furthermore, the technical aspects of a particular randomised controlled trial may nothave been implemented properly, for one reason or another. The effects of these shortcomings can be minimised by grouping results of a number of studies.

The results of randomised controlled trials may be published in any one of thousands of journals world wide. Indeed some studies are not published at all. In reality the studies found most easily tend to have over-optimistic results and finding reliable information about the effects of care is particularly difficult when there are negative results (the intervention is no better than placebo or another treatment). Sometimes published trials are too small to provide a conclusive result in their own right - as to whether a treatment really does work. Consequently, to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months, but it will be much less work than conducting a new randomised controlled trial. This process also will not unnecessarily exclude people from effective interventions because of allocation to a placebo (or inactive treatment) group.Discussions are underway in The Cochrane Collaboration as to how qualitative studies can be used to add to the information obtained from controlled studies - those that consider outcomes measured in numerical terms (and so are termed quantitative studies). Qualitative measures include ‘quality of life’ and lifestyle changes obtained from detailed questionnaires. Qualitative studies may also use narrative interviews where participants are asked to talk about their experiences around sets of semi-structured questions and prompts to explore particular issues that information is needed on for a study.

Top of page

What consumers can, and cannot, get from systematic reviews

Systematic reviews ask a very specific research question about a particular intervention in a clearly defined group of people who have a clear health condition or problem. Reviews provide powerful information on the state of knowledge about a healthcare intervention and whether that intervention is an effective treatment of a healthcare condition.Reviews:

Healthcare studies differ dramatically in what they look for and how well they are carried out and, therefore, how much weight one canput on their conclusions. Part of the reason for performing systematic reviews is to reduce the effects of these shortcomings. Issues of conflict of interest and corporate funding of healthcare studies are also important considerations in drawing conclusions from any study.

Well-designed healthcare studies generally set out to determine the efficacy of a healthcare intervention. Information on potential harms is less well investigated.

Carefully controlled studies take place over a limited period of time so that the researchers can account for all people who entered the study from beginning to the end of the study. Harms are generally less common than benefits and may be apparent over a different time period. This may be, for example, only in the long term so that the intervention would have to be given to more people for a long time period for adverse effects to be studied effectively.

Participants of studies are selected to reduce the risk of other problems interfering with the efficacy of an intervention. How selective thisprocess is needs to be carefully considered when assessing the relevance of a study to an individual.

Randomised controlled trials are expensive to run. They are very time consuming and multiple factors may limit how many participants are involved, the outcomes measured and the length of the trial. How many people complete the study is also very important.

Levels of evidence for healthcare interventions

The National Health and Medical Research Council of Australia (1999) defines the ‘dimensions of evidence’ using three main areas.

Level of evidence: the study design used – a systematic review of all relevant randomised controlled trials is the highest level, followed by at least one randomised controlled trial, then a pseudo-randomised trial Quality of evidence: the methods used to minimise bias within a study design Statistical precision: the degree of certainty about the existence of a true effect

2. Size of effect

How much the determined intervention effect is above a ‘no apparent effect’ value for clinically relevant effects

3. Relevance of the evidence

How appropriate the outcome measure is for the healthcare problem, and its usefulness in measuring effectiveness of treatment

Adapted from AD.Oxman Checklists for review articles. BMJ 1994;309:648-51

Level I. For a randomised controlled trial, the lower limit of the confidence interval (expressed as a range) for a measure of effect is still above a meaningful benefit in healthcare terms

Level II. For a randomised controlled trial, the lower limit of the confidence interval (expressed as a range) for a measure of effect is less than a meaningful beneficial effect in healthcare terms; but the point estimate of effect still shows effectiveness of the intervention

Level III. Measures of effectiveness are taken from non-randomised studies of groups of people where a control group has run concurrently with the group receiving the intervention being assessed

Level IV. Measures of effectiveness are taken from non-randomised studies of groups of people where intervention effects are compared with previous or historical information

Level V. Evidence is from single case studies

Even studies perfectly designed and carried out may show variable results because of the play of chance. CI covers the likely range of the true effect. For example, the result of a study may be that 40 per cent (95% CI 30% to 50%) of people are helped by a treatment. That means that we can be 95 per cent certain the true effect is between 30 and 50 per cent. ( Smart Health Choices How to make informed health decisions by Judy Irwig, Les Irwig and Melissa Sweet, Allen and Unwin 1999)

Top of page       

Cochrane Groups

Different groups exist for different health conditions: International Cochrane review groups cover important areas of health care diseases and conditions. Review groups are responsible for producing and maintaining Cochrane reviews on specific health care questions. You will see in The Cochrane Library, for example, a Cochrane Consumers and Communication Group, Cochrane Epilepsy Group, Cochrane Heart Group and a Cochrane Pregnancy and Childbirth Group.

The activities of each group (or entity in Cochrane language) are monitored and co-ordinated by one person for each group, known as the managing editor (review group co-ordinator). This person manages the day to day running of the group and is usually the contact person. The co-ordinating editor leads the group and is responsible for the quality and subject of reviews.

Each group attracts members with a variety of backgrounds, experience and expertise, who contribute to the process of developing systematic reviews. They may be doctors, nurses, researchers, health advisers, consumers and caregivers.

Fields cover health care in a broader sense than do review groups. These may include a major section of health care such as cancer, the setting of care (e.g. primary care), the type of patient/consumer (e.g. older persons), the type of provider (e.g. nurses), or the type of intervention (e.g. vaccines). The role of fields is to facilitate the work of collaborative review groups and to ensure that Cochrane reviews appropriate to an area of interest are both relevant and accessible to service providers and consumers.

Each field works to:

The principal contact person in a field is its field co-ordinator.

Cochrane centres provide a range of services designed to support collaborative review groups in their area and to facilitate the review process. They serve as a regional source of information about The Cochrane Collaboration, provide support to Cochrane contributors within a defined geographical area and promote access to The Cochrane Library. Each centre has a director.

The Consumer Network supports consumer participation within The Cochrane Collaboration, internationally. The Network is available to any active consumer. Its mission is to enable and support consumers in contributing to the function of collaborative reviews groups and other Cochrane entities. The Network enables communication with other consumers, provides a sense of belonging within The Cochrane Collaboration, links and dissemination of information from Cochrane reviews.

© The Cochrane Collaboration Comments about this page to: [email protected]

Jump to navigation


VIDEO: What are systematic reviews?

What are systematic reviews?

A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question. Researchers conducting systematic reviews use explicit, systematic methods that are selected with a view aimed at minimizing bias, to produce more reliable findings to inform decision making. 

Here is a video from Cochrane Consumers and Communication that explains what a systematic review is clearly and simply for people who may not be familiar with the concepts and terminology of systematic reviews: what they are, how researchers prepare them, and why they’re an important part of making informed decisions about health - for everyone. 

Cochrane evidence provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence, including systematic reviews, and other activities of Cochrane. 



You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

definition systematic review cochrane


Article Menu

definition systematic review cochrane

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A systematic review and meta-analysis of supramarginal resection versus gross total resection in glioblastoma: can we enhance progression-free survival time and preserve postoperative safety.

definition systematic review cochrane

Simple Summary

1. introduction, 2. materials and methods, 2.1. inclusion and exclusion criteria, 2.2. data extraction and clinical endpoints, 2.3. statistics, 3.1. literature search, 3.2. characteristics of included studies regarding the analysis of progression-free survival, 3.3. impact of supramarginal resection on progression-free survival in glioblastoma, 3.4. characteristics of included studies regarding the analysis of perioperative complications, 3.5. impact of supramarginal resection on postoperative complications, 3.5.1. postoperative meningitis, 3.5.2. postoperative intracranial hemorrhage, 3.5.3. postoperative csf leaks, 3.6. publication bias, 4. discussion, 4.1. supramarginal resection and postoperative complications, 4.2. supramarginal resection and probability of progression-free survival, 4.3. limitations, 5. conclusions, supplementary materials, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

Share and Cite

Wach, J.; Vychopen, M.; Kühnapfel, A.; Seidel, C.; Güresir, E. A Systematic Review and Meta-Analysis of Supramarginal Resection versus Gross Total Resection in Glioblastoma: Can We Enhance Progression-Free Survival Time and Preserve Postoperative Safety? Cancers 2023 , 15 , 1772. https://doi.org/10.3390/cancers15061772

Wach J, Vychopen M, Kühnapfel A, Seidel C, Güresir E. A Systematic Review and Meta-Analysis of Supramarginal Resection versus Gross Total Resection in Glioblastoma: Can We Enhance Progression-Free Survival Time and Preserve Postoperative Safety? Cancers . 2023; 15(6):1772. https://doi.org/10.3390/cancers15061772

Wach, Johannes, Martin Vychopen, Andreas Kühnapfel, Clemens Seidel, and Erdem Güresir. 2023. "A Systematic Review and Meta-Analysis of Supramarginal Resection versus Gross Total Resection in Glioblastoma: Can We Enhance Progression-Free Survival Time and Preserve Postoperative Safety?" Cancers 15, no. 6: 1772. https://doi.org/10.3390/cancers15061772

Article Metrics

Article access statistics, supplementary material.

ZIP-Document (ZIP, 224 KiB)

Further Information

Mdpi initiatives, follow mdpi.


Subscribe to receive issue release notifications and newsletters from MDPI journals


Cochrane Community

Living systematic reviews.

Visit Project Transform

What is a living systematic review?

We define an LSR as a systematic review which is continually updated, incorporating relevant new evidence as it becomes available . 

Practically, this means that LSRs:

While core review methods are not fundamentally different to other Cochrane Reviews, an LSR should additionally include explicit, transparent and predefined decisions on:

Why living systematic reviews?

Living systematic reviews (LSRs) provide a new approach to support the ongoing efforts of Cochrane and others to produce evidence that is both trustworthy and current. 

The concept of living evidence synthesis and related outputs, such as living guidelines, are of increasing interest to evidence producers, decision makers, guideline developers, funders and publishers, as a way to seamlessly connect evidence and practice.

The possibility of a scaled-up living evidence approach has only recently been within reach, due to a number of technological and data-related innovations, such as online platforms, linked data, and machine learning. Concurrently, research groups are embracing larger collaborations, open and shared data, and the growth of the citizen science movement, opening up the possibility of communities with a common interest maintaining high value datasets and associated LSR. 

Cochrane LSR planning and methods support

Cochrane authors and review groups planning to undertake a living systematic review are encouraged to contact the Cochrane LSR team at [email protected]  for support with developing their protocol. We're here to help you work through the factors to consider when planning and conducting an LSR.

Living systematic review pilots

Project Transform evaluated LSRs as part of the Production Models component with the following groups:

An evaluation was undertaken alongside the pilots to understand the feasibility of LSRs, implications for the people and processes involved and identify opportunities to refine the LSR model before scaling up. The final evaluation report is available to download here .

Following completion of the pilot, additional Cochrane teams are conducting LSRs, with the aim of at least one published LSR for each Network.

LSR guidance for production and publication

The purpose of this document is to outline the methods related to production and publication for LSRs published on The Cochrane Library. This document is primarily designed to be a practical guidance document for the authors, Review Groups and Central Editorial Unit staff involved in Cochrane LSRs. The approach described is the revised guidance based on our review of current literature relevant to LSRs and consultation with a range of stakeholders, including the LSR Network, which includes members within Cochrane and beyond.

LSR Webinars

Other LSR Resources

Living Evidence Network

The Living Evidence Network is an informal network that was launched in February 2016, with members including Cochrane and non-Cochrane researchers, policymakers and guideline developers.

The Living Evidence Network aims to: 

You can download the Living Evidence Network governance structure document here . If you would like to join the Living Evidence Network, please email  [email protected]

LSR icon

Cochrane UK

What are systematic reviews, "what are systematic reviews".

If you’re a Cochrane contributor and have ever attempted to explain Cochrane’s work to someone, chances are you’ve tried to answer this question. And if you’re reading this because you’re new to Cochrane and the work we do, you may be wondering about this too.

Thanks to a team of creative colleagues from  Cochrane Consumers and Communication , we’re pleased to share a video resource which answers this question clearly and simply for people who may not be familiar with the concept of systematic reviews: what they are, how researchers prepare them, and why they’re an important part of making informed decisions about health - for everyone. You can find this video on  Cochrane’s YouTube channel , and we hope you’ll share and spread the word about the importance of evidence!

Ask an Expert

Ask an expert about access to resources, publishing, grants, and more.

MD Anderson faculty and staff can also request a one-on-one consultation with a librarian or scientific editor.

Log in to the Library's remote access system using your MyID account.

The University of Texas MD Anderson Cancer Center Home

Cochrane Systematic Reviews

Systematic reviews: cochrane systematic reviews.

Data Extraction

About Cochrane

Proposing a Cochrane Systematic Review

Proposing a New Cochrane Review : you must submit a protocol to Cochrane for peer review and acceptance.

Cochrane Training

Cochrane Training : recorded and live webinars, events, guides, and handbooks. Many of the introductory resources are free while many of the advanced ones are restricted to Cochrane members. 

Finding Cochrane Systematic Reviews

Cochrane guidelines, software for cochrane systematic reviews.

Cochrane's preferred core software is: 

Study Selection

The Cochrane Handbook allows also allows the use of multiple applications with machine learning or text-mining to semi-automate study selection. After you screen a certain number of citations, the software prioritizes the rest of the citations in order of estimated relevance. You must still review each record but it helps you identify studies that are most likely to be relevant earlier in the screening process: 

The  Cochrane Handbook allows the use of many applications that collect, store, and share data:

More From Forbes

Cochrane says review does not show that ‘face masks don’t work’ against covid-19.

The Cochrane review did not include studies that have shown that face masks can reduce transmission ... [+] of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Pictured here Olivia Palermo wears a face mask while walking in New York City. (Photo by Jose Perez/Bauer-Griffin/GC Images)

You know that recently published Cochrane review that some have claimed as evidence that face masks don’t work? Well, have some of these claims been a bit, shall we say, too cocky and not Cochrane -y enough? Well, in a statement issued on March 10 by the Cochrane Library , Karla Soares-Weiser, MD, PhD, MSc, Editor-in-Chief of the Cochrane Library , emphasized, “Many commentators have claimed that a recently-updated Cochrane Review shows that ‘masks don't work,’ which is an inaccurate and misleading interpretation.” Yikes.

Soares-Weiser didn’t specify in the statement who exactly those “commentators” were. But she could have been referring to Brett Stephens, who wrote a February 21, 2023, Opinion piece for The New York Times entitled, “ The Mask Mandates Did Nothing. Will Any Lessons Be Learned .” Or maybe she was referring to Tom Jefferson, the Cochrane review’s first author, a Senior Associate Tutor at the Department for Continuing Education at the University of Oxford, and notably an Author for The Brownstone Institute . Stephens quoted Jefferson as saying the following about face masks: “There is just no evidence that they make any difference. Full stop.” That’s presumably based on the systematic review of face mask studies that Jefferson and his team conducted and published in the Cochrane Library on January 30. Well, looks like such statements didn’t really accurately represent what the systematic review found, full stop. And now the Cochrane Library is essentially saying, full stop it with all such misleading interpretations.

In fact, Soares-Weiser’s statement went on to delineate what the systematic review can and can’t do and what claims ultimately may be doo-do0: “It would be accurate to say that the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive. Given the limitations in the primary evidence, the review is not able to address the question of whether mask-wearing itself reduces people's risk of contracting or spreading respiratory viruses.”

A number of scientists and public health experts have already emphasized various key flaws in the systematic review conducted by Jefferson and his team. For example, Lucky Tran, PhD, an organizer for the March for Science and a science communicator at Columbia University, wrote in an Opinion piece for The Guardian entitled, “ Don’t believe those who claim science proves masks don’t work ,” that “The analysis [by Jefferson and his team] is flawed because it compares apples to oranges.” The systematic review didn’t compare actual apples with actual oranges, which has already been done before and has shown that you should really peel oranges before eating them. Rather, what Tran meant is that “The paper mixes together studies that were conducted in different environments with different transmission risks. It also combines studies where masks were worn part of the time with studies where masks are worn all the time. And it blends studies that looked at Covid-19 with studies that looked at influenza.”

Lucky Tran, PhD, (not pictured here) wrote in an Opinion piece for The Guardian entitled, “Don’t ... [+] believe those who claim science proves masks don’t work,” that “The analysis [by Jefferson and his team] is flawed because it compares apples to oranges.” (Photo: Getty)

Best Travel Insurance Companies

Best covid-19 travel insurance plans.

That’s a whole lot of mixing going on and makes you wonder why the authors of the review didn’t instead clearly separate out the studies. After all, no one wants apple and orange pie and as Tran emphasized in his Opinion piece, “If apples work and oranges don’t, but your analysis mixes them together, you may come to the false conclusion that apples don’t work. Out of the 78 papers analyzed in the review, only two actually studied masking during the Covid-19 pandemic. And both of those found that masks did protect wearers from Covid-19.”

Yeah, mixing Covid-19 studies and flu studies would be sort of like mixing orange spray tan with an Apple computer or a Tim Apple computer. Things could get very messy. Remember Covid-19 ain’t the flu—it just ain’t. While both are respiratory viruses, the flu virus can’t float around in the air nearly as well as the Covid-19 coronavirus, otherwise known as the severe acute respiratory syndrome coronavirus 2 (SARS-CoVC-2), can. Early on in 2020, authorities thought that the SARS-CoV-2 behaved more like the flu virus in the air. The thinking was that like the flu virus the SARS-CoV-2 was carried primarily by large respiratory droplets that would travel no further than six feet due to that thing called gravity. That was the rationale for the initial stay six feet (or one Gosling because Ryan Gosling is about six feet tall) apart from one another.

However, a pandemic game changer was when studies revealed that the SARS-CoV-2 could actually travel in smaller and lighter respiratory droplets that could hang in the air for much longer periods of time and could float for more than six feet. This game-changing finding that the SARS-CoV-2 can go all aerosol is what prompted the change in face mask guidance in the Spring of 2020. Therefore, including so many studies focused on the flu virus rather than the SARS-CoV-2, in the words of Tran, “drowned out” the SARS-CoV-2 “by the greater number of studies on influenza included, where the benefit of masking is harder to detect because it’s a far less contagious virus than Covid-19.”

Tran brought in another piece of fruit into the mix by calling the Cochrane review “a classic example of cherry-picking, where biased groups highlight a subset of data that support their position, while ignoring the larger pool of evidence that disagrees with them.” Tran went on to point out that “Many direct studies in labs show that high-quality masks significantly reduce the number of viral particles mask-wearers inhale and emit, but these are intentionally omitted in the arguments of anti-maskers.” I’ve also described for Forbes other scientific studies that have supported the use of face masks, studies that were omitted by Jefferson’s Cochrane review. Moreover, if face masks don’t work, why did the spread of respiratory viruses plummet during the Winters of 2020-2021 and 2021-2022 when, guess what, face masks were being used more widely? Life may be a bowl of cherries, but cherry picking only certain studies to support a particular viewpoint would be the “pits.”

Hmm, why might Jefferson and his team have “cherry picked” certain studies? Could a certain A-word have been involved? No, not that A-word. Rather, the operative A-word here may be “agenda.” Gavin Yamey, MD, MPH, MA, the Hymowitz Professor of Global Health and a Professor of Public Policy at Duke University, tweeted out that the Jefferson works with the Brownstone Institute and that “Brownstone has always been an extremely anti-mask, anti-vaxx right wing think tank”:

From Twitter

In fact, a closer look at the Cochrane Library publication shows that even the authors admitted that their review is very limited. This was emphasized by Soares-Weiser’s statement that read, “The review authors are clear on the limitations in the abstract: ‘The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions.’” Soares-Weiser’s statement continued by saying, “Adherence in this context refers to the number of people who actually wore the provided masks when encouraged to do so as part of the intervention. For example, in the most heavily-weighted trial of interventions to promote community mask wearing, 42.3% of people in the intervention arm wore masks compared to 13.3% of those in the control arm.” Less than 50% adherence is not great adherence. So when less than half of people are adhering to an intervention, is the intervention really in place?

Nevertheless, these stated limitations didn’t full stop Jefferson from saying that “full stop” quote in an interview . They haven’t stopped various political leaders, TV personalities, and anonymous social media accounts either from further amplifying Jefferson’s Cochrane publication to keep pushing the narrative that face masks somehow don’t work. Will these same people be as zealous at amplifying Soares-Weiser’s statement about the limitations of Jefferson’s Cochrane publication? Don’t hold your breath for this to happen.

One challenge throughout the Covid-19 pandemic has been people employing the political tactic of using vague language to suggest something without coming right out and saying it. This is a bit like saying, “I love your new outfit, I wish I had one like that but I think I’m too skinny for that style.” An even closer look at Jefferson’s Cochrane review reveals some questionable language. As Soares-Weiser’s statement mentioned, “The original Plain Language Summary for this review stated that ‘We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.’” Such a statement may sound a little more passive-aggressive than science-y. Indeed, Soares-Weiser’s statement admitted that “This wording was open to misinterpretation, for which we apologize.” She added, “While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset.” Soares-Weiser indicated that “We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses.”

The effectiveness of a face mask does depend on the type of mask that's being worn, how it's being ... [+] worn, and how many people around you are wearing the face mask. (Photo by Oli Scarff/Getty Images)

This is reminder that you should never use a single scientific publication to draw a conclusion. Instead, it’s important to look at what studies say as a whole. Some have claimed that publications in the Cochrane Library should always serve as “gold standards.” For example, Vinayak Prasad, MD, MPH, a Professor of Epidemiology & Biostatistics at the University of California, San Francisco, whose work prior to the pandemic has been mainly in oncology and not infectious diseases, called Jefferson’s Cochrane review a “gold standard for mask studies” in a tweet . A Cochrane review may serve as a “gold standard” to some degree when evaluating a bunch of randomized controlled trials (RCTs) that evaluate a single medication under very controlled and similar circumstances such as a cancer medication that’s being administered and monitored by doctors from a hospital or clinic.

But such claims about Cochrane reviews and RCTs are more like “fool’s gold” when it comes to other types of scientific questions. It’s important to remember that face mask use is a population-based intervention and not simply an individual-based intervention. The effectiveness of a face mask does depend not only on what kind of face mask you are wearing and how you wear it but also how many people around you are wearing face masks.

It’s time to face mask the facts. Many people have politicized the heck out of face masks to the point that they are not even acknowledging what scientific studies prior to the Cochrane review have shown. Instead, they’ve been doing a lot of cherry-picking, selecting bits and pieces of things here and there to support their narrative. So, when someone keeps pointing to just a single study to prove his or her point rather the entire body of scientific evidence, be skeptical, be very skeptical. Heck, as has been demonstrated, even a Cochrane Library publication can be used in a cockeyed manner.

Bruce Y. Lee


  1. PPT

    definition systematic review cochrane

  2. Writing a Systematic Review Following Cochrane Methods (Part II)

    definition systematic review cochrane

  3. Presentation s rs

    definition systematic review cochrane

  4. How to Find Systematic Reviews In Cochrane

    definition systematic review cochrane

  5. Definition of a systematic review

    definition systematic review cochrane

  6. Learning lessons from a Cochrane review of school based sexual abuse

    definition systematic review cochrane


  1. Unit 1: Part 3

  2. Webinar on Types of Systematic Review

  3. Causal Research

  4. Overview of Systematic reviews

  5. Part 1: Presentation

  6. 1st Annual Cochrane Lecture


  1. 1.2.2 What is a systematic review?

    a systematic search that attempts to identify all studies that would meet the eligibility criteria; an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

  2. What are systematic reviews?

    What are systematic reviews? Watch on Cochrane evidence, including our systematic reviews, provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence and what Cochrane does. About Cochrane Watch on

  3. About Cochrane Reviews

    A Cochrane Review is a systematic review of research in health care and health policy that is published in the Cochrane Database of Systematic Reviews. Types of Cochrane Review Intervention reviews assess the effectiveness/safety of a treatment, vaccine, device, preventative measure, procedure or policy.

  4. Chapter V: Overviews of Reviews

    The primary reason for conducting Cochrane Overviews is that using systematic reviews as the unit of searching, inclusion, and data analysis allows authors to address research questions that are broader in scope than those examined in individual systematic reviews (also see Section V.2.3 ).

  5. Chapter 1: Starting a review

    Cochrane Reviews use primary research to generate new knowledge about the effects of an intervention (or interventions) used in clinical, public health or policy settings. They aim to provide users with a balanced summary of the potential benefits and harms of interventions and give an indication of how certain they can be of the findings.

  6. What is a Cochrane review?

    Systematic reviews carried out by Cochrane Collaboration (an international network of researchers belonging to this independent, not-for-profit organization) are recognized worldwide as the highest standard in evidence-based healthcare. The main reason is that Cochrane reviews follow a common and specific methodology to limit bias and random error.

  7. Systematic Review

    A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review

  8. Cochrane and systematic reviews

    The Cochrane Library is an electronic collection of databases published on the internet and also available on CD-Rom. It is updated quarterly in an effort to add to and keep the information current. The Library is made up of a number of parts. The Cochrane Database of Systematic Reviews (CDSR) contains the published Cochrane reviews and protocols.

  9. Cochrane Database of Systematic Reviews

    Each Cochrane Review is a systematic review that has been prepared and supervised by a Cochrane Review Group (editorial team). It attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question.

  10. VIDEO: What are systematic reviews?

    What are systematic reviews? Watch on Cochrane evidence provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence, including systematic reviews, and other activities of Cochrane. About Cochrane Watch on

  11. PDF Cochrane Rapid Reviews

    Cochrane Rapid Review (RR) Definition Definition of a Cochrane Rapid Review (RR) We recommend the following definition: "A rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting specific methods to produce evidence for stakeholders in a r ...

  12. Cancers

    Multicentric prospective randomized trials with a standardized definition of SMR and analysis of neurologic functioning and health-related quality of life are justified and needed to improve the level of evidence. ... and the Cochrane Handbook for Systematic Reviews of Interventions version 6.3 . The study was registered in the ...

  13. Living systematic reviews

    Living systematic reviews (LSRs) provide a new approach to support the ongoing efforts of Cochrane and others to produce evidence that is both trustworthy and current. The concept of living evidence synthesis and related outputs, such as living guidelines, are of increasing interest to evidence producers, decision makers, guideline developers ...

  14. What are systematic reviews?

    What are systematic reviews? Watch on If you're a Cochrane contributor and have ever attempted to explain Cochrane's work to someone, chances are you've tried to answer this question. And if you're reading this because you're new to Cochrane and the work we do, you may be wondering about this too.

  15. Cochrane Systematic Reviews

    Cochrane Handbook for Systematic Reviews of Interventions . The manual for authors of Cochrane systematic reviews. It provides detailed information on planning a review, searching, selecting studies, data collection, risk of bias assessment, statistical analysis, GRADE and interpreting results, as well as more specialized topics (non-randomized studies, adverse effects, economics, patient ...

  16. Cochrane Says Review Does Not Show That 'Face Masks Don't Work' Against

    Karla Soares-Weiser, Editor-in-Chief of the Cochrane Library, emphasized, "Many commentators have claimed that a recently-updated Cochrane Review shows that 'masks don't work,' which is an ...