1.2.2 What is a systematic review?
A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993) . The key characteristics of a systematic review are:
a clearly stated set of objectives with pre-defined eligibility criteria for studies;
an explicit, reproducible methodology;
a systematic search that attempts to identify all studies that would meet the eligibility criteria;
an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and
a systematic presentation, and synthesis, of the characteristics and findings of the included studies.
Many systematic reviews contain meta-analyses. Meta-analysis is the use of statistical methods to summarize the results of independent studies (Glass 1976). By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review (see Chapter 9, Section 9.1.3 ). They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies.
Jump to navigation
Cochrane Training
Chapter 1: starting a review.
Toby J Lasserson, James Thomas, Julian PT Higgins
Key Points:
- Systematic reviews address a need for health decision makers to be able to access high quality, relevant, accessible and up-to-date information.
- Systematic reviews aim to minimize bias through the use of pre-specified research questions and methods that are documented in protocols, and by basing their findings on reliable research.
- Systematic reviews should be conducted by a team that includes domain expertise and methodological expertise, who are free of potential conflicts of interest.
- People who might make – or be affected by – decisions around the use of interventions should be involved in important decisions about the review.
- Good data management, project management and quality assurance mechanisms are essential for the completion of a successful systematic review.
Cite this chapter as: Lasserson TJ, Thomas J, Higgins JPT. Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook .
1.1 Why do a systematic review?
Systematic reviews were developed out of a need to ensure that decisions affecting people’s lives can be informed by an up-to-date and complete understanding of the relevant research evidence. With the volume of research literature growing at an ever-increasing rate, it is impossible for individual decision makers to assess this vast quantity of primary research to enable them to make the most appropriate healthcare decisions that do more good than harm. By systematically assessing this primary research, systematic reviews aim to provide an up-to-date summary of the state of research knowledge on an intervention, diagnostic test, prognostic factor or other health or healthcare topic. Systematic reviews address the main problem with ad hoc searching and selection of research, namely that of bias. Just as primary research studies use methods to avoid bias, so should summaries and syntheses of that research.
A systematic review attempts to collate all the empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman et al 1992, Oxman and Guyatt 1993). Systematic review methodology, pioneered and developed by Cochrane, sets out a highly structured, transparent and reproducible methodology (Chandler and Hopewell 2013). This involves: the a priori specification of a research question; clarity on the scope of the review and which studies are eligible for inclusion; making every effort to find all relevant research and to ensure that issues of bias in included studies are accounted for; and analysing the included studies in order to draw conclusions based on all the identified research in an impartial and objective way.
This Handbook is about systematic reviews on the effects of interventions, and specifically about methods used by Cochrane to undertake them. Cochrane Reviews use primary research to generate new knowledge about the effects of an intervention (or interventions) used in clinical, public health or policy settings. They aim to provide users with a balanced summary of the potential benefits and harms of interventions and give an indication of how certain they can be of the findings. They can also compare the effectiveness of different interventions with one another and so help users to choose the most appropriate intervention in particular situations. The primary purpose of Cochrane Reviews is therefore to inform people making decisions about health or health care.
Systematic reviews are important for other reasons. New research should be designed or commissioned only if it does not unnecessarily duplicate existing research (Chalmers et al 2014). Therefore, a systematic review should typically be undertaken before embarking on new primary research. Such a review will identify current and ongoing studies, as well as indicate where specific gaps in knowledge exist, or evidence is lacking; for example, where existing studies have not used outcomes that are important to users of research (Macleod et al 2014). A systematic review may also reveal limitations in the conduct of previous studies that might be addressed in the new study or studies.
Systematic reviews are important, often rewarding and, at times, exciting research projects. They offer the opportunity for authors to make authoritative statements about the extent of human knowledge in important areas and to identify priorities for further research. They sometimes cover issues high on the political agenda and receive attention from the media. Conducting research with these impacts is not without its challenges, however, and completing a high-quality systematic review is often demanding and time-consuming. In this chapter we introduce some of the key considerations for potential review authors who are about to start a systematic review.
1.2 What is the review question?
Getting the research question right is critical for the success of a systematic review. Review authors should ensure that the review addresses an important question to those who are expected to use and act upon its conclusions.
We discuss the formulation of questions in detail in Chapter 2 . For a question about the effects of an intervention, the PICO approach is usually used, which is an acronym for Population, Intervention, Comparison(s) and Outcome. Reviews may have additional questions, for example about how interventions were implemented, economic issues, equity issues or patient experience.
To ensure that the review addresses a relevant question in a way that benefits users, it is important to ensure wide input. In most cases, question formulation should therefore be informed by people with various relevant – but potentially different – perspectives (see Chapter 2, Section 2.4 ).
1.3 Who should do a systematic review?
Systematic reviews should be undertaken by a team. Indeed, Cochrane will not publish a review that is proposed to be undertaken by a single person. Working as a team not only spreads the effort, but ensures that tasks such as the selection of studies for eligibility, data extraction and rating the certainty of the evidence will be performed by at least two people independently, minimizing the likelihood of errors. First-time review authors are encouraged to work with others who are experienced in the process of systematic reviews and to attend relevant training.
Review teams must include expertise in the topic area under review. Topic expertise should not be overly narrow, to ensure that all relevant perspectives are considered. Perspectives from different disciplines can help to avoid assumptions or terminology stemming from an over-reliance on a single discipline. Review teams should also include expertise in systematic review methodology, including statistical expertise.
Arguments have been made that methodological expertise is sufficient to perform a review, and that content expertise should be avoided because of the risk of preconceptions about the effects of interventions (Gøtzsche and Ioannidis 2012). However, it is important that both topic and methodological expertise is present to ensure a good mix of skills, knowledge and objectivity, because topic expertise provides important insight into the implementation of the intervention(s), the nature of the condition being treated or prevented, the relationships between outcomes measured, and other factors that may have an impact on decision making.
A Cochrane Review should represent an independent assessment of the evidence and avoiding financial and non-financial conflicts of interest often requires careful management. It will be important to consider if there are any relevant interests that may constitute a conflict of interest. There are situations where employment, holding of patents and other financial support should prevent people joining an author team. Funding of Cochrane Reviews by commercial organizations with an interest in the outcome of the review is not permitted. To ensure that any issues are identified early in the process, authors planning Cochrane Reviews should consult the Conflict of Interest Policy . Authors should make complete declarations of interest before registration of the review, and refresh these annually thereafter until publication and just prior to publication of the protocol and the review. For authors of review updates, this must be done at the time of the decision to update the review, annually thereafter until publication, and just prior to publication. Authors should also update declarations of interest at any point when their circumstances change.
1.3.1 Involving consumers and other stakeholders
Because the priorities of decision makers and consumers may be different from those of researchers, it is important that review authors consider carefully what questions are important to these different stakeholders. Systematic reviews are more likely to be relevant to a broad range of end users if they are informed by the involvement of people with a range of experiences, in terms of both the topic and the methodology (Thomas et al 2004, Rees and Oliver 2017). Engaging consumers and other stakeholders, such as policy makers, research funders and healthcare professionals, increases relevance, promotes mutual learning, improved uptake and decreases research waste.
Mapping out all potential stakeholders specific to the review question is a helpful first step to considering who might be invited to be involved in a review. Stakeholders typically include: patients and consumers; consumer advocates; policy makers and other public officials; guideline developers; professional organizations; researchers; funders of health services and research; healthcare practitioners, and, on occasion, journalists and other media professionals. Balancing seniority, credibility within the given field, and diversity should be considered. Review authors should also take account of the needs of resource-poor countries and regions in the review process (see Chapter 16 ) and invite appropriate input on the scope of the review and the questions it will address.
It is established good practice to ensure that consumers are involved and engaged in health research, including systematic reviews. Cochrane uses the term ‘consumers’ to refer to a wide range of people, including patients or people with personal experience of a healthcare condition, carers and family members, representatives of patients and carers, service users and members of the public. In 2017, a Statement of Principles for consumer involvement in Cochrane was agreed. This seeks to change the culture of research practice to one where both consumers and other stakeholders are joint partners in research from planning, conduct, and reporting to dissemination. Systematic reviews that have had consumer involvement should be more directly applicable to decision makers than those that have not (see online Chapter II ).
1.3.2 Working with consumers and other stakeholders
Methods for working with consumers and other stakeholders include surveys, workshops, focus groups and involvement in advisory groups. Decisions about what methods to use will typically be based on resource availability, but review teams should be aware of the merits and limitations of such methods. Authors will need to decide who to involve and how to provide adequate support for their involvement. This can include financial reimbursement, the provision of training, and stating clearly expectations of involvement, possibly in the form of terms of reference.
While a small number of consumers or other stakeholders may be part of the review team and become co-authors of the subsequent review, it is sometimes important to bring in a wider range of perspectives and to recognize that not everyone has the capacity or interest in becoming an author. Advisory groups offer a convenient approach to involving consumers and other relevant stakeholders, especially for topics in which opinions differ. Important points to ensure successful involvement include the following.
- The review team should co-ordinate the input of the advisory group to inform key review decisions.
- The advisory group’s input should continue throughout the systematic review process to ensure relevance of the review to end users is maintained.
- Advisory group membership should reflect the breadth of the review question, and consideration should be given to involving vulnerable and marginalized people (Steel 2004) to ensure that conclusions on the value of the interventions are well-informed and applicable to all groups in society (see Chapter 16 ).
Templates such as terms of reference, job descriptions, or person specifications for an advisory group help to ensure clarity about the task(s) required and are available from INVOLVE . The website also gives further information on setting and organizing advisory groups. See also the Cochrane training website for further resources to support consumer involvement.
1.4 The importance of reliability
Systematic reviews aim to be an accurate representation of the current state of knowledge about a given issue. As understanding improves, the review can be updated. Nevertheless, it is important that the review itself is accurate at the time of publication. There are two main reasons for this imperative for accuracy. First, health decisions that affect people’s lives are increasingly taken based on systematic review findings. Current knowledge may be imperfect, but decisions will be better informed when taken in the light of the best of current knowledge. Second, systematic reviews form a critical component of legal and regulatory frameworks; for example, drug licensing or insurance coverage. Here, systematic reviews also need to hold up as auditable processes for legal examination. As systematic reviews need to be both correct, and be seen to be correct, detailed evidence-based methods have been developed to guide review authors as to the most appropriate procedures to follow, and what information to include in their reports to aid auditability.
1.4.1 Expectations for the conduct and reporting of Cochrane Reviews
Cochrane has developed methodological expectations for the conduct, reporting and updating of systematic reviews of interventions (MECIR) and their plain language summaries ( Plain Language Expectations for Authors of Cochrane Summaries ; PLEACS). Developed collaboratively by methodologists and Cochrane editors, they are intended to describe the desirable attributes of a Cochrane Review. The expectations are not all relevant at the same stage of review conduct, so care should be taken to identify those that are relevant at specific points during the review. Different methods should be used at different stages of the review in terms of the planning, conduct, reporting and updating of the review.
Each expectation has a title, a rationale and an elaboration. For the purposes of publication of a review with Cochrane, each has the status of either ‘mandatory’ or ‘highly desirable’. Items described as mandatory are expected to be applied, and if they are not then an appropriate justification should be provided; failure to implement such items may be used as a basis for deciding not to publish a review in the Cochrane Database of Systematic Reviews (CDSR). Items described as highly desirable should generally be implemented, but there are reasonable exceptions and justifications are not required.
All MECIR expectations for the conduct of a review are presented in the relevant chapters of this Handbook . Expectations for reporting of completed reviews (including PLEACS) are described in online Chapter III . The recommendations provided in the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement have been incorporated into the Cochrane reporting expectations, ensuring compliance with the PRISMA recommendations and summarizing attributes of reporting that should allow a full assessment of the methods and findings of the review (Moher et al 2009).
1.5 Protocol development
Preparing a systematic review is complex and involves many judgements. To minimize the potential for bias in the review process, these judgements should be made as far as possible in ways that do not depend on the findings of the studies included in the review. Review authors’ prior knowledge of the evidence may, for example, influence the definition of a systematic review question, the choice of criteria for study eligibility, or the pre-specification of intervention comparisons and outcomes to analyse. It is important that the methods to be used should be established and documented in advance (see MECIR Box 1.5.a , MECIR Box 1.5.b and MECIR Box 1.5.c ).
Publication of a protocol for a review that is written without knowledge of the available studies reduces the impact of review authors’ biases, promotes transparency of methods and processes, reduces the potential for duplication, allows peer review of the planned methods before they have been completed, and offers an opportunity for the review team to plan resources and logistics for undertaking the review itself. All chapters in the Handbook should be consulted when drafting the protocol. Since systematic reviews are by their nature retrospective, an element of knowledge of the evidence is often inevitable. This is one reason why non-content experts such as methodologists should be part of the review team (see Section 1.3 ). Two exceptions to the retrospective nature of a systematic review are a meta-analysis of a prospectively planned series of trials and some living systematic reviews, as described in Chapter 22 .
The review question should determine the methods used in the review, and not vice versa. The question may concern a relatively straightforward comparison of one treatment with another; or it may necessitate plans to compare different treatments as part of a network meta-analysis, or assess differential effects of an intervention in different populations or delivered in different ways.
The protocol sets out the context in which the review is being conducted. It presents an opportunity to develop ideas that are foundational for the review. This concerns, most explicitly, definition of the eligibility criteria such as the study participants and the choice of comparators and outcomes. The eligibility criteria may also be defined following the development of a logic model (or an articulation of the aspects of an extent logic model that the review is addressing) to explain how the intervention might work (see Chapter 2, Section 2.5.1 ).
MECIR Box 1.5.a Relevant expectations for conduct of intervention reviews
A key purpose of the protocol is to make plans to minimize bias in the eventual findings of the review. Reliable synthesis of available evidence requires a planned, systematic approach. Threats to the validity of systematic reviews can come from the studies they include or the process by which reviews are conducted. Biases within the studies can arise from the method by which participants are allocated to the intervention groups, awareness of intervention group assignment, and the collection, analysis and reporting of data. Methods for examining these issues should be specified in the protocol. Review processes can generate bias through a failure to identify an unbiased (and preferably complete) set of studies, and poor quality assurance throughout the review. The availability of research may be influenced by the nature of the results (i.e. reporting bias). To reduce the impact of this form of bias, searching may need to include unpublished sources of evidence (Dwan et al 2013) ( MECIR Box 1.5.b ).
MECIR Box 1.5.b Relevant expectations for the conduct of intervention reviews
Developing a protocol for a systematic review has benefits beyond reducing bias. Investing effort in designing a systematic review will make the process more manageable and help to inform key priorities for the review. Defining the question, referring to it throughout, and using appropriate methods to address the question focuses the analysis and reporting, ensuring the review is most likely to inform treatment decisions for funders, policy makers, healthcare professionals and consumers. Details of the planned analyses, including investigations of variability across studies, should be specified in the protocol, along with methods for interpreting the results through the systematic consideration of factors that affect confidence in estimates of intervention effect ( MECIR Box 1.5.c ).
MECIR Box 1.5.c Relevant expectations for conduct of intervention reviews
While the intention should be that a review will adhere to the published protocol, changes in a review protocol are sometimes necessary. This is also the case for a protocol for a randomized trial, which must sometimes be changed to adapt to unanticipated circumstances such as problems with participant recruitment, data collection or event rates. While every effort should be made to adhere to a predetermined protocol, this is not always possible or appropriate. It is important, however, that changes in the protocol should not be made based on how they affect the outcome of the research study, whether it is a randomized trial or a systematic review. Post hoc decisions made when the impact on the results of the research is known, such as excluding selected studies from a systematic review, or changing the statistical analysis, are highly susceptible to bias and should therefore be avoided unless there are reasonable grounds for doing this.
Enabling access to a protocol through publication (all Cochrane Protocols are published in the CDSR ) and registration on the PROSPERO register of systematic reviews reduces duplication of effort, research waste, and promotes accountability. Changes to the methods outlined in the protocol should be transparently declared.
This Handbook provides details of the systematic review methods developed or selected by Cochrane. They are intended to address the need for rigour, comprehensiveness and transparency in preparing a Cochrane systematic review. All relevant chapters – including those describing procedures to be followed in the later stages of the review – should be consulted during the preparation of the protocol. A more specific description of the structure of Cochrane Protocols is provide in online Chapter II .
1.6 Data management and quality assurance
Systematic reviews should be replicable, and retaining a record of the inclusion decisions, data collection, transformations or adjustment of data will help to establish a secure and retrievable audit trail. They can be operationally complex projects, often involving large research teams operating in different sites across the world. Good data management processes are essential to ensure that data are not inadvertently lost, facilitating the identification and correction of errors and supporting future efforts to update and maintain the review. Transparent reporting of review decisions enables readers to assess the reliability of the review for themselves.
Review management software, such as Covidence and EPPI-Reviewer , can be used to assist data management and maintain consistent and standardized records of decisions made throughout the review. These tools offer a central repository for review data that can be accessed remotely throughout the world by members of the review team. They record independent assessment of studies for inclusion, risk of bias and extraction of data, enabling checks to be made later in the process if needed. Research has shown that even experienced reviewers make mistakes and disagree with one another on risk-of-bias assessments, so it is particularly important to maintain quality assurance here, despite its cost in terms of author time. As more sophisticated information technology tools begin to be deployed in reviews (see Chapter 4, Section 4.6.6.2 and Chapter 22, Section 22.2.4 ), it is increasingly apparent that all review data – including the initial decisions about study eligibility – have value beyond the scope of the individual review. For example, review updates can be made more efficient through (semi-) automation when data from the original review are available for machine learning.
1.7 Chapter information
Authors: Toby J Lasserson, James Thomas, Julian PT Higgins
Acknowledgements: This chapter builds on earlier versions of the Handbook . We would like to thank Ruth Foxlee, Richard Morley, Soumyadeep Bhaumik, Mona Nasser, Dan Fox and Sally Crowe for their contributions to Section 1.3 .
Funding: JT is supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. JPTH is a member of the NIHR Biomedical Research Centre at University Hospitals Bristol NHS Foundation Trust and the University of Bristol. JPTH received funding from National Institute for Health Research Senior Investigator award NF-SI-0617-10145. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
1.8 References
Antman E, Lau J, Kupelnick B, Mosteller F, Chalmers T. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: treatment for myocardial infarction. JAMA 1992; 268 : 240–248.
Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, Howells DW, Ioannidis JP, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet 2014; 383 : 156–165.
Chandler J, Hopewell S. Cochrane methods – twenty years experience in developing systematic review methods. Systematic Reviews 2013; 2 : 76.
Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias: an updated review. PloS One 2013; 8 : e66844.
Gøtzsche PC, Ioannidis JPA. Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses? BMJ 2012; 345 .
Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, Al-Shahi Salman R, Chan AW, Glasziou P. Biomedical research: increasing value, reducing waste. Lancet 2014; 383 : 101–104.
Moher D, Liberati A, Tetzlaff J, Altman D, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine 2009; 6 : e1000097.
Oxman A, Guyatt G. The science of reviewing research. Annals of the New York Academy of Sciences 1993; 703 : 125–133.
Rees R, Oliver S. Stakeholder perspectives and participation in reviews. In: Gough D, Oliver S, Thomas J, editors. An Introduction to Systematic Reviews . 2nd ed. London: Sage; 2017. p. 17–34.
Steel R. Involving marginalised and vulnerable people in research: a consultation document (2nd revision). INVOLVE; 2004.
Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton G, Kavanagh J. Integrating qualitative research with trials in systematic reviews. BMJ 2004; 328 : 1010–1012.
For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.
An official website of the United States government
The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
- Publications
- Account settings
- My Bibliography
- Collections
- Citation manager
Save citation to file
Email citation, add to collections.
- Create a new collection
- Add to an existing collection
Add to My Bibliography
Your saved search, create a file for external citation management software, your rss feed.
- Search in PubMed
- Search in NLM Catalog
- Add to Search
What is a Cochrane review?
Affiliation.
- 1 Department of Public Health and Community Medicine, Section of Psychiatry and Clinical Psychology, University of Verona, Verona, Italy. [email protected]
- PMID: 21922964
- DOI: 10.1017/s2045796011000436
Systematic reviews carried out by Cochrane Collaboration (an international network of researchers belonging to this independent, not-for-profit organization) are recognized worldwide as the highest standard in evidence-based healthcare. The main reason is that Cochrane reviews follow a common and specific methodology to limit bias and random error. In this issue, we highlight the most important methodological features of Cochrane reviews, also reporting details on the editorial process to publish the review in the Cochrane Library.
Similar articles
- The Cochrane Collaboration. Scholten RJ, Clarke M, Hetherington J. Scholten RJ, et al. Eur J Clin Nutr. 2005 Aug;59 Suppl 1:S147-9; discussion S195-6. doi: 10.1038/sj.ejcn.1602188. Eur J Clin Nutr. 2005. PMID: 16052183
- Methodological developments in searching for studies for systematic reviews: past, present and future? Lefebvre C, Glanville J, Wieland LS, Coles B, Weightman AL. Lefebvre C, et al. Syst Rev. 2013 Sep 25;2:78. doi: 10.1186/2046-4053-2-78. Syst Rev. 2013. PMID: 24066664 Free PMC article.
- [Present status and perspectives in international cooperation in the Cochrane Collaboration]. Kasal P, Naidr J, Feberová J, Hladíková M. Kasal P, et al. Sb Lek. 1998;99(4):601-7. Sb Lek. 1998. PMID: 10803309 Czech.
- Evidence-based medicine for consumers: a role for the Cochrane Collaboration. White PJ. White PJ. J Med Libr Assoc. 2002 Apr;90(2):218-22. J Med Libr Assoc. 2002. PMID: 11999180 Free PMC article. Review.
- International activity in the Cochrane Collaboration with particular reference to India. Allen C, Clarke M, Tharyan P. Allen C, et al. Natl Med J India. 2007 Sep-Oct;20(5):250-5. Natl Med J India. 2007. PMID: 18254524
- Reviews of attitude research in implementation science require comprehensiveness, accuracy, and specificity. Aarons GA. Aarons GA. Implement Sci. 2022 May 10;17(1):28. doi: 10.1186/s13012-022-01198-4. Implement Sci. 2022. PMID: 35534892 Free PMC article. No abstract available.
- Treatment Guidelines for Rare, Early-Onset, Treatment-Resistant Epileptic Conditions: A Literature Review on Dravet Syndrome, Lennox-Gastaut Syndrome and CDKL5 Deficiency Disorder. Chin RF, Mingorance A, Ruban-Fell B, Newell I, Evans J, Vyas K, Nortvedt C, Amin S. Chin RF, et al. Front Neurol. 2021 Oct 25;12:734612. doi: 10.3389/fneur.2021.734612. eCollection 2021. Front Neurol. 2021. PMID: 34759881 Free PMC article.
Publication types
- Search in MeSH
- Citation Manager
NCBI Literature Resources
MeSH PMC Bookshelf Disclaimer
The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.
Have a language expert improve your writing
Run a free plagiarism check in 10 minutes, generate accurate citations for free.
- Knowledge Base
- Methodology
- Systematic Review | Definition, Example, & Guide
Systematic Review | Definition, Example & Guide
Published on June 15, 2022 by Shaun Turney . Revised on December 7, 2022.
A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.
They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”
In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.
Table of contents
What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, frequently asked questions about systematic reviews.
A review is an overview of the research that’s already been completed on a topic.
What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:
- Formulate a research question
- Develop a protocol
- Search for all relevant studies
- Apply the selection criteria
- Extract the data
- Synthesize the data
- Write and publish a report
Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.
Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.
Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.
Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.
A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .
Receive feedback on language, structure, and formatting
Professional editors proofread and edit your paper by focusing on:
- Academic style
- Vague sentences
- Style consistency
See an example
A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.
Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.
Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.
However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.
Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.
A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.
To conduct a systematic review, you’ll need the following:
- A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
- If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
- Access to databases and journal archives. Often, your educational institution provides you with access.
- Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
- Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.
A systematic review has many pros .
- They minimize research bias by considering all available evidence and evaluating each study for bias.
- Their methods are transparent , so they can be scrutinized by others.
- They’re thorough : they summarize all available evidence.
- They can be replicated and updated by others.
Systematic reviews also have a few cons .
- They’re time-consuming .
- They’re narrow in scope : they only answer the precise research question.
The 7 steps for conducting a systematic review are explained with an example.
Step 1: Formulate a research question
Formulating the research question is probably the most important step of a systematic review. A clear research question will:
- Allow you to more effectively communicate your research to other researchers and practitioners
- Guide your decisions as you plan and conduct your systematic review
A good research question for a systematic review has four components, which you can remember with the acronym PICO :
- Population(s) or problem(s)
- Intervention(s)
- Comparison(s)
You can rearrange these four components to write your research question:
- What is the effectiveness of I versus C for O in P ?
Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .
- Type of study design(s)
- The population of patients with eczema
- The intervention of probiotics
- In comparison to no treatment, placebo , or non-probiotic treatment
- The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
- Randomized control trials, a type of study design
Their research question was:
- What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?
Step 2: Develop a protocol
A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.
Your protocol should include the following components:
- Background information : Provide the context of the research question, including why it’s important.
- Research objective (s) : Rephrase your research question as an objective.
- Selection criteria: State how you’ll decide which studies to include or exclude from your review.
- Search strategy: Discuss your plan for finding studies.
- Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.
If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.
It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .
Step 3: Search for all relevant studies
Searching for relevant studies is the most time-consuming step of a systematic review.
To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:
- Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
- Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
- Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
- Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.
At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .
- Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
- Handsearch: Conference proceedings and reference lists of articles
- Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
- Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics
Step 4: Apply the selection criteria
Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.
To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.
If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.
You should apply the selection criteria in two phases:
- Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
- Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.
It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .
Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.
When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.
Step 5: Extract the data
Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:
- Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
- Your judgment of the quality of the evidence, including risk of bias .
You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .
Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.
They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.
Step 6: Synthesize the data
Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:
- Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
- Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.
Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.
Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.
Step 7: Write and publish a report
The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.
Your article should include the following sections:
- Abstract : A summary of the review
- Introduction : Including the rationale and objectives
- Methods : Including the selection criteria, search method, data extraction method, and synthesis method
- Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
- Discussion : Including interpretation of the results and limitations of the review
- Conclusion : The answer to your research question and implications for practice, policy, or research
To verify that your report includes everything it needs, you can use the PRISMA checklist .
Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.
A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .
It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.
A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other academic texts , with an introduction , a main body, and a conclusion .
An annotated bibliography is a list of source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a paper .
A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.
Cite this Scribbr article
If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.
Turney, S. (2022, December 07). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved March 17, 2023, from https://www.scribbr.com/methodology/systematic-review/
Is this article helpful?
Shaun Turney
Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.
Cochrane Consumer Network
Cochrane and systematic reviews, about the cochrane library, systematic reviews.
- How do I know an intervention works
- What consumers can and cannot get from a review
- Levels of evidence
- Cochrane groups
Top of page
The Cochrane Library is an electronic collection of databases published on the internet and also available on CD-Rom. It is updated quarterly in an effort to add to and keep the information current. The Library is made up of a number of parts.
The Cochrane Database of Systematic Reviews (CDSR) contains the published Cochrane reviews and protocols.
The Cochrane Central Register of Controlled Trials (CENTRAL) collates references to controlled trials in health care. These healthcare trial references are entered by Cochrane groups. The main way of finding health care studies is by looking in electronic databases (such as MEDLINE, EMBASE, CINAHL) using special search terms. Other ways are by asking experts in a particular health field and through hand searching journals.
The Database of Abstracts of Reviews of Effects (DARE) is a collection of structured abstracts and bibliographic references of systematic reviews of the effects of health care. It is developed by the Centre for Research and Dissemination, University of York, UK.
Methodological reviews and articles are also presented in The Cochrane Library.
In addition, each Cochrane group (termed an entity) has a section (module) in the Library that gives information on the group’s organisation, contact details, function, reviews, and other general information.
Abstracts of reviews are readily accessible at www.cochrane.org/reviews . In countries such as Australia, Denmark, Finland, Ireland, Latin America, Norway and UK the full reviews are freely available as the governments of these countries have subscriptions to The Cochrane Library . Consumers who live in other countries and who wish to read a full review may need to access The Cochrane Library through a university, hospital or large public library.
A Cochrane Library Users Guide is available ( https://www.cochrane.org.au/libraryguide/ ) to help you find the information you want from The Cochrane Library.
Brief summaries (plain language summaries) of Cochrane reviews are written for consumers and others to highlight the information in a review. A What’s New Digest summarises the newest reviews.
If you would like to make comments on any existing review in The Cochrane Library, you will find a special section for 'Comments and Criticisms' with the review.
If someone decides to look critically at articles that have appeared in the medical or health literature on a particular topic they are said to be ‘reviewing the literature’. The authors may review, say, all the drug treatments available for one type of heart disease. A review is very clearly defined and sets out to find what evidence there is for prescribing one particular intervention or drug in a specific health condition, often in a certain group of people.
Examples of review topics are: Single dose celecoxib for acute postoperative pain; Artichoke leaf extract for treating hypercholesterolaemia; Chocolate avoidance for preventing migraine; Etidronate for treating and preventing postmenopausal osteoporosis.
A systematic review summarises the results of available carefully designed healthcare studies (controlled trials) and provides a high level of evidence on the effectiveness of healthcare interventions.
The review authors set about their task very methodically following, step by step, an advance plan that covers:
- the way existing studies are found;
- how the relevant studies are judged in terms of their usefulness in answering the review question;
- how the results of the separate studies are brought together to give an overall measure of effectiveness (benefits and harms) – statistical techniques used to combine the results are called meta-analysis.
A protocol is the plan or set of steps to be followed in preparing a review. A protocol for a systematic review clearly describes why the review is needed (the review question), what the review is about (the healthcare context of the review), and how the reviewer authors will go about developing the review. It details how they will seek, select as relevant, critically appraise studies, and collect and analyse data (combine data and check for significance to the healthcare situation) from the included studies.
Cochrane protocols are published in the Cochrane Database of Systematic Reviews so that people can comment on them before the actual review has been carried out.
How do I know a healthcare intervention works?
The aim of a systematic review is to thoroughly assess, by means of a set procedure, the best possible evidence about the effects of a healthcare intervention or treatment in a particular healthcare situation.
Healthcare studies are generally designed to assess the benefits, rather than the harms, of an intervention. Studies generally have a relatively short designated time period. Any possible harms of an intervention may be expected to occur less frequently and over a longer period of time than the studies cover.
The process of a review is clearly defined, before starting the actual review of the literature, to minimise associations of expectations of effects and other sources of bias. Bias is a systematic ‘error’ or mistake in the judgments and decisions made that influence the results of a study or a review. Bias differs from a ‘placebo effect’, which is where participants of a study (or assessors of the outcomes) perceive a beneficial effect, or harm, with an inactive treatment.
The specific methods used in a review are carefully set out by The Cochrane Collaboration and are described in each review.
A Cochrane review is prepared and maintained using specific methodologies described in the Cochrane Handbook .
Systematic reviews of randomised controlled trials provide the clearest evidence for the benefits of a healthcare intervention.
This is because the best way to assess the effects of a health care treatment is to use procedures that reduce the influence of chance effects and associations of cause and effect. Individual expectations on the part of a service provider, assessor and the person receiving an intervention can all contribute to modifying observed findings from a healthcare study. Randomised controlled trials where none of these people know the exact intervention a study participant is receiving (intervention under investigation, a placebo, or a comparator) may be expected to provide the best evidence.
By assessing the health of the two comparative groups in a study after their treatments, we can tell which intervention is more successful – but only if the two groups of people were very similar before treatment began. Otherwise we might be misled. For instance, one group may become healthier not because their treatment was better but because they were younger, not so ill, at less risk of ill health before treatment began, or even self selected to a particular intervention because of a particular personality trait, for example, people who chose to take a hormone may have wanted to stay younger and be more active.
Randomised controlled trials are studies that are rigorously designed. People are allocated to intervention groups in a way that minimises the chances of predicting which treatment group a study participant is in. The intervention under investigation is compared against a well-known intervention or an inactive treatment (placebo). Studies are controlled so that participants have similar associated care in all ways other than the intervention. Ideally, depending on the type of intervention, the service provider is unaware of which group a participant is in and those assessing outcomes are also unaware – this is termed ’ blinding ’.
The strength of evidence for a particular intervention can be increased further by systematically looking at (reviewing) all available randomized controlled trials that have been reported relevant to a particular healthcare situation.
Many people are needed to properly test an intervention. This is more than can be recruited into a single trial; it is also important to investigate the intervention in different populations. Furthermore, the technical aspects of a particular randomised controlled trial may nothave been implemented properly, for one reason or another. The effects of these shortcomings can be minimised by grouping results of a number of studies.
The results of randomised controlled trials may be published in any one of thousands of journals world wide. Indeed some studies are not published at all. In reality the studies found most easily tend to have over-optimistic results and finding reliable information about the effects of care is particularly difficult when there are negative results (the intervention is no better than placebo or another treatment). Sometimes published trials are too small to provide a conclusive result in their own right - as to whether a treatment really does work. Consequently, to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months, but it will be much less work than conducting a new randomised controlled trial. This process also will not unnecessarily exclude people from effective interventions because of allocation to a placebo (or inactive treatment) group.Discussions are underway in The Cochrane Collaboration as to how qualitative studies can be used to add to the information obtained from controlled studies - those that consider outcomes measured in numerical terms (and so are termed quantitative studies). Qualitative measures include ‘quality of life’ and lifestyle changes obtained from detailed questionnaires. Qualitative studies may also use narrative interviews where participants are asked to talk about their experiences around sets of semi-structured questions and prompts to explore particular issues that information is needed on for a study.
Top of page
What consumers can, and cannot, get from systematic reviews
Systematic reviews ask a very specific research question about a particular intervention in a clearly defined group of people who have a clear health condition or problem. Reviews provide powerful information on the state of knowledge about a healthcare intervention and whether that intervention is an effective treatment of a healthcare condition.Reviews:
- cannot offer a guideline for treatment, especially if a person differs from those defined in the review. Individuals may have accompanying health problems, be in a different healthcare setting, or receive more than one intervention, for example;
- follow stringent guidelines as to what types of studies are included and how healthcare measures of effectiveness can be expressed and combined;
- may consider outcomes other than the one you are interested in and may not look at long term effects of an intervention;
- may only find studies that are limited in the healthcare setting in which they take place;
- may provide conclusions that are limited because of the question asked and/or the studies that were found.
Healthcare studies differ dramatically in what they look for and how well they are carried out and, therefore, how much weight one canput on their conclusions. Part of the reason for performing systematic reviews is to reduce the effects of these shortcomings. Issues of conflict of interest and corporate funding of healthcare studies are also important considerations in drawing conclusions from any study.
Well-designed healthcare studies generally set out to determine the efficacy of a healthcare intervention. Information on potential harms is less well investigated.
Carefully controlled studies take place over a limited period of time so that the researchers can account for all people who entered the study from beginning to the end of the study. Harms are generally less common than benefits and may be apparent over a different time period. This may be, for example, only in the long term so that the intervention would have to be given to more people for a long time period for adverse effects to be studied effectively.
Participants of studies are selected to reduce the risk of other problems interfering with the efficacy of an intervention. How selective thisprocess is needs to be carefully considered when assessing the relevance of a study to an individual.
Randomised controlled trials are expensive to run. They are very time consuming and multiple factors may limit how many participants are involved, the outcomes measured and the length of the trial. How many people complete the study is also very important.
Levels of evidence for healthcare interventions
The National Health and Medical Research Council of Australia (1999) defines the ‘dimensions of evidence’ using three main areas.
Level of evidence: the study design used – a systematic review of all relevant randomised controlled trials is the highest level, followed by at least one randomised controlled trial, then a pseudo-randomised trial Quality of evidence: the methods used to minimise bias within a study design Statistical precision: the degree of certainty about the existence of a true effect
2. Size of effect
How much the determined intervention effect is above a ‘no apparent effect’ value for clinically relevant effects
3. Relevance of the evidence
How appropriate the outcome measure is for the healthcare problem, and its usefulness in measuring effectiveness of treatment
Adapted from AD.Oxman Checklists for review articles. BMJ 1994;309:648-51
Level I. For a randomised controlled trial, the lower limit of the confidence interval (expressed as a range) for a measure of effect is still above a meaningful benefit in healthcare terms
Level II. For a randomised controlled trial, the lower limit of the confidence interval (expressed as a range) for a measure of effect is less than a meaningful beneficial effect in healthcare terms; but the point estimate of effect still shows effectiveness of the intervention
Level III. Measures of effectiveness are taken from non-randomised studies of groups of people where a control group has run concurrently with the group receiving the intervention being assessed
Level IV. Measures of effectiveness are taken from non-randomised studies of groups of people where intervention effects are compared with previous or historical information
Level V. Evidence is from single case studies
Even studies perfectly designed and carried out may show variable results because of the play of chance. CI covers the likely range of the true effect. For example, the result of a study may be that 40 per cent (95% CI 30% to 50%) of people are helped by a treatment. That means that we can be 95 per cent certain the true effect is between 30 and 50 per cent. ( Smart Health Choices How to make informed health decisions by Judy Irwig, Les Irwig and Melissa Sweet, Allen and Unwin 1999)
Top of page
Cochrane Groups
Different groups exist for different health conditions: International Cochrane review groups cover important areas of health care diseases and conditions. Review groups are responsible for producing and maintaining Cochrane reviews on specific health care questions. You will see in The Cochrane Library, for example, a Cochrane Consumers and Communication Group, Cochrane Epilepsy Group, Cochrane Heart Group and a Cochrane Pregnancy and Childbirth Group.
The activities of each group (or entity in Cochrane language) are monitored and co-ordinated by one person for each group, known as the managing editor (review group co-ordinator). This person manages the day to day running of the group and is usually the contact person. The co-ordinating editor leads the group and is responsible for the quality and subject of reviews.
Each group attracts members with a variety of backgrounds, experience and expertise, who contribute to the process of developing systematic reviews. They may be doctors, nurses, researchers, health advisers, consumers and caregivers.
Fields cover health care in a broader sense than do review groups. These may include a major section of health care such as cancer, the setting of care (e.g. primary care), the type of patient/consumer (e.g. older persons), the type of provider (e.g. nurses), or the type of intervention (e.g. vaccines). The role of fields is to facilitate the work of collaborative review groups and to ensure that Cochrane reviews appropriate to an area of interest are both relevant and accessible to service providers and consumers.
Each field works to:
- identify relevant healthcare trials and make them accessible in a specialised register;
- ensure the proper representation of its specialist area of health care in review groups;
- act as a liaison point between the entities within The Cochrane Collaboration and the specialist area of health care;
- promote the accessibility of Cochrane reviews.
The principal contact person in a field is its field co-ordinator.
Cochrane centres provide a range of services designed to support collaborative review groups in their area and to facilitate the review process. They serve as a regional source of information about The Cochrane Collaboration, provide support to Cochrane contributors within a defined geographical area and promote access to The Cochrane Library. Each centre has a director.
The Consumer Network supports consumer participation within The Cochrane Collaboration, internationally. The Network is available to any active consumer. Its mission is to enable and support consumers in contributing to the function of collaborative reviews groups and other Cochrane entities. The Network enables communication with other consumers, provides a sense of belonging within The Cochrane Collaboration, links and dissemination of information from Cochrane reviews.
© The Cochrane Collaboration Comments about this page to: [email protected]
Jump to navigation
- Bahasa Malaysia
VIDEO: What are systematic reviews?
A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question. Researchers conducting systematic reviews use explicit, systematic methods that are selected with a view aimed at minimizing bias, to produce more reliable findings to inform decision making.
Here is a video from Cochrane Consumers and Communication that explains what a systematic review is clearly and simply for people who may not be familiar with the concepts and terminology of systematic reviews: what they are, how researchers prepare them, and why they’re an important part of making informed decisions about health - for everyone.
Cochrane evidence provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence, including systematic reviews, and other activities of Cochrane.
- What is the difference between a Cochrane systematic review of interventions and a Cochrane diagnostic test accuracy review?
- Learn more about Cochrane and our work
- Join Cochrane
Information
- Author Services
Initiatives
You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.
All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .
Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.
Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.
Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.
- Active Journals
- Find a Journal
- Proceedings Series
- For Authors
- For Reviewers
- For Editors
- For Librarians
- For Publishers
- For Societies
- For Conference Organizers
- Open Access Policy
- Institutional Open Access Program
- Special Issues Guidelines
- Editorial Process
- Research and Publication Ethics
- Article Processing Charges
- Testimonials
- Preprints.org
- SciProfiles
- Encyclopedia
Article Menu
- Subscribe SciFeed
- Google Scholar
- on Google Scholar
- Table of Contents
Find support for a specific problem in the support section of our website.
Please let us know what you think of our products and services.
Visit our dedicated information section to learn more about MDPI.
JSmol Viewer
A systematic review and meta-analysis of supramarginal resection versus gross total resection in glioblastoma: can we enhance progression-free survival time and preserve postoperative safety.
Simple Summary
1. introduction, 2. materials and methods, 2.1. inclusion and exclusion criteria, 2.2. data extraction and clinical endpoints, 2.3. statistics, 3.1. literature search, 3.2. characteristics of included studies regarding the analysis of progression-free survival, 3.3. impact of supramarginal resection on progression-free survival in glioblastoma, 3.4. characteristics of included studies regarding the analysis of perioperative complications, 3.5. impact of supramarginal resection on postoperative complications, 3.5.1. postoperative meningitis, 3.5.2. postoperative intracranial hemorrhage, 3.5.3. postoperative csf leaks, 3.6. publication bias, 4. discussion, 4.1. supramarginal resection and postoperative complications, 4.2. supramarginal resection and probability of progression-free survival, 4.3. limitations, 5. conclusions, supplementary materials, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.
- Louis, D.N.; Perry, A.; Wesseling, P.; Brat, D.J.; Cree, I.A.; Figarella-Branger, D.; Hawkins, C.; Ng, H.K.; Pfister, S.M.; Reifenberger, G.; et al. The 2021 WHO Classification of Tumors of the Central Nervous System: A summary. Neuro Oncol. 2021 , 23 , 1231–1251. [ Google Scholar ] [ CrossRef ]
- Stupp, R.; Mason, W.P.; van den Bent, M.J.; Weller, M.; Fisher, B.; Taphoorn, M.J.; Belanger, K.; Brandes, A.A.; Marosi, C.; Bogdahn, U.; et al. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N. Engl. J. Med. 2005 , 352 , 987–996. [ Google Scholar ] [ CrossRef ]
- Omuro, A.; Brandes, A.A.; Carpentier, A.F.; Idbaih, A.; Reardon, D.A.; Cloughesy, T.; Sumrall, A.; Baehring, J.; van den Bent, M.; Bähr, O.; et al. Radiotherapy combined with nivolumab or temozolomide for newly diagnosed glioblastoma with unmethylated MGMT promoter: An international randomized phase III trial. Neuro Oncol. 2023 , 25 , 123–134. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Lassman, A.B.; Pugh, S.L.; Wang, T.J.C.; Aldape, K.; Gan, H.K.; Preusser, M.; Vogelbaum, M.A.; Sulman, E.P.; Won, M.; Zhang, P.; et al. Depatuxizumab-mafodotin in EGFR-amplified newly diagnosed glioblastoma: A phase III randomized clinical trial. Neuro Oncol. 2022 , 15 , noac173. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Sahm, F.; Capper, D.; Jeibmann, A.; Habel, A.; Paulus, W.; Troost, D.; von Deimling, A. Addressing diffuse glioma as a systemic brain disease with single-cell analysis. Arch. Neurol. 2012 , 69 , 523–526. [ Google Scholar ] [ PubMed ]
- Sherriff, J.; Tamangani, J.; Senthil, L.; Cruickshank, G.; Spooner, D.; Jones, B.; Brookes, C.; Sanghera, P. Patterns of relapse in glioblastoma multiforme following concomitant chemoradiotherapy with temozolomide. Br. J. Radiol. 2013 , 86 , 20120414. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Wallner, K.E.; Galicich, J.H.; Krol, G.; Arbit, E.; Malkin, M.G. Patterns of failure following treatment for glioblastoma multiforme and anaplastic astrocytoma. Int. J. Radiat. Oncol. Biol. Phys. 1989 , 16 , 1405–1409. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Hoelzinger, D.B.; Demuth, T.; Berens, M.E. Autocrine factors that sustain glioma invasion and paracrine biology in the brain microenvironment. J. Natl. Cancer Inst. 2007 , 99 , 1583–1593. [ Google Scholar ] [ CrossRef ]
- Osswald, M.; Jung, E.; Sahm, F.; Solecki, G.; Venkataramani, V.; Blaes, J.; Weil, S.; Horstmann, H.; Wiestler, B.; Syed, M.; et al. Brain tumour cells interconnect to a functional and resistant network. Nature 2015 , 528 , 93–98. [ Google Scholar ] [ CrossRef ]
- Tripathi, S.; Vivas-Buitrago, T.; Domingo, R.A.; Biase, G.; Brown, D.; Akinduro, O.O.; Ramos-Fresnedo, A.; Sherman, W.; Gupta, V.; Middlebrooks, E.H.; et al. IDH-wild-type glioblastoma cell density and infiltration distribution influence on supramarginal resection and its impact on overall survival: A mathematical model. J. Neurosurg. 2021 , 136 , 1567–1575. [ Google Scholar ] [ CrossRef ]
- Stummer, W.; Reulen, H.J.; Meinel, T.; Pichlmeier, U.; Schumacher, W.; Tonn, J.C.; Rohde, V.; Oppel, F.; Turowski, B.; Woiciechowsky, C.; et al. Extent of resection and survival in glioblastoma multiforme: Identification of and adjustment for bias. Neurosurgery 2008 , 62 , 564–576. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Brown, T.J.; Brennan, M.C.; Li, M.; Church, E.W.; Brandmeir, N.J.; Rakszawski, K.L.; Patel, A.S.; Rizk, E.B.; Suki, D.; Sawaya, R.; et al. Association of the Extent of Resection with Survival in Glioblastoma: A Systematic Review and Meta-analysis. JAMA Oncol. 2016 , 2 , 1460–1469. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Jackson, C.; Choi, J.; Khalafallah, A.M.; Price, C.; Bettegowda, C.; Lim, M.; Gallia, G.; Weingart, J.; Brem, H.; Mukherjee, D. A systematic review and meta-analysis of supratotal versus gross total resection for glioblastoma. J. Neurooncol. 2020 , 148 , 419–431. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Duffau, H. Surgery for Malignant Brain Gliomas: Fluorescence-Guided Resection or Functional-Based Resection? Front. Surg. 2019 , 6 , 21. [ Google Scholar ] [ CrossRef ]
- Guerrini, F.; Roca, E.; Spena, G. Supramarginal Resection for Glioblastoma: It Is Time to Set Boundaries! A Critical Review on a Hot Topic. Brain Sci. 2022 , 12 , 652. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Goenka, A.; Tiek, D.; Song, X.; Huang, T.; Hu, B.; Cheng, S.Y. The Many Facets of Therapy Resistance and Tumor Recurrence in Glioblastoma. Cells 2021 , 10 , 484. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Sagberg, L.M.; Iversen, D.H.; Fyllingen, E.H.; Jakola, A.S.; Reinertsen, I.; Solheim, O. Brain atlas for assessing the impact of tumor location on perioperative quality of life in patients with high-grade glioma: A prospective population-based cohort study. Neuroimage Clin. 2019 , 21 , 101658. [ Google Scholar ] [ CrossRef ]
- Sagberg, L.M.; Solheim, O.; Jakola, A.S. Quality of survival the 1st year with glioblastoma: A longitudinal study of patient-reported quality of life. J. Neurosurg. 2016 , 124 , 989–997. [ Google Scholar ] [ CrossRef ]
- Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Int. J. Surg. 2021 , 88 , 105906. [ Google Scholar ] [ CrossRef ]
- Higgins, J.P.T.; Thomas, J.; Chandler, J.; Cumpston, M.; Li, T.; Page, M.J.; Welch, V.A. (Eds.) Cochrane Handbook for Systematic Reviews of Interventions Version 6.3 (Updated February 2022): Cochrane. 2022. Available online: https://training.cochrane.org/handbook/current (accessed on 30 November 2022).
- Schardt, C.; Adams, M.B.; Owens, T.; Keitz, S.; Fontelo, P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med. Inform. Decis. Mak. 2007 , 7 , 16. [ Google Scholar ] [ CrossRef ]
- Higgins, J.P.; Thompson, S.G.; Deeks, J.J.; Altman, D.G. Measuring inconsistency in meta-analyses. BMJ 2003 , 327 , 557–560. [ Google Scholar ] [ CrossRef ]
- Begg, C.B.; Mazumdar, M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994 , 50 , 1088–1101. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Glenn, C.A.; Baker, C.M.; Conner, A.K.; Burks, J.D.; Bonney, P.A.; Briggs, R.G.; Smitherman, A.D.; Battiste, J.D.; Sughrue, M.E. An Examination of the Role of Supramaximal Resection of Temporal Lobe Glioblastoma Multiforme. World Neurosurg. 2018 , 114 , e747–e755. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- De Bonis, P.; Anile, C.; Pompucci, A.; Fiorentino, A.; Balducci, M.; Chiesa, S.; Lauriola, L.; Maira, G.; Mangiola, A. The influence of surgery on recurrence pattern of glioblastoma. Clin. Neurol. Neurosurg. 2013 , 115 , 37–43. [ Google Scholar ] [ CrossRef ]
- Schneider, M.; Potthoff, A.L.; Keil, V.C.; Güresir, A.; Weller, J.; Borger, V.; Hamed, M.; Waha, A.; Vatter, H.; Güresir, E.; et al. Surgery for temporal glioblastoma: Lobectomy outranks oncosurgical-based gross-total resection. J. Neurooncol. 2019 , 145 , 143–150. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Pessina, F.; Navarria, P.; Cozzi, L.; Ascolese, A.M.; Simonelli, M.; Santoro, A.; Clerici, E.; Rossi, M.; Scorsetti, M.; Bello, L. Maximize surgical resection beyond contrast-enhancing boundaries in newly diagnosed glioblastoma multiforme: Is it useful and safe? A single institution retrospective experience. J. Neurooncol. 2017 , 135 , 129–139. [ Google Scholar ] [ CrossRef ]
- Roh, T.H.; Kang, S.G.; Moon, J.H.; Sung, K.S.; Park, H.H.; Kim, S.H.; Kim, E.H.; Hong, C.K.; Suh, C.O.; Chang, J.H. Survival benefit of lobectomy over gross-total resection without lobectomy in cases of glioblastoma in the noneloquent area: A retrospective study. J. Neurosurg. 2019 , 132 , 895–901. [ Google Scholar ] [ CrossRef ]
- Shah, A.H.; Mahavadi, A.; Di, L.; Sanjurjo, A.; Eichberg, D.G.; Borowy, V.; Figueroa, J.; Luther, E.; de la Fuente, M.I.; Semonche, A.; et al. Survival benefit of lobectomy for glioblastoma: Moving towards radical supramaximal resection. J. Neurooncol. 2020 , 148 , 501–508. [ Google Scholar ] [ CrossRef ]
- Mampre, D.; Ehresman, J.; Pinilla-Monsalve, G.; Osorio, M.A.G.; Olivi, A.; Quinones-Hinojosa, A.; Chaichana, K.L. Extending the resection beyond the contrast-enhancement for glioblastoma: Feasibility, efficacy, and outcomes. Br. J. Neurosurg. 2018 , 32 , 528–535. [ Google Scholar ] [ CrossRef ]
- Hamada, S.M.; Ahmed, H.A. Anatomical resection in glioblastoma: Extent of resection and its impact on duration of survival. Egypt J. Neurol. Psychiatry Neurosurg. 2016 , 53 , 135–145. [ Google Scholar ] [ CrossRef ]
- Li, Y.M.; Suki, D.; Hess, K.; Sawaya, R. The influence of maximum safe resection of glioblastoma on survival in 1229 patients: Can we do better than gross-total resection? J. Neurosurg. 2016 , 124 , 977–988. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Schneider, M.; Ilic, I.; Potthoff, A.L.; Hamed, M.; Schäfer, N.; Velten, M.; Güresir, E.; Herrlinger, U.; Borger, V.; Vatter, H.; et al. Safety metric profiling in surgery for temporal glioblastoma: Lobectomy as a supra-total resection regime preserves perioperative standard quality-rates. J. Neurooncol. 2020 , 149 , 455–461. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Figueroa, J.; Morell, A.; Bowory, V.; Shah, A.H.; Eichberg, D.; Buttrick, S.S.; Richardson, A.; Sarkiss, C.; Ivan, M.E.; Komotar, R.J. Minimally invasive keyhole temporal lobectomy approach for supramaximal glioma resection: A safety and feasibility study. J. Clin. Neurosci. 2020 , 72 , 57–62. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Merkow, R.P.; Bilimoria, K.Y.; Tomlinson, J.S.; Paruch, J.L.; Fleming, J.B.; Talamonti, M.S.; Ko, C.Y.; Bentrem, D.J. Postoperative complications reduce adjuvant chemotherapy use in resectable pancreatic cancer. Ann. Surg. 2014 , 260 , 372–377. [ Google Scholar ] [ CrossRef ]
- Weber, L.; Padevit, L.; Müller, T.; Velz, J.; Vasella, F.; Voglis, S.; Gramatzki, D.; Weller, M.; Regli, L.; Sarnthein, J.; et al. Association of perioperative adverse events with subsequent therapy and overall survival in patients with WHO grade III and IV gliomas. Front. Oncol. 2022 , 12 , 959072. [ Google Scholar ] [ CrossRef ]
- Rahman, M.; Abbatematteo, J.; De Leo, E.K.; Kubilis, P.S.; Vaziri, S.; Bova, F.; Sayour, E.; Mitchell, D.; Quinones-Hinojosa, A. The effects of new or worsened postoperative neurological deficits on survival of patients with glioblastoma. J. Neurosurg. 2017 , 127 , 123–131. [ Google Scholar ] [ CrossRef ]
- De la Garza-Ramos, R.; Kerezoudis, P.; Tamargo, R.J.; Brem, H.; Huang, J.; Bydon, M. Surgical complications following malignant brain tumor surgery: An analysis of 2002–2011 data. Clin. Neurol. Neurosurg. 2016 , 140 , 6–10. [ Google Scholar ] [ CrossRef ]
- Jackson, C.; Westphal, M.; Quinones-Hinojosa, A. Complications of glioma surgery. Handb. Clin. Neurol. 2016 , 134 , 201–208. [ Google Scholar ]
- McGirt, M.J.; Mukherjee, D.; Chaichana, K.L.; Than, K.D.; Weingart, J.D.; Quinones-Hinojosa, A. Association of surgically acquired motor and language deficits on overall survival after resection of glioblastoma multiforme. Neurosurgery 2009 , 65 , 463–469. [ Google Scholar ] [ CrossRef ]
- Sanai, N.; Berger, M.S. Glioma extent of resection and its impact on patient outcome. Neurosurgery 2008 , 62 , 753–766. [ Google Scholar ] [ CrossRef ]
- Incekara, F.; Koene, S.; Vincent, A.J.P.E.; van den Bent, M.J.; Smits, M. Association Between Supratotal Glioblastoma Resection and Patient Survival: A Systematic Review and Meta-Analysis. World Neurosurg. 2019 , 127 , 617–624. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Vivas-Buitrago, T.; Domingo, R.A.; Tripathi, S.; De Biase, G.; Brown, D.; Akinduro, O.O.; Ramos-Fresnedo, A.; Sabsevitz, D.S.; Bendok, B.R.; Sherman, W.; et al. Influence of supramarginal resection on survival outcomes after gross-total resection of IDH-wild-type glioblastoma. J. Neurosurg. 2021 , 136 , 1–8. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Sanai, N.; Berger, M.S. Surgical oncology for gliomas: The state of the art. Nat. Rev. Clin. Oncol. 2018 , 15 , 112–125. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Van Kessel, E.; Baumfalk, A.E.; van Zandvoort, M.J.E.; Robe, P.A.; Snijders, T.J. Tumor-related neurocognitive dysfunction in patients with diffuse glioma: A systematic review of neurocognitive functioning prior to anti-tumor treatment. J. Neurooncol. 2017 , 134 , 9–18. [ Google Scholar ] [ CrossRef ]
- Campanella, F.; Mondani, M.; Skrap, M.; Shallice, T. Semantic access dysphasia resulting from left temporal lobe tumors. Brain 2009 , 132 Pt 1 , 87–102. [ Google Scholar ] [ CrossRef ]
- You, G.; Sha, Z.; Jiang, T. Clinical Diagnosis and Perioperative Management of Glioma-Related Epilepsy. Front. Oncol. 2021 , 10 , 550353. [ Google Scholar ] [ CrossRef ]
- Borger, V.; Hamed, M.; Ilic, I.; Potthoff, A.L.; Racz, A.; Schäfer, N.; Güresir, E.; Surges, R.; Herrlinger, U.; Vatter, H.; et al. Seizure outcome in temporal glioblastoma surgery: Lobectomy as a supratotal resection regime outclasses conventional gross-total resection. J. Neurooncol. 2021 , 152 , 339–346. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Ung, T.H.; Ney, D.E.; Damek, D.; Rusthoven, C.G.; Youssef, A.S.; Lillehei, K.O.; Ormond, D.R. The Neurologic Assessment in Neuro-Oncology (NANO) Scale as an Assessment Tool for Survival in Patients with Primary Glioblastoma. Neurosurgery 2019 , 84 , 687–695. [ Google Scholar ] [ CrossRef ]
- Sawaya, R.; Hammoud, M.; Schoppa, D.; Hess, K.R.; Wu, S.Z.; Shi, W.M.; Wildrick, D.M. Neurosurgical outcomes in a modern series of 400 craniotomies for treatment of parenchymal tumors. Neurosurgery 1998 , 42 , 1044–1055; discussion 1055–1056. [ Google Scholar ] [ CrossRef ]
Share and Cite
Wach, J.; Vychopen, M.; Kühnapfel, A.; Seidel, C.; Güresir, E. A Systematic Review and Meta-Analysis of Supramarginal Resection versus Gross Total Resection in Glioblastoma: Can We Enhance Progression-Free Survival Time and Preserve Postoperative Safety? Cancers 2023 , 15 , 1772. https://doi.org/10.3390/cancers15061772
Wach J, Vychopen M, Kühnapfel A, Seidel C, Güresir E. A Systematic Review and Meta-Analysis of Supramarginal Resection versus Gross Total Resection in Glioblastoma: Can We Enhance Progression-Free Survival Time and Preserve Postoperative Safety? Cancers . 2023; 15(6):1772. https://doi.org/10.3390/cancers15061772
Wach, Johannes, Martin Vychopen, Andreas Kühnapfel, Clemens Seidel, and Erdem Güresir. 2023. "A Systematic Review and Meta-Analysis of Supramarginal Resection versus Gross Total Resection in Glioblastoma: Can We Enhance Progression-Free Survival Time and Preserve Postoperative Safety?" Cancers 15, no. 6: 1772. https://doi.org/10.3390/cancers15061772
Article Metrics
Article access statistics, supplementary material.
ZIP-Document (ZIP, 224 KiB)
Further Information
Mdpi initiatives, follow mdpi.
Subscribe to receive issue release notifications and newsletters from MDPI journals
Cochrane Community
Living systematic reviews.
Visit Project Transform
What is a living systematic review?
We define an LSR as a systematic review which is continually updated, incorporating relevant new evidence as it becomes available .
Practically, this means that LSRs:
- Are underpinned by continual, active monitoring of the evidence (i.e. monthly searches)
- Immediately include any new important evidence (meaning data, studies or information) that is identified
- Are supported by up-to-date communication about the status of the review, and any new evidence being incorporated
While core review methods are not fundamentally different to other Cochrane Reviews, an LSR should additionally include explicit, transparent and predefined decisions on:
- How frequently new evidence is sought and screened
- When and how new evidence is incorporated into the review
Why living systematic reviews?
Living systematic reviews (LSRs) provide a new approach to support the ongoing efforts of Cochrane and others to produce evidence that is both trustworthy and current.
The concept of living evidence synthesis and related outputs, such as living guidelines, are of increasing interest to evidence producers, decision makers, guideline developers, funders and publishers, as a way to seamlessly connect evidence and practice.
The possibility of a scaled-up living evidence approach has only recently been within reach, due to a number of technological and data-related innovations, such as online platforms, linked data, and machine learning. Concurrently, research groups are embracing larger collaborations, open and shared data, and the growth of the citizen science movement, opening up the possibility of communities with a common interest maintaining high value datasets and associated LSR.
Cochrane LSR planning and methods support
Cochrane authors and review groups planning to undertake a living systematic review are encouraged to contact the Cochrane LSR team at [email protected] for support with developing their protocol. We're here to help you work through the factors to consider when planning and conducting an LSR.
Living systematic review pilots
Project Transform evaluated LSRs as part of the Production Models component with the following groups:
- Cochrane Gynaecological, Neuro-oncology and Orphan Cancers
- Cochrane Heart
- Cochrane Acute Respiratory Infections
An evaluation was undertaken alongside the pilots to understand the feasibility of LSRs, implications for the people and processes involved and identify opportunities to refine the LSR model before scaling up. The final evaluation report is available to download here .
Following completion of the pilot, additional Cochrane teams are conducting LSRs, with the aim of at least one published LSR for each Network.
LSR guidance for production and publication
The purpose of this document is to outline the methods related to production and publication for LSRs published on The Cochrane Library. This document is primarily designed to be a practical guidance document for the authors, Review Groups and Central Editorial Unit staff involved in Cochrane LSRs. The approach described is the revised guidance based on our review of current literature relevant to LSRs and consultation with a range of stakeholders, including the LSR Network, which includes members within Cochrane and beyond.
LSR Webinars
- Tech enablers for living evidence - Covidence & MAGICapp [slides] and video (18 September 2019)
- Publishing living evidence [slides] and video (3 July 2019)
- Tech enablers for living evidence – Screen4Me, RCT-classifier, RevMan Replicant, Systematic Review Accelerator (20 May 2019)
- Living Network Meta-Analysis [slides] and video (21 March 2019)
- Getting better all the time: Considerations and approaches for LSR searching [slides] and video (July 2018)
- Practical experiences of piloting an LSR (May 2018)
- Living guideline recommendations: key challenges, questions and progress (April 2018)
- Introducing living systematic reviews (March 2017)
Other LSR Resources
- Presentations from Living Evidence Network meeting , Cochrane Colloquium, Edinburgh (15 September 2018)
- Presentation from HTAi 2018 Advanced Information Retrieval on the Edge: LSRs (2 June 2018)
- Presentations from Special Session: From living systematic reviews to living recommendations , Global Evidence Summit, Cape Town (14 Sep 2017)
- Presentations from LSR Network meeting , Global Evidence Summit, Cape Town (12 Sep 2017)
- Report from Cochrane Canada Symposium, LSR Workshop (13-14 May 2017)
- Presentations from Cochrane Canada Symposium, LSR Workshop (13-14 May 2017)
- Living systematic review methods symposium slides , Cochrane Colloquium, Seoul (26 October 2016)
- Living Systematic Reviews: towards real-time evidence for healthcare decision making (12 May 2016)
- Publically available Mendeley Library with relevant references
- Useful LSR references and resources
- Expert Searching: Living Evidence and Living Systematic Reviews: What You Need to Know (17 December 2018)
- Living proof of living evidence (25 June 2018)
- From living systematic reviews to living recommendations (25 June 2018)
- We've done it again! Fruit & veg LSR re-published in record time (22 May 2018)
- Living systematic review re-published in record time (14 February 2018)
- Living systematic review series published in Journal of Clinical Epidemiology (12 September 2017)
- First two living systematic reviews now live on Cochrane Library! (8 September 2017)
- LSRs: On the road with Annie and Julian (3 July 2017)
- Living systematic reviews are going live (28 June 2017)
Living Evidence Network
The Living Evidence Network is an informal network that was launched in February 2016, with members including Cochrane and non-Cochrane researchers, policymakers and guideline developers.
The Living Evidence Network aims to:
- Share experiences, information and resources
- Further the thinking on the living evidence concept and methods
- Develop and refine approaches for Cochrane LSRs
You can download the Living Evidence Network governance structure document here . If you would like to join the Living Evidence Network, please email [email protected]
Cochrane UK
What are systematic reviews, "what are systematic reviews".
If you’re a Cochrane contributor and have ever attempted to explain Cochrane’s work to someone, chances are you’ve tried to answer this question. And if you’re reading this because you’re new to Cochrane and the work we do, you may be wondering about this too.
Thanks to a team of creative colleagues from Cochrane Consumers and Communication , we’re pleased to share a video resource which answers this question clearly and simply for people who may not be familiar with the concept of systematic reviews: what they are, how researchers prepare them, and why they’re an important part of making informed decisions about health - for everyone. You can find this video on Cochrane’s YouTube channel , and we hope you’ll share and spread the word about the importance of evidence!
- Interlibrary Loan
Ask an Expert
Ask an expert about access to resources, publishing, grants, and more.
MD Anderson faculty and staff can also request a one-on-one consultation with a librarian or scientific editor.
- Library Calendar
Log in to the Library's remote access system using your MyID account.
- Library Home
- Research Guides
- Systematic Reviews
Cochrane Systematic Reviews
Systematic reviews: cochrane systematic reviews.
- Introduction
- Step-by-Step
- Standards and Guidelines
- Designing the Protocol
- Selecting Studies
- Appraising Studies
Data Extraction
- Writing a Systematic Review
- Meta-analyses
- Evaluating a Systematic Review
- Updating a Systematic Review
- Get more help
- Related Review Types
About Cochrane
Proposing a Cochrane Systematic Review
Proposing a New Cochrane Review : you must submit a protocol to Cochrane for peer review and acceptance.
Cochrane Training
Cochrane Training : recorded and live webinars, events, guides, and handbooks. Many of the introductory resources are free while many of the advanced ones are restricted to Cochrane members.
Finding Cochrane Systematic Reviews
Cochrane guidelines, software for cochrane systematic reviews.
Cochrane's preferred core software is:
- Review Manager (RevMan) - software for preparing and maintaining Cochrane Reviews: protocols, manuscripts, characteristics of studies, comparison tables, study data, meta-analysis. (web and desktop versions available)
- Covidence - a web-based tool which helps you screen records, assess risk-of-bias, and extract data. Free for authors of Cochrane Reviews.
- GRADEpro GDT - an online tool to create a Summary of Findings (SoF) table
- EPPI-Reviewer – a web-based tool which helps you with all stages of the systematic review process: reference management, screening, risk of bias assessment, data extraction and synthesis. Free for authors of Cochrane Reviews.
- Archie - Cochrane's database for managing contacts and documents and delivering them for publication.
Study Selection
The Cochrane Handbook allows also allows the use of multiple applications with machine learning or text-mining to semi-automate study selection. After you screen a certain number of citations, the software prioritizes the rest of the citations in order of estimated relevance. You must still review each record but it helps you identify studies that are most likely to be relevant earlier in the screening process:
- Covidence – a web-based tool which helps you screen records, assess risk-of-bias, and extract data. Free for authors of Cochrane Reviews.
- DistillerSR – a web-based tool which helps you screen records and extract data. It has a number of management features to track progress, assess interrater reliability and export data for further analysis. Reduced pricing for Cochrane authors.
- Rayyan – a free web-based application for collaborative citation screening.
The Cochrane Handbook allows the use of many applications that collect, store, and share data:
- Systematic Review Data Repository (SRDR) - a web-based tool for data extraction and management
- << Previous: Updating a Systematic Review
- Next: Get more help >>
- Last Updated: Jan 27, 2023 12:47 PM
- URL: https://mdanderson.libguides.com/systematicreviews
More From Forbes
Cochrane says review does not show that ‘face masks don’t work’ against covid-19.
- Share to Facebook
- Share to Twitter
- Share to Linkedin
The Cochrane review did not include studies that have shown that face masks can reduce transmission ... [+] of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Pictured here Olivia Palermo wears a face mask while walking in New York City. (Photo by Jose Perez/Bauer-Griffin/GC Images)
You know that recently published Cochrane review that some have claimed as evidence that face masks don’t work? Well, have some of these claims been a bit, shall we say, too cocky and not Cochrane -y enough? Well, in a statement issued on March 10 by the Cochrane Library , Karla Soares-Weiser, MD, PhD, MSc, Editor-in-Chief of the Cochrane Library , emphasized, “Many commentators have claimed that a recently-updated Cochrane Review shows that ‘masks don't work,’ which is an inaccurate and misleading interpretation.” Yikes.
Soares-Weiser didn’t specify in the statement who exactly those “commentators” were. But she could have been referring to Brett Stephens, who wrote a February 21, 2023, Opinion piece for The New York Times entitled, “ The Mask Mandates Did Nothing. Will Any Lessons Be Learned .” Or maybe she was referring to Tom Jefferson, the Cochrane review’s first author, a Senior Associate Tutor at the Department for Continuing Education at the University of Oxford, and notably an Author for The Brownstone Institute . Stephens quoted Jefferson as saying the following about face masks: “There is just no evidence that they make any difference. Full stop.” That’s presumably based on the systematic review of face mask studies that Jefferson and his team conducted and published in the Cochrane Library on January 30. Well, looks like such statements didn’t really accurately represent what the systematic review found, full stop. And now the Cochrane Library is essentially saying, full stop it with all such misleading interpretations.
In fact, Soares-Weiser’s statement went on to delineate what the systematic review can and can’t do and what claims ultimately may be doo-do0: “It would be accurate to say that the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive. Given the limitations in the primary evidence, the review is not able to address the question of whether mask-wearing itself reduces people's risk of contracting or spreading respiratory viruses.”
A number of scientists and public health experts have already emphasized various key flaws in the systematic review conducted by Jefferson and his team. For example, Lucky Tran, PhD, an organizer for the March for Science and a science communicator at Columbia University, wrote in an Opinion piece for The Guardian entitled, “ Don’t believe those who claim science proves masks don’t work ,” that “The analysis [by Jefferson and his team] is flawed because it compares apples to oranges.” The systematic review didn’t compare actual apples with actual oranges, which has already been done before and has shown that you should really peel oranges before eating them. Rather, what Tran meant is that “The paper mixes together studies that were conducted in different environments with different transmission risks. It also combines studies where masks were worn part of the time with studies where masks are worn all the time. And it blends studies that looked at Covid-19 with studies that looked at influenza.”
Lucky Tran, PhD, (not pictured here) wrote in an Opinion piece for The Guardian entitled, “Don’t ... [+] believe those who claim science proves masks don’t work,” that “The analysis [by Jefferson and his team] is flawed because it compares apples to oranges.” (Photo: Getty)
Best Travel Insurance Companies
Best covid-19 travel insurance plans.
That’s a whole lot of mixing going on and makes you wonder why the authors of the review didn’t instead clearly separate out the studies. After all, no one wants apple and orange pie and as Tran emphasized in his Opinion piece, “If apples work and oranges don’t, but your analysis mixes them together, you may come to the false conclusion that apples don’t work. Out of the 78 papers analyzed in the review, only two actually studied masking during the Covid-19 pandemic. And both of those found that masks did protect wearers from Covid-19.”
Yeah, mixing Covid-19 studies and flu studies would be sort of like mixing orange spray tan with an Apple computer or a Tim Apple computer. Things could get very messy. Remember Covid-19 ain’t the flu—it just ain’t. While both are respiratory viruses, the flu virus can’t float around in the air nearly as well as the Covid-19 coronavirus, otherwise known as the severe acute respiratory syndrome coronavirus 2 (SARS-CoVC-2), can. Early on in 2020, authorities thought that the SARS-CoV-2 behaved more like the flu virus in the air. The thinking was that like the flu virus the SARS-CoV-2 was carried primarily by large respiratory droplets that would travel no further than six feet due to that thing called gravity. That was the rationale for the initial stay six feet (or one Gosling because Ryan Gosling is about six feet tall) apart from one another.
However, a pandemic game changer was when studies revealed that the SARS-CoV-2 could actually travel in smaller and lighter respiratory droplets that could hang in the air for much longer periods of time and could float for more than six feet. This game-changing finding that the SARS-CoV-2 can go all aerosol is what prompted the change in face mask guidance in the Spring of 2020. Therefore, including so many studies focused on the flu virus rather than the SARS-CoV-2, in the words of Tran, “drowned out” the SARS-CoV-2 “by the greater number of studies on influenza included, where the benefit of masking is harder to detect because it’s a far less contagious virus than Covid-19.”
Tran brought in another piece of fruit into the mix by calling the Cochrane review “a classic example of cherry-picking, where biased groups highlight a subset of data that support their position, while ignoring the larger pool of evidence that disagrees with them.” Tran went on to point out that “Many direct studies in labs show that high-quality masks significantly reduce the number of viral particles mask-wearers inhale and emit, but these are intentionally omitted in the arguments of anti-maskers.” I’ve also described for Forbes other scientific studies that have supported the use of face masks, studies that were omitted by Jefferson’s Cochrane review. Moreover, if face masks don’t work, why did the spread of respiratory viruses plummet during the Winters of 2020-2021 and 2021-2022 when, guess what, face masks were being used more widely? Life may be a bowl of cherries, but cherry picking only certain studies to support a particular viewpoint would be the “pits.”
Hmm, why might Jefferson and his team have “cherry picked” certain studies? Could a certain A-word have been involved? No, not that A-word. Rather, the operative A-word here may be “agenda.” Gavin Yamey, MD, MPH, MA, the Hymowitz Professor of Global Health and a Professor of Public Policy at Duke University, tweeted out that the Jefferson works with the Brownstone Institute and that “Brownstone has always been an extremely anti-mask, anti-vaxx right wing think tank”:
From Twitter
In fact, a closer look at the Cochrane Library publication shows that even the authors admitted that their review is very limited. This was emphasized by Soares-Weiser’s statement that read, “The review authors are clear on the limitations in the abstract: ‘The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions.’” Soares-Weiser’s statement continued by saying, “Adherence in this context refers to the number of people who actually wore the provided masks when encouraged to do so as part of the intervention. For example, in the most heavily-weighted trial of interventions to promote community mask wearing, 42.3% of people in the intervention arm wore masks compared to 13.3% of those in the control arm.” Less than 50% adherence is not great adherence. So when less than half of people are adhering to an intervention, is the intervention really in place?
Nevertheless, these stated limitations didn’t full stop Jefferson from saying that “full stop” quote in an interview . They haven’t stopped various political leaders, TV personalities, and anonymous social media accounts either from further amplifying Jefferson’s Cochrane publication to keep pushing the narrative that face masks somehow don’t work. Will these same people be as zealous at amplifying Soares-Weiser’s statement about the limitations of Jefferson’s Cochrane publication? Don’t hold your breath for this to happen.
One challenge throughout the Covid-19 pandemic has been people employing the political tactic of using vague language to suggest something without coming right out and saying it. This is a bit like saying, “I love your new outfit, I wish I had one like that but I think I’m too skinny for that style.” An even closer look at Jefferson’s Cochrane review reveals some questionable language. As Soares-Weiser’s statement mentioned, “The original Plain Language Summary for this review stated that ‘We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.’” Such a statement may sound a little more passive-aggressive than science-y. Indeed, Soares-Weiser’s statement admitted that “This wording was open to misinterpretation, for which we apologize.” She added, “While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset.” Soares-Weiser indicated that “We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses.”
The effectiveness of a face mask does depend on the type of mask that's being worn, how it's being ... [+] worn, and how many people around you are wearing the face mask. (Photo by Oli Scarff/Getty Images)
This is reminder that you should never use a single scientific publication to draw a conclusion. Instead, it’s important to look at what studies say as a whole. Some have claimed that publications in the Cochrane Library should always serve as “gold standards.” For example, Vinayak Prasad, MD, MPH, a Professor of Epidemiology & Biostatistics at the University of California, San Francisco, whose work prior to the pandemic has been mainly in oncology and not infectious diseases, called Jefferson’s Cochrane review a “gold standard for mask studies” in a tweet . A Cochrane review may serve as a “gold standard” to some degree when evaluating a bunch of randomized controlled trials (RCTs) that evaluate a single medication under very controlled and similar circumstances such as a cancer medication that’s being administered and monitored by doctors from a hospital or clinic.
But such claims about Cochrane reviews and RCTs are more like “fool’s gold” when it comes to other types of scientific questions. It’s important to remember that face mask use is a population-based intervention and not simply an individual-based intervention. The effectiveness of a face mask does depend not only on what kind of face mask you are wearing and how you wear it but also how many people around you are wearing face masks.
It’s time to face mask the facts. Many people have politicized the heck out of face masks to the point that they are not even acknowledging what scientific studies prior to the Cochrane review have shown. Instead, they’ve been doing a lot of cherry-picking, selecting bits and pieces of things here and there to support their narrative. So, when someone keeps pointing to just a single study to prove his or her point rather the entire body of scientific evidence, be skeptical, be very skeptical. Heck, as has been demonstrated, even a Cochrane Library publication can be used in a cockeyed manner.
- Editorial Standards
- Reprints & Permissions
IMAGES
VIDEO
COMMENTS
a systematic search that attempts to identify all studies that would meet the eligibility criteria; an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and a systematic presentation, and synthesis, of the characteristics and findings of the included studies.
What are systematic reviews? Watch on Cochrane evidence, including our systematic reviews, provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence and what Cochrane does. About Cochrane Watch on
A Cochrane Review is a systematic review of research in health care and health policy that is published in the Cochrane Database of Systematic Reviews. Types of Cochrane Review Intervention reviews assess the effectiveness/safety of a treatment, vaccine, device, preventative measure, procedure or policy.
The primary reason for conducting Cochrane Overviews is that using systematic reviews as the unit of searching, inclusion, and data analysis allows authors to address research questions that are broader in scope than those examined in individual systematic reviews (also see Section V.2.3 ).
Cochrane Reviews use primary research to generate new knowledge about the effects of an intervention (or interventions) used in clinical, public health or policy settings. They aim to provide users with a balanced summary of the potential benefits and harms of interventions and give an indication of how certain they can be of the findings.
Systematic reviews carried out by Cochrane Collaboration (an international network of researchers belonging to this independent, not-for-profit organization) are recognized worldwide as the highest standard in evidence-based healthcare. The main reason is that Cochrane reviews follow a common and specific methodology to limit bias and random error.
A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review
The Cochrane Library is an electronic collection of databases published on the internet and also available on CD-Rom. It is updated quarterly in an effort to add to and keep the information current. The Library is made up of a number of parts. The Cochrane Database of Systematic Reviews (CDSR) contains the published Cochrane reviews and protocols.
Each Cochrane Review is a systematic review that has been prepared and supervised by a Cochrane Review Group (editorial team). It attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question.
What are systematic reviews? Watch on Cochrane evidence provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence, including systematic reviews, and other activities of Cochrane. About Cochrane Watch on
Cochrane Rapid Review (RR) Definition Definition of a Cochrane Rapid Review (RR) We recommend the following definition: "A rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting specific methods to produce evidence for stakeholders in a r ...
Multicentric prospective randomized trials with a standardized definition of SMR and analysis of neurologic functioning and health-related quality of life are justified and needed to improve the level of evidence. ... and the Cochrane Handbook for Systematic Reviews of Interventions version 6.3 . The study was registered in the ...
Living systematic reviews (LSRs) provide a new approach to support the ongoing efforts of Cochrane and others to produce evidence that is both trustworthy and current. The concept of living evidence synthesis and related outputs, such as living guidelines, are of increasing interest to evidence producers, decision makers, guideline developers ...
What are systematic reviews? Watch on If you're a Cochrane contributor and have ever attempted to explain Cochrane's work to someone, chances are you've tried to answer this question. And if you're reading this because you're new to Cochrane and the work we do, you may be wondering about this too.
Cochrane Handbook for Systematic Reviews of Interventions . The manual for authors of Cochrane systematic reviews. It provides detailed information on planning a review, searching, selecting studies, data collection, risk of bias assessment, statistical analysis, GRADE and interpreting results, as well as more specialized topics (non-randomized studies, adverse effects, economics, patient ...
Karla Soares-Weiser, Editor-in-Chief of the Cochrane Library, emphasized, "Many commentators have claimed that a recently-updated Cochrane Review shows that 'masks don't work,' which is an ...