Conducting risk of bias assessment, sometimes called quality assessment, is a defining feature of the systematic review process that elevates the methodological rigor and transparency of reported results. To do so, you will need a tool that fits the study designs included in your systematic review and establish a method to conduct the assessment in a dual-blind manner.
3.6.1 Systematically assess the risk of bias, using predefined criteria
3.6.2 Assess the relevance of the study’s populations, interventions, and outcome measures
3.6.3 Assess the fidelity of the implementation of interventions
Chapter 8: Assessing risk of bias in included studies
This is a chapter from "Methods Guide for Effectiveness and Comparative Effectiveness Reviews."
From the AHRQ - "Methods Guide for Effectiveness and Comparative Effectiveness Reviews"
A searchable spreadsheet to help find an appropriate quality assessment or risk of bias tool for the study types in your systematic review.
Download the file for full functionality.
Common risk of bias tools:
Randomized Controlled Trials
Non-randomized (observational) studies or cohorts of Interventions
Non-randomized (observational) studies or cohorts of Exposures other than interventions, including environmental and occupational exposures
Case control or cohort studies
Diagnostic accuracy
Cross-sectional/prevalence studies
Mixed-methods SRs with a variety of study types including both qualitative & quantitative,
Case series
Case reports
Oxford-Centre for Evidence Based Medicine
Am Fam Physician. 2004 Feb 1;69(3):548-56. PubMed PMID: 14971837
GRADE and Confidence Assessment
The recent PRISMA 2020 reporting guidelines now request an assessment of the certainty of the evidence. This can be tied into risk of bias through some tools, such as GRADE.
"GRADE (Grading of Recommendations, Assessment, Development and Evaluations) is a transparent framework for developing and presenting summaries of evidence and provides a systematic approach for making clinical practice recommendations.[1-3] It is the most widely adopted tool for grading the quality of evidence and for making recommendations with over 100 organizations worldwide officially endorsing GRADE." - BMJ Clinical Evidence (2015)
While GRADE does consider risk of bias, it considers the body of evidence at the outcome level, not the study level. Thus, a risk of bias tool is still needed when conducting a systematic review.
The Grading of Recommendations Assessment, Development and Evaluation (short GRADE) Working Group began in the year 2000 as an informal collaboration of people with an interest in addressing the shortcomings of present grading systems in health care. The working group has developed a common, sensible and transparent approach to grading quality of evidence and strength of recommendations. Many international organizations have provided input into the development of the approach and have started using it.
The GRADE handbook describes the process of rating the quality of the best available evidence and developing health care recommendations following the approach proposed by the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group
Confidence in the effect estimate
The 4 levels of Certainty in effect estimates / Quality of evidence