The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials
BMJ 2011; 343 doi: https://doi.org/10.1136/bmj.d5928 (Published 18 October 2011) Cite this as: BMJ 2011;343:d5928
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
To the Editor: Randomised controlled trials (RCTs) are considered to
be the gold standard of clinical research and the building blocks of most
systematic reviews and practice guidelines. Yet, the recently published
recommended criteria for assessing risk of bias in RCTs by the Cochrane
Collaboration are almost exclusively concerned with internal
validity.<1> Although the recommended standards are both timely and
welcomed, the perennial problem with poor uptake of clinical practice
guidelines, especially in primary care settings, is an on-going neglect of
the external validity or applicability in the design and conduct of
randomized controlled trials (RCTs). Some of the frequently identified
threats to external validity include narrow inclusion criteria, use of run
-in periods, and length of follow-up or clinical relevance of the primary
end-points used. <2,3,4>
Although, as stated by the authors, applicability is less relevant
without internal validity, the opposite is equally true. It is
disappointing to see that the external validity, once gain, was placed on
the back burner. Internal and external validity are equally important, and
both ought to be maximized in order to generate the evidence that is both
valid and applicable. This is not a zero-sum game.
References:
1. Higgins JPT, Altman DG, Peter C G?tzsche PC, et al. The Cochrane
Collaboration's tool for assessing risk of bias in randomised trials. BMJ
343:d5928. 2011
2. Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L.
Randomized controlled trials: do they have external validity for patients
with multiple comorbidities? Ann Fam Med 4 104-8. 2006.
3. Rothwell PM. External validity of randomised controlled trials:
"To whom do the results of this trial apply?" Lancet 365 82-93. 2005.
4. Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more
translation of health promotion research to practice? Rethinking the
efficacy-to-effectiveness transition. Am J Public Health 93, 1261-1267.
2003.
Competing interests: No competing interests
Risk of bias assessment should be taught to all junior doctors
Dear Editor,
I read with interest the article on the Cochrane Collaboration's tool
for assessing risk of bias by Higgins et al. I am an academic foundation
year two doctor working with a Cochrane review group based at the
University of York. My role within the team has been to help update
Cochrane reviews, adding new studies identified since the review was last
published, and to make sure the review meets the current Cochrane
Collaboration standards. One of the main aspects of this role was to
complete the new risk of bias assessment for all the studies in the review
update comprising those included before the development of the new tool.
The following is a brief summary of my experiences of assessing risk of
bias using the Cochrane Collaboration tool.
Initially, the task of updating Cochrane reviews was a daunting one
and I had a lot to learn. I found the Cochrane Handbook1 very helpful and
was well supported by the excellent team at the editorial base. For me,
the main positive aspect of the risk of bias tool is its ease of use and
transparency. The Cochrane Handbook1 gives very clear instructions on how
to use the tool and within RevMan (Cochrane's software package) the risk
of bias tables are generated and summary figures produced automatically.
The process of completing the risk of bias tables is also straightforward,
as every decision made by the review authors must be supported by a quote
from the study text. This requirement makes the decision making process
transparent and allows the reader of the review to see how the judgement
of bias was supported. Finally, the summary diagrams give a clear overview
of, not only each individual study's risk of bias, but of the review as a
whole.
I feel I have learnt a great deal in my short time working with the
review group. I quickly learnt about the different types of bias, why they
are important and how to detect them within the included studies of a
Cochrane review. I have acquired a valuable skill, now being able to
thoroughly appraise studies and assess their potential biases. I have a
better understanding of how studies should be designed and importantly
what information authors should report in the study publication to enable
readers to detect any potential biases and be able to form a judgement on
the evidence presented. I feel these skills are essential for all doctors
in order to assess the medical literature upon which their clinical
decision making is based.
1. Higgins JPT, Green S, eds. Cochrane handbook for systematic
reviews of interventions. Wiley, 2008.
Competing interests: No competing interests