Why certain systematic reviews reach uncertain conclusions
BMJ 2003; 326 doi: https://doi.org/10.1136/bmj.326.7392.756 (Published 05 April 2003) Cite this as: BMJ 2003;326:756
- Mark Petticrew, associate director (mark@msoc.mrc.gla.ac.uk)
- MRC Social and Public Health Sciences Unit, University of Glasgow, Glasgow G12 8RZ
Research syntheses are essential for putting studies in their proper scientific context and are increasingly common in public health, education, crime, and social welfare. A key criticism of systematic reviews, however, is that they are often unable to provide specific guidance on effective (or even ineffective) interventions; instead, they often conclude that little evidence exists to allow the question to be answered. This problem has been recognised in reviews of healthcare interventions,1 and the electronic journal Bandolier recently lamented the absence of systematic reviews containing a solid take home message.2 However, the problem is even more common in reviews of social and public health interventions, and this paper explains why.
Summary points
Systematic reviews are often criticised for being unable to provide specific guidance
This is often because the primary studies that they include contain few outcome evaluations
A “stainless steel” law of systematic reviews may also be operating—namely, the more rigorous the review, the less evidence there will be that the intervention is effective
Narrative review methods and narrative and meta-analytic approaches to reviewing observational data need to be improved
Uncertainty will often remain, but systematic reviews help us to acknowledge this and to map the areas of doubt
Sound systematic reviews may not guide practice
In public health there are few trials to review and indeed few other types of outcome assessment.3 Unsurprisingly, research users often regard reviews of such a limited evidence base as unhelpful and find their conclusions confusing and frustrating.4 This is ironic, given that systematic reviews are intended (among other things) to reduce uncertainty (box 1). Systematic reviews are certainly capable of doing this, and …
Log in
Log in using your username and password
Log in through your institution
Subscribe from £184 *
Subscribe and get access to all BMJ articles, and much more.
* For online subscription
Access this article for 1 day for:
£50 / $60/ €56 (excludes VAT)
You can download a PDF version for your personal record.