Speed of updating online evidence based point of care summaries: prospective cohort analysis
BMJ 2011; 343 doi: https://doi.org/10.1136/bmj.d5856 (Published 23 September 2011) Cite this as: BMJ 2011;343:d5856
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Point of care tools differ in their updating speeds (1) and this also
varies between different categories of evidence based resources. Clinical
Evidence does not make treatment recommendations and is not a point of
care tool but its comparison against these tools helpfully shows that
different trade off's between expediency and rigour are made by different
organisations.
Clinical Evidence examines both systematic reviews and individual
clinical trials. It is designed as a rigorous evidence resource and
deliberately takes a deeper look at the evidence than if it only focussed
on systematic reviews, and we believe this is reflected by its high
quality score in the authors earlier study(2). We examine individual
randomised trials to capture important new evidence and our peer review
process ensures that this is weighed and considered carefully by clinical
experts. Our experience is that this results in better quality assurance
than if we relied on non specialist editors.
In 2009 we introduced Best Practice, which unfortunately meant that
it could not be included in this study. As our website explains, "Best
Practice incorporates much of the research evidence from Clinical Evidence
and adds expert opinion and guidelines to cover diagnosis, prognosis,
treatment and prevention. As such it is specifically designed as a
practical tool to support decision making at the point of care." Best
Practice was awarded the highest quality score in an evaluation by the
Norwegian Electronic Library for Health (3), and we happily encourage it's
evaluation in future point of care studies.
Of course, evidence producers must balance expediency against rigour.
But different strategies are emerging and many key questions remain
unanswered. What is the 'need for speed'? How quick is too quick and how
long is too long? Are priority based approaches better than time based
ones? Should users of evidence based point of care tools expect publishers
to be transparent about the quality of their products? And is it finally
time we hade a CONSORT-type framework for clinical decision support tools?
To explore these and many other questions we are planning to make the
quality of decision support a key theme at next year's Evidence 2012
conference. We'll be inviting publishers, guideline developers,
researchers and clinicians to attend and contribute to this long overdue
and topical debate.
(1) Banzi et al. BMJ 2011; 343:d5856 doi: 10.1136/bmj.d5856
(2) Banzi R, Liberati A, Moschetti I, Tagliabue L, Moja L. A review
of online evidence-based practice point-of-care information summary
providers. J Med Internet Res2010;12:e26
(3) K Tjensvoll. National licensing for the Norwegian Electronic
Health Library. Serials 2011; 24(1): 52-55.
http://uksg.metapress.com/media/3aa8lu4d7p6yumff5p6j/contributions/k/2/0...
(accessed 30 September 2011)
Competing interests: RM is Editor in Chief of Clinical Evidence and Best Practice and Clinical Director of the BMJ Evidence Centre.
Review of evidence based reference incomplete
Dear Editor,
We read with great interest the article by Banzi and colleagues
describing how quickly online, evidence-based references cite important
systematic reviews (1). However, we were disappointed that the article
failed to include Essential Evidence Plus (EE+;
http://www.essentialevidenceplus.com). EE+ is an evidence-based point of
care information summary covering approximately 800 disease topics, with
evidence grading for all key recommendations, over 4000 critical
appraisals, several thousand clinical calculators, a process for
continuously updating content, and daily evidence alerts to subscribers.
It clearly meets their inclusion criteria as a continuously updated online
evidence-based 'point of care information summary'.
Banzi considers any systematic review cited by one of two
surveillance services or any Cochrane review flagged as "changed" to be
important information that should be cited by a point of care information
summary tool. We use a different, more explicit filter to guide our
updating process. For 17 years we have been doing continuous surveillance
of the literature for articles that meet our criteria for validity and
relevance: well designed studies that report improved patient-oriented
outcomes and have the potential to change clinical practice. These
criteria are reproducible and have good inter-rater agreement (2). Our
standards for relevance are stricter than those Banzi proposes, which
reduces work for busy clinicians by limiting the amount of new information
they have to review and absorb.
Of the 128 systematic reviews identified by Banzi and colleagues, 42
are cited by one or more topics in EE+, all of the Cochrane review
abstracts are included in the EE+ references, and many of the systematic
reviews were reported to our subscribers as part of our "Daily InfoPOEM"
evidence alerting service. Of the studies not cited or included in the
alerting service, almost all were reviewed within 2 months of publication
by our editors, but were not cited because they did not report improved
patient oriented outcomes or would not change practice for a substantial
number of physicians. Examples of systematic reviews identified as
'important information that should be cited by a point of care information
summary tool' that did not meet our criteria for relevance included
systematic reviews on the effectiveness of prayer on ill health (no
previous evidence of efficacy, therefore no change in practice) (3), the
effectiveness of continuing education (not relevant to clinical
practice)(4), and confirmation of effectiveness of CPAP for obstructive
sleep apnea (consistent with usual care, does not change practice). (5) We
use a "higher bar" for selecting evidence relevant to clinical practice,
to assure that our point of care information tool is truly a concise
information summary for use at the bedside.
Sincerely,
Mark Ebell MD, MS
Editor-in-chief, Essential Evidence Plus
Maria Kouimtzi, MRPharmS, PhD
Editor Director Pharmacy and Informatics, Publisher of Essential Evidence
Plus, Wiley-Blackwell
Charles Young MRCP
VP & Publishing Director, Wiley-Blackwell
References
1. Banzi R, Cinquini M, Liberati A, et al. Speed of updating online
evidence based point of care summaries: prospective cohort analysis. BMJ
2011; 343:d5856 doi: 10.1136/bmj.d5856.
2. Ebell MH, Barry HC, Slawson DC, Shaughnessy AF. Finding POEMs in the
medical literature. J Fam Pract. 1999 May;48(5):350-5.
3. Roberts Leanne, Ahmed Irshad, Hall Steve, Davison Andrew. Intercessory
prayer for the alleviation of ill health. Cochrane Database of Systematic
Reviews: Reviews 2009 Issue 2 Cochrane Library- Conclusion Changed
4. Forsetlund Louise, Bj?rndal Arild, Rashidian Arash, Jamtvedt Gro,
O'Brien Mary Ann, Wolf Fredric, Davis Dave, Odgaard-Jensen Jan, Oxman
Andrew D. Continuing education meetings and workshops: effects on
professional practice and health care outcomes. Cochrane Database of
Systematic Reviews: Reviews 2009 Issue 2 Cochrane Library.
5. McDaid C, Griffin S, Weatherly H, et al. Continuous positive airway
pressure devices for the treatment of obstructive sleep apnoea-hypopnoea
syndrome: a systematic review and economic analysis. Health Technol Assess
2009;13(4);1-119.
Competing interests: The authors are the editor, editorial director, and VP Publishing for Essential Evidence Plus from Wiley-Blackwell INc.