Evidence debased medicine

Our current evidence base on the benefits and harms of many treatments contains incomplete and questionable evidence. This week Elizabeth Loder and Fiona Godlee call for the record to be set straight and announce a BMJ theme issue in late 2011 for research that analyses uncovered evidence (doi:10.1136/bmj.c5641). The aim is to restore trust in the evidence base, not to point fingers.

This is an early call, but such studies take time because they often depend on freedom of information requests and protracted negotiations with companies. Dirk Eyding, Beate Wieseler, and colleagues from the German Institute for Quality and Efficiency in Health Care (IQWiG) show how it’s done.

Their health technology assessment of the antidepressant reboxetine began in the usual way last year with an extensive literature search for primary and secondary studies (doi:10.1136/bmj.c4737). They found a yawning gap: around 4600 people had taken part in trials of reboxetine, yet adequate data on outcomes had been published for only 1600 or so of the participants (doi:10.1136/bmj.c4942). Over the next seven months the institute issued its preliminary conclusion that no benefit of reboxetine could therefore be proved; Pfizer (the drug’s manufacturer) stated “we provided IQWiG with sufficient data” but then released most of the missing data; and the full assessment showed that, overall, reboxetine had no benefit. At the start, IQWiG had asked the manufacturer to sign an agreement requiring: (1) submission of a list of all sponsored published and unpublished trials investigating reboxetine; (2) submission of documents (generally the clinical study reports) compliant with the CONSORT criteria for all relevant trials selected by IQWiG; and (3) permission for publication of all previously unpublished relevant data.

Pfizer’s provision of the data was an important, if belated, move. We hope that other drug companies and device manufacturers will see the BMJ’s theme issue next year as an opportunity to release and use data constructively and to improve clinical decision making. Some have already asked editors’ advice on transparency and, despite—or perhaps owing to—the BMJ’s reputation for being tough on industry, this journal’s editors have recently accepted unpaid invitations to talk with companies about publication ethics. Fiona Godlee and I have visited GlaxoSmithKline and AstraZeneca, and several of us have spoken at industry focused conferences run by organisations including the Drug Information Association and the International Society for Medical Publication Professionals. I have participated, too, in the consultations that produced two sets of guidance for industry based authors: the Good Publication Practice guidance (GPP2) (BMJ 2009;339:b4330) and the Authors’ Submission Toolkit (doi:10.1185/03007995.2010.499344).

There’s ever more that journals can and should do to improve the evidence base, however. And this week Gerd Gigerenzer and colleagues rightly chastise us for failing to enforce transparent reporting of risks in journal abstracts (doi:10.1136/bmj.c4830). They urge institutions to cancel their subscriptions if we don’t implement policies in the next two years to frame risks properly. We have been warned.