Reporting of harms in systematic reviews and their primary studies

In a linked paper, the Outcome Reporting Bias in Trials (ORBIT) collaborators report a new study on outcome reporting bias (doi:10.1136/bmj.g6501),1 a sequel to their earlier groundbreaking publication.2 Outcome reporting bias is defined as “selection (on the basis of the results) of a subset of the original variables recorded for inclusion in a study publication.” It can arise from selective non-reporting or incomplete reporting of an outcome, as studied by the ORBIT collaborators, or fully reporting a particular outcome selectively from among multiple outcomes. Selective non-reporting has the same impact on a systematic review as the failure to report a study altogether (“publication bias”), biasing the pooled estimate away from the null.

The earlier paper2 considered benefit outcomes, whereas this new research investigated harm outcomes.1 This is important research because harms are poorly reported.3 Authors of systematic reviews are being encouraged to identify a primary harm outcome for each review as well as a primary hypothesised benefit,4 and motivations for selectively reporting outcomes may well vary for beneficial and harm outcomes. Kirkham and colleagues studied two cohorts of systematic reviews, one formed from new Cochrane reviews in 2012 and the other from a database of reviews specifically evaluating harms.1 The researchers investigated the reporting of primary harm outcomes both in the reviews and in studies included in the reviews …