Table 4

Criteria used in existing tools to inform a judgement of ‘high’ risk of bias due to selective non-reporting

‘High’ risk of bias criteria proposed in existing toolsAHRQ ORBAHRQ RRBAXISGRADEQUIPSRoB 1.0SYRCLE RoBRoBANSORBIT-ISAQATReidORBIT-IINMA-QualityRTI 2012RTI 2013Total, n (%)
Assessment directed at study as a whole
 One or more outcomes of interest were clearly measured, but no results were reported✓✓✓✓✓✓✓✓8 (53)
 One or more outcomes of interest were reported incompletely so that they could not be entered in a meta-analysis✓✓2 (13)
 The study report fails to include results for a key outcome that would be expected to have been reported for such a study✓✓✓✓✓5 (33)
Assessment directed at a specific outcome
 Particular outcome clearly measured but no results were reported✓✓✓✓✓✓6 (40)
 Particular outcome of interest is reported incompletely so that it cannot be entered in a meta-analysis (typically stating only that P>0.05)✓✓✓✓✓✓6 (40)
 Judgement says particular outcome is likely to have been measured and analysed but not reported on the basis of its results✓✓✓✓✓✓6 (40)
 Composite outcomes are presented without the individual component outcomes✓✓2 (13)
 Result reported globally across all groups✓1 (7)
 Result reported for some groups only✓1 (7)
 Data were not reported consistently for the outcome of interest✓1 (7)
Assessment directed at a specific synthesis
 Selective non-reporting suspected in a number of included studies✓✓✓✓4 (27)

  • AHRQ ORB, AHRQ outcome and analysis reporting bias framework28; AHRQ RRB, AHRQ tool for evaluating the risk of reporting bias29; AXIS, Appraisal tool for Cross-Sectional Studies30; GRADE, GRADE rating of quality of evidence34–37; NMA-Quality, Framework for evaluating the quality of evidence from a network meta-analysis49; ORBIT-I, Outcome Reporting Bias In Trials classification system for benefit outcomes3 32; ORBIT-II, Outcome Reporting Bias In Trials classification system for harm outcomes48; QUIPS, Quality In Prognosis Studies tool38; Reid, Reid et al selective reporting bias algorithm47; RoB 1.0, Cochrane risk of bias tool for randomised trials21 39 40; RoBANS, Risk of Bias Assessment Tool for Nonrandomized Studies44; RTI 2012, RTI Item Bank for Assessment of Risk of Bias and Precision for Observational Studies of Interventions or Exposures51; RTI 2013, RTI Item Bank for Assessing Risk of Bias and Confounding for Observational Studies of Interventions or Exposures52; SAQAT, Semi-Automated Quality Assessment Tool45 46; SYRCLE RoB, SYstematic Review Centre for Laboratory animal Experimentation risk of bias tool.43