Intended for healthcare professionals

  1. Research
  2. Analysis of the...
  3. Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review
CCBYNC Open access
Research

Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review

BMJ 2013; 347 doi: https://doi.org/10.1136/bmj.f3675 (Published 01 July 2013) Cite this as: BMJ 2013;347:f3675
  1. Aïda Bafeta, PhD student12,
  2. Ludovic Trinquart, senior statistician1234,
  3. Raphaèle Seror, associate professor of rheumatology124,
  4. Philippe Ravaud, professor of epidemiology and director12345
  1. 1INSERM U738 Research Unit, Paris, France
  2. 2Assistance Publique-Hôpitaux de Paris, Centre d’Epidémiologie Clinique, Hôpital Hôtel-Dieu, 75004 Paris, France
  3. 3Université Paris Descartes-Sorbonne Paris Cité, Paris, France
  4. 4French Cochrane Centre, Paris, France
  5. 5Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, USA
  1. Correspondence to: A Bafeta aida.bafeta{at}htd.aphp.fr
  • Accepted 31 May 2013

Abstract

Objective To examine whether network meta-analyses, increasingly used to assess comparative effectiveness of healthcare interventions, follow the key methodological recommendations for reporting and conduct of systematic reviews.

Design Methodological systematic review of reports of network meta-analyses.

Data sources Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Medline, and Embase, searched from inception to 12 July 2012.

Review methods All network meta-analyses comparing clinical efficacy of three or more interventions based on randomised controlled trials, excluding meta-analyses with an open loop network of three interventions. We assessed the reporting of general characteristics and key methodological components of the systematic review process using two composite outcomes. For some components, if reporting was adequate, we assessed their conduct quality.

Results Of 121 network meta-analyses covering a wide range of medical areas, 100 (83%) assessed pharmacological interventions and 11 (9%) non-pharmacological interventions; 56 (46%) were published in journals with a high impact factor. The electronic search strategy for each database was not reported in 88 (73%) network meta-analyses; for 36 (30%), the primary outcome was not clearly identified. Overall, 61 (50%) network meta-analyses did not report any information regarding the assessment of risk of bias of individual studies, and 103 (85%) did not report any methods to assess the likelihood of publication bias. Overall, 87 (72%) network meta-analyses did not report the literature search, searched only one database, did not search other sources, or did not report an assessment of risk of bias of individual studies. These methodological components did not differ by publication in a general or specialty journal or by public or private funding.

Conclusions Essential methodological components of the systematic review process—conducting a literature search and assessing risk of bias of individual studies—are frequently lacking in reports of network meta-analyses, even when published in journals with high impact factors.

Introduction

Assessing the comparative effectiveness of many or all available interventions for a clinical indication is challenging.1 Direct evidence from head to head trials is often lacking.2 3 4 5 Indirect comparison between two interventions can still be estimated from the results of randomised controlled trials, without jeopardising the randomised comparisons within each trial, if each intervention is compared with an identical common comparator.6 7 When both direct and indirect comparisons are available, the two sources of information can be combined by using network meta-analyses—also known as multiple treatments meta-analyses or mixed treatment comparison meta-analyses.8 9 These methods allow for estimating all possible pairwise comparisons between interventions and placing them in rank order.

In the past few years, network meta-analyses have been increasingly adopted for comparing healthcare interventions.10 11 12 Network meta-analyses are attractive for clinical researchers because they seem to respond to their main concern: determining the best available treatment. Moreover, national agencies for health technology assessment and drug regulators increasingly use such methods.13 14 15 16

Several reviews previously evaluated how indirect comparisons have been conducted and reported.7 10 11 12 17 18 These reviews focused on checking validity assumptions (based on homogeneity, similarity, and consistency) and did not assess the key components of the systematic review process. Network meta-analyses are primarily meta-analyses, and should therefore be performed according to the explicit and rigorous methods used in systematic reviews and meta-analyses to minimise bias.19 We performed a methodological systematic review of published reports of network meta-analyses to examine how they were reported and conducted. In particular, we assessed whether the reports adequately featured the key methodological components of the systematic review process.

Methods

Search strategy

We systematically searched for all published network meta-analyses in the following databases from their dates of inception: Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Medline, and Embase. Search equations were developed for each database and were based partly on search terms used by Song and colleagues12 and Salanti and colleagues20 (web table 1). The equations were based on specific free text words pertaining to network meta-analyses or overviews of reviews. The date of the last search was 12 July 2012. We also screened the references of methodology papers and reviews11 12 13 14 15 16 17 21 and searched for papers citing landmark statistical articles through the Web of Science or SCOPUS database.6 8 22 23 24

Eligibility criteria

All reports of network meta-analyses comparing the clinical efficacy of three or more interventions based on randomised controlled trials were eligible. We excluded reports of adjusted meta-analyses involving indirect comparisons (that is, an open loop network of three interventions)10 13 17 as well as publications of methodology, editorial style reviews or reports, cost effectiveness reviews, reviews based on individual patient data, and reviews not involving human participants. We also excluded reviews not published in English, French, German, Spanish, and Italian.

Selection of relevant reports of network meta-analyses

Two reviewers (AB, LT) independently selected potentially relevant reports of network meta-analyses on the basis of the title and abstract and, if needed, the full text according to prespecified eligibility criteria. Disagreements were discussed to reach consensus.

Data extraction

A standardised form was used to collect all data from the original reports of network meta-analyses and supplementary appendices when available. We collected all data for epidemiological and descriptive characteristics and those pertaining to the key methodological components of the systematic review process. Two reviewers (AB, RS) independently extracted all data from a random sample of 20% of reports. From all reports, two reviewers (AB, RS) independently extracted data for items that involved subjective interpretation. The corresponding items were questions referring to participants, interventions, comparisons, and outcomes; the type of comparator; how reporting bias was assessed; the publication status of selected randomised controlled trials; and whether statistical assumptions (homogeneity, similarity, and consistency) were mentioned. Disagreements were resolved by discussion.

General characteristics

The following general characteristics were collected: journal name, year of publication, country of corresponding author, medical area, funding source, and type of interventions included in the network. We categorised journal types into general or specialty; we also identified journals with high impact factors (that is, the 10 journals with the highest impact factor for each medical subject category of the Journal Citation Reports 2011). Regarding network typology, we collected the number of interventions, the number of comparisons conducted by at least one randomised controlled trial (that is, direct comparisons), and the number of randomised controlled trials included in the network.

Reporting of key methodological components of the systematic review process

According to PRISMA25 and AMSTAR26 guidelines, we assessed whether key methodological components were reported or not. In the introduction and methods sections of each report, these factors included:

  • Questions referring to participants, interventions, comparisons, and outcomes

  • Existence of a review protocol

  • Primary outcome explicitly specified in the article or abstract, in the primary study objectives, or as the only outcome reported in the article27

  • Information sources including databases searched, electronic search strategy, date of last search and period covered by the search of each database, any other sources (conference abstracts, unpublished studies, textbooks, specialty registers (for example, the US Food and Drug Administration), contact with study authors, reviewing the references in the studies found or any relevant systematic reviews), ongoing studies searched, and any or no restrictions related to language or publication status (that is, published studies or any other sources or ongoing studies)

  • Methods for study selection and data extraction

  • Methods used for assessing risk of bias of individual studies, such as use of any scales, checklists, or domain based evaluation recommended by the Cochrane Collaboration, with separate critical assessments for different domains (that is, allocation concealment, generation of sequence allocation, blinding, incomplete outcome data, or selective reporting)

  • Methods to incorporate assessment of risk of bias of individual studies in the analysis or conclusion of review (that is, subgroup analysis, inclusion criteria, sensitivity analysis, or the grading recommendations assessment development and evaluation (GRADE))28

  • Methods to assess publication or reporting bias (statistical or graphical evaluation).

Regarding the search strategy, we distinguished between a “de novo” search (a new comprehensive literature search) and a mixed search strategy (an initial search for systematic reviews complemented by an updated search of trials not covered by the search period of these systematic reviews).

In the results section, we assessed whether reports included the following components:

  • Study selection, including the number of studies screened, assessed for eligibility, and included in the review; with reasons for exclusions at each stage (for example, a PRISMA flowchart) and the list of studies included and excluded

  • Study characteristics, including a description of the trial network with the number of interventions and number of comparisons conducted by at least one randomised controlled trial (that is, direct comparisons), characteristics of patients (for example, age, sex, disease status, severity), duration of follow-up of patients and duration of interventions, description of interventions, number of study groups and patients, and funding sources

  • Risk of bias within studies

  • Risk of bias across studies (publication bias).

Finally, we assessed whether the discussion section included:

  • A limitation regarding reporting bias

  • Whether the authors mentioned or discussed the assumptions required in network meta-analyses (based on homogeneity, similarity, consistency, and exchangeability)

  • Whether authors mentioned a conflict of interest.

Conduct quality when the reporting was adequate

When items were adequately reported, we subsequently assessed whether the conduct quality was adequate. We considered the following items as inadequate conduct: electronic search of only one bibliographic database, restricting study selection to published reports, lack of independent duplicate study selection, and lack of independent duplicate data extraction.

Composite outcome assessing inadequate reporting or inadequate conduct quality

To summarise data, we built two composite outcomes assessing inadequate reporting or inadequate conduct quality (when the reporting was adequate). The first composite outcome was based on guidelines from Li and colleagues.29 Inadequate reporting or conduct quality was considered as one of the following:

  • The authors did not report a literature search, they reported an electronic search of only one bibliographic database, or they did not supplement their investigation by searching for any other sources

  • The authors did not report an assessment of risk of bias of individual studies.

The second composite outcome was defined as an all or none approach, based on seven methodological items identified as mandatory according to the methodological expectations of Cochrane intervention reviews.30 Conduct or reporting quality was considered inadequate if:

  • The authors did not report questions referring to participants, interventions, comparisons, and outcomes

  • The authors did not report eligibility criteria

  • The authors did not report a literature search; they reported an electronic search of only one bibliographic database; or they did not supplement their investigation by searching for references in studies found, any relevant systematic reviews, or ongoing studies (for example, clinicaltrials.gov or the World Health Organization International Clinical Trials Registry Platform)

  • The authors did not report the methods of study selection and the data extraction process, or they reported inadequate methods

  • The authors did not report the results of the selection of primary studies

  • The authors did not provide a description of the primary studies

  • The authors did not report an assessment of the risk of bias of individual studies.

Statistical analysis

Quantitative data were summarised by medians and interquartile ranges, and categorical data summarised by numbers and percentages. We compared general characteristics and reporting of key items for network meta-analyses with results published in general and specialty journals and with public and private funding sources. All or none measurements were analysed by χ2 tests for categorical data. All tests were two sided, and P<0.05 was considered significant. Statistical analyses involved use of SAS version 9.3 (SAS Institute).

Results

General characteristics

Our search identified 1226 citations, including 333 duplicate citations. Among the 893 potentially relevant publications, 118 were eligible reports of network meta-analyses. Three additional network meta-analyses were identified from methodological articles, resulting in 121 eligible reports of network meta-analyses (figs 1⇓ and 2⇓; web table 2 shows the main characteristics of the network meta-analysesw1-w121).

Figure1

Fig 1 Flowchart of selection of network meta-analyses. CDSR=Cochrane Database of Systematic Reviews; DARE= Database of Abstracts of Reviews of Effects; NMA=network meta-analysis; RCT= randomised controlled trial

Figure2

Fig 2 Number of reports of network meta-analyses published per year. *We estimated the number of reports published from July to December 2012 on the basis of the number of published network meta-analyses from the six previous months. Between January and June 2012, 27 reports were published (dark region); therefore, we estimated 27 reports from July to December 2012 (light region)

Reports of network meta-analyses were published in 75 different journals, with 55 reports (45%) published in general journals, 66 (55%) in specialty journals, and 56 (46%) in journals with a high impact factor (table 1⇓; web table 3). The network meta-analyses covered a wide range of medical areas, and 100 (83%) described the assessment of pharmacological interventions. The median number of interventions assessed per network meta-analysis was seven (interquartile range 5-9), and the median number of randomised controlled trials included per network meta-analysis was 22 (15-40).

Table 1

 Epidemiological and descriptive characteristics of 121 reports of network meta-analyses

View this table:

Reporting of key methodological components of the systematic review process

Only three (2%) network meta-analyses did not report the databases searched in the methods section; of the remaining 118, 99 (82%) used a de novo search strategy and 19 (16%) a mixed search strategy. However, 42 (35%) did not report searching other sources, and 39 (32%) did not report any information regarding restriction or no restriction of study selection related to publication status (table 2⇓). In addition, 42 (35%) and 32 (26%) network meta-analyses did not report the methods used for study selection and data extraction, respectively. A total of 61 (50%) network meta-analyses did not report any information regarding the assessment of risk of bias of individual studies. Of 121 network meta-analyses, 103 (85%) did not report any methods to assess the likelihood of publication bias.

Table 2

 Reporting of key methodological components of the systematic review process in network meta-analyses, by journal type

View this table:

In the results section, 95 (79%) network meta-analyses did not describe the characteristics of primary studies (that is, characteristics of the network, patient characteristics, and interventions); 70 (58%) did not report the risk of bias assessment within studies.

The similarity and consistency assumptions were not frequently mentioned in network meta-analyses (66% (n=80) and 44% (n=63) of reports, respectively; table 2).These findings did not differ by journal type or funding source (web table 4).

Conduct quality when the reporting was adequate

For network meta-analyses with adequate reporting, the conduct quality was frequently inadequate: 11% of these network meta-analyses involved a search of only one bibliographic database, 20% restricted the study selection to published articles, 43% lacked an independent duplicate study selection, and 21% lacked an independent extraction of duplicate data. These findings did not differ by journal type or funding source (table 3⇓; web table 5).

Table 3

 Inadequate quality of conduct of the systematic review process with adequate reporting in network meta-analyses, by journal type

View this table:

Composite outcome assessing inadequate reporting or inadequate conduct quality

Overall, 87 network meta-analyses (72%) showed inadequate reporting of key methodological components or inadequate conduct quality according to guidelines from Li and colleagues.29 This measurement did not differ by journal type (general journal, 69% (95% confidence interval 57% to 81%); specialty journal, 74% (63% to 85%); P=0.5) or funding source (public funding, 67% (74% to 79%); private funding, 79% (67% to 90%); P=0.2).

We used a second composite according to the seven mandatory items of methodological expectations of Cochrane intervention reviews.30 Based on this composite, 120 network meta-analyses (99%) showed inadequate reporting of key methodological components or inadequate conduct. These findings did not differ by journal type or funding source.

Discussion

We identified 121 reports of network meta-analyses covering a wide range of medical areas. Key methodological components of the systematic review process were missing in most reports. The reporting did not differ by publication in a general or specialty journal or by public or private funding.

Several guidelines have been developed to assess the methodological quality and reporting of systematic reviews and meta-analyses,26 30 31 32 33 and according to all these guidelines, retrieving all relevant studies is crucial for the success of every systematic review. The literature search needs to be exhaustive and adequate to maximise the likelihood of capturing all relevant studies and minimise the effects of reporting biases that might influence the nature and direction of results.34 35 36 More than half of our network meta-analyses failed to report exhaustive searches; for those with adequate reporting, 11% searched only one electronic database (web table 6).

In addition, more than half of our network meta-analyses did not describe the characteristics of patients, the duration of follow-up or treatment, or the intervention. The lack of adequate reporting of the characteristics of primary studies raises several problems. Knowing precisely the type of intervention, characteristics, and duration of follow-up and treatment is a preliminary step for researchers and readers to assess the assumption of similarity required in network meta-analyses.12 37

An important step of the systematic review process is to assess the risk of bias of individual studies.38 39 40 41 42 The risk of bias in individual studies could affect the findings of network meta-analyses.43 However, 58% of our network meta-analyses did not report the risk of bias of individual studies in the results sections, and 84% did not report a method to incorporate assessment of risk of bias of individual studies in the analysis or conclusions of the review (web table 6).

Publication bias could affect the results of meta-analyses and network meta-analyses.44 45 46 47 This factor is all the more important, considering that we lacked the validated methods to detect or to appropriately adjust or exclude reporting bias in network meta-analyses. Moreover, reporting bias across trials could differentially affect the various comparisons in the network and modify the rank order efficacy of treatments. However, most of our network meta-analyses (85%) did not report a method to assess the publication bias. Although reporting bias could have a substantial effect on the conclusions of a network meta-analysis, it is probably neglected.

Strengths and limitations of study

To the best of our knowledge, no study investigated the systematic review process in network meta-analyses (mixed treatment, comparison meta-analyses).10 13 Recently, the Agency for Healthcare Research and Quality assessed the statistical methods used in a sample of published network meta-analyses, but did not analyse the conduct and reporting of the systematic review process.48

Our study had some limitations. Assessing conduct quality from published reports alone could be unreliable, as has been shown for randomised trials.49 The study authors may have used adequate methods but omitted important details from their reports, or key information may have been deleted during the publication process. However, we were in the position of the reader, who can only assess what was reported. In addition, we judged that the conduct quality was inadequate only if the reporting was adequate. Furthermore, we did not assess health technology assessment reports, which could have had higher quality or precision than the reports analysed.

Conclusions and implications

Our study identified some important methodological components in the reports of network meta-analyses that may raise doubts about the confidence we could have in their conclusions. Potential flaws in the conduct of network meta-analyses could affect their findings. Performing a network meta-analysis outside the realm of a systematic review, without extensive and thorough searches for eligible trials, and without risk of bias assessments of trials, would increase the risk of a network meta-analysis producing biased results. Guidelines are needed to improve the quality of reporting and conduct of network meta-analyses.

What is already known on this topic

  • Network meta-analyses are primarily meta-analyses, and should be conducted by respecting the methodological rules of systematic reviews

  • Network meta-analyses are subject to the same methodological risks of standard pairwise systematic reviews, and because this method in network meta-analyses is complex, it is probably more vulnerable to these risks

What this study adds

  • Key methodological components of the systematic review process are frequently inadequately reported in publications of network meta-analyses

  • The level of inadequate reporting does not differ between reports published by a general or specialty journal or between reports with public or private funding

  • Inadequate reporting of results of network meta-analyses raises doubts about the ability of network meta-analyses to help clinical researchers determine the best available treatment

Notes

Cite this as: BMJ 2013;347:f3675

Footnotes

  • Contributors: AB was responsible for the study conception, search of trials, selection of trials, data extraction, data analysis, interpretation of results, and drafting the manuscript. LT was responsible for the study conception, search of trials, selection of trials, interpretation of results, and drafting the manuscript. RS was responsible for the data extraction, interpretation of results, and drafting the manuscript. PR was the guarantor and was responsible for the study conception, interpretation of results, and drafting the manuscript. All authors, external and internal, had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

  • Funding: This study was funded by an academic grant for the doctoral student from “Pierre et Marie Curie University.” Our team is supported by an academic grant (DEQ20101221475) for the programme “Equipe espoir de la Recherche,” from the Fondation pour la Recherche Médicale. The funding agencies have no role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation and review of the manuscript.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: Ethical approval not required.

  • Data sharing: No additional data available.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.

References

  1. ↵
    Dickersin K. Health-care policy. To reform US health care, start with systematic reviews. Science2010;329:516-7.
    OpenUrlAbstract/FREE Full Text
  2. ↵
    Lathyris DN, Patsopoulos NA, Salanti G, Ioannidis JP. Industry sponsorship and selection of comparators in randomized clinical trials. Eur J Clin Invest2010;40:172-82.
    OpenUrlCrossRefPubMedWeb of Science
  3. ↵
    Hochman M, McCormick D. Characteristics of published comparative effectiveness studies of medications. JAMA2010;303:951-8.
    OpenUrlCrossRefPubMed
  4. ↵
    Volpp KG, Das A. Comparative effectiveness—thinking beyond medication A versus medication B. N Engl J Med2009;361:331-3.
    OpenUrlCrossRefPubMedWeb of Science
  5. ↵
    Estellat C, Ravaud P. Lack of head-to-head trials and fair control arms: randomized controlled trials of biologic treatment for rheumatoid arthritis. Arch Intern Med2012;172:237-44.
    OpenUrlCrossRefPubMedWeb of Science
  6. ↵
    Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol1997;50:683-91.
    OpenUrlCrossRefPubMedWeb of Science
  7. ↵
    Song F, Altman DG, Glenny AM, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ2003;326:472.
    OpenUrlAbstract/FREE Full Text
  8. ↵
    Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med2002;21:2313-24.
    OpenUrlCrossRefPubMedWeb of Science
  9. ↵
    Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med2004;23:3105-24.
    OpenUrlCrossRefPubMedWeb of Science
  10. ↵
    Jansen JP, Fleurence R, Devine B, Itzler R, Barrett A, Hawkins N, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR task force on indirect treatment comparisons good research practices: part 1. Value Health2011;14:417-28.
    OpenUrlCrossRefPubMedWeb of Science
  11. ↵
    Donegan S, Williamson P, Gamble C, Tudur-Smith C. Indirect comparisons: a review of reporting and methodological quality. PLoS One2010;5:e11054.
    OpenUrlCrossRefPubMed
  12. ↵
    Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ2009;338:b1147.
    OpenUrlAbstract/FREE Full Text
  13. ↵
    Wells GA, Sultan SA, Chen L, Khan M, Coyle D. Indirect evidence: indirect treatment comparisons in meta-analysis. Canadian Agency for Drugs and Technologies in Health, 2009.
  14. ↵
    Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D’Amico R, et al. Indirect comparisons of competing interventions. Health Technol Assess2005;9:1-134, iii-iv.
    OpenUrlPubMed
  15. ↵
    Schöttker B, Luhmann D, Boulkheimair D, Raspe H. Indirekte vergleiche von therapieverfahren. GMS Health Technology Assessment, 2009.
  16. ↵
    Cucherat M, Izard V. Les comparaisons indirectes Méthodes et validité. HAS-Service de l’Evaluation des Medicaments, 2009:66.
  17. ↵
    Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, et al. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR task force on indirect treatment comparisons good research practices: part 2. Value Health2011;14:429-37.
    OpenUrlCrossRefPubMedWeb of Science
  18. ↵
    Song F, Harvey I, Lilford R. Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. J Clin Epidemiol2008;61:455-63.
    OpenUrlCrossRefPubMedWeb of Science
  19. ↵
    Li T, Puhan MA, Vedula SS, Singh S, Dickersin K. Network meta-analysis-highly attractive but more methodological research is needed. BMC Med2011;9:79.
    OpenUrlCrossRefPubMed
  20. ↵
    Salanti G, Kavvoura FK, Ioannidis JP. Exploring the geometry of treatment networks. Ann Intern Med2008;148:544-53.
    OpenUrlCrossRefPubMedWeb of Science
  21. ↵
    Edwards SJ, Clarke MJ, Wordsworth S, Borrill J. Indirect comparisons of treatments based on systematic reviews of randomised controlled trials. Int J Clin Pract2009;63:841-54.
    OpenUrlCrossRefPubMedWeb of Science
  22. ↵
    Lu G, Ades AE, Sutton AJ, Cooper NJ, Briggs AH, Caldwell DM. Meta-analysis of mixed treatment comparisons at multiple follow-up times. Stat Med2007;26:3681-99.
    OpenUrlCrossRefPubMedWeb of Science
  23. ↵
    Salanti G, Higgins JP, Ades AE, Ioannidis JP. Evaluation of networks of randomized trials. Stat Methods Med Res2008;17:279-301.
    OpenUrlAbstract/FREE Full Text
  24. ↵
    Salanti G, Ades AE, Ioannidis JP. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol2011;64:163-71.
    OpenUrlCrossRefPubMedWeb of Science
  25. ↵
    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med2009;151:W65-94.
    OpenUrlPubMedWeb of Science
  26. ↵
    Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol2007;7:10.
    OpenUrlCrossRefPubMed
  27. ↵
    Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA2004;291:2457-65.
    OpenUrlCrossRefPubMedWeb of Science
  28. ↵
    Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ2008;336:924-6.
    OpenUrlFREE Full Text
  29. ↵
    Li T, Vedula SS, Scherer R, Dickersin K. What comparative effectiveness research is needed? A framework for using guidelines and systematic reviews to identify evidence gaps and research priorities. Ann Intern Med2012;156:367-77.
    OpenUrlPubMedWeb of Science
  30. ↵
    Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.1.0. Cochrane Collaboration, 2011.
  31. ↵
    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med2009;6:e1000100.
    OpenUrlCrossRefPubMed
  32. ↵
    Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses. Lancet1999;354:1896-900.
    OpenUrlCrossRefPubMedWeb of Science
  33. ↵
    Mills EJ, Ioannidis JP, Thorlund K, Schunemann HJ, Puhan MA, Guyatt GH. How to use an article reporting a multiple treatment comparison meta-analysis. JAMA2012;308:1246-53.
    OpenUrlCrossRefPubMedWeb of Science
  34. ↵
    McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet2000;356:1228-31.
    OpenUrlCrossRefPubMedWeb of Science
  35. ↵
    Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care2003;19:591-603.
    OpenUrlPubMedWeb of Science
  36. ↵
    Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol2003;56:943-55.
    OpenUrlCrossRefPubMedWeb of Science
  37. ↵
    Xiong T, Parekh-Bhurke S, Loke YK, Abdelhamid A, Sutton AJ, Eastwood AJ, et al. Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates. J Clin Epidemiol2013;66:184-91.
    OpenUrlCrossRefPubMed
  38. ↵
    Gluud LL. Bias in clinical intervention research. Am J Epidemiol2006;163:493-501.
    OpenUrlAbstract/FREE Full Text
  39. ↵
    Juni P, Altman DG, Egger M. Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ2001;323:42-6.
    OpenUrlFREE Full Text
  40. ↵
    Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ2008;336:601-5.
    OpenUrlAbstract/FREE Full Text
  41. ↵
    Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet1998;352:609-13.
    OpenUrlCrossRefPubMedWeb of Science
  42. ↵
    Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA1995;273:408-12.
    OpenUrlCrossRefPubMedWeb of Science
  43. ↵
    Dias S, Welton NJ, Marinho V, Salanti G, Higgins JP, Ades AE. Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta-analysis. J R Stat Soc2010;173:613-29.
    OpenUrlCrossRef
  44. ↵
    Thornton A, Lee P. Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol2000;53:207-16.
    OpenUrlCrossRefPubMedWeb of Science
  45. ↵
    Trinquart L, Abbe A, Ravaud P. Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials. PLoS One2012;7:e35219.
    OpenUrlCrossRefPubMed
  46. ↵
    Trinquart L, Chatellier G, Ravaud P. Adjustment for reporting bias in network meta-analysis of antidepressant trials. BMC Med Res Methodol2012;12:150.
    OpenUrlCrossRefPubMed
  47. ↵
    Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods2012;3:80-97.
    OpenUrlCrossRef
  48. ↵
    Coleman CI, Phung OJ, Cappelleri JC, Baker WL, Kluger J, White CM, et al. Use of mixed treatment comparisons in systematic reviews. AHRQ Methods for Effective Health Care2012. www.ncbi.nlm.nih.gov/books/NBK107330/.
  49. ↵
    Vale CL, Tierney JF, Burdett S. Can trial quality be reliably assessed from published reports of cancer trials: evaluation of risk of bias assessments in systematic reviews. BMJ2013;346:f1798.
    OpenUrlAbstract/FREE Full Text
View Abstract
Back to top