Article Text
PDF
Abstract
Objective To characterise the early diffusion of indirect comparison meta-analytic methods to study drugs.
Design Systematic literature synthesis.
Data sources Cochrane Database of Systematic Reviews, EMBASE, MEDLINE, Scopus and Web of Science.
Study selection English language papers that used indirect comparison meta-analytic methods to study the efficacy or safety of three or more interventions, where at least one was a drug.
Data extraction The number of publications and authors was plotted by year and type: methodological contribution, review or empirical application. Author and methodological details were summarised for empirical applications, and animated coauthorship networks were created to visualise contributors by country and affiliation type (academia, industry, government or other) over time.
Results We identified 477 papers (74 methodological contributions, 42 reviews and 361 empirical applications) by 1689 distinct authors from 1997 to 2013. Prior to 2002, only three applications were published, with contributions from the USA (n=2) and Canada (n=1). The number of applications gradually increased annually with rapid uptake between 2011 and 2013 (n=254, 71%). Early diffusion occurred primarily in Europe with the first application credited to the UK in 2003. Application spread to other European countries in 2005, and may have been supported by regulatory requirements for drug approval. By the end of 2013, contributions included 49% credited to Europe (22% UK, 27% other), 37% credited to North America (11% Canada, 26% USA) and 14% from other regions.
Conclusion Indirect comparison meta-analytic methods are an important innovation for health research. Although Canada and the USA were the first to apply these methods, Europe led their diffusion. The increase in uptake of these methods may have been facilitated by acceptance by regulatory agencies, which are calling for more comparative drug effect data to assist in drug accessibility and reimbursement decisions.
- diffusion of innovations
- indirect comparisons
- methodological innovation
- network meta-analysis
- social networks
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:©http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- diffusion of innovations
- indirect comparisons
- methodological innovation
- network meta-analysis
- social networks
Strengths and limitations of this study
Our paper walks through the development and history of indirect comparison meta-analytic methods and its early diffusion in the study of drugs using coauthorship networks.
Our animated coauthorship networks are innovative by allowing the visualisation of social structures and collaboration trends through authors and their connections with each other over time.
Mapping contributions based on first and last authors may miss some important contributions by coauthors, yet including second authors had little impact in a prior study.
We conducted an extensive systematic literature search that identified 477 (361 empirical applications) eligible English language papers that used indirect comparison meta-analytic methods to study drugs through to December 2013, yet some relevant papers such as non-English language papers and grey literature may have been missed.
We did not consider methodological and reporting quality, since focus was on early uptake by country and institutional affiliation, yet our results set the stage for further research that may consider quality.
Introduction
Randomised controlled trials (RCT) are essential for bringing novel pharmaceutical products to market. RCTs for drug approval typically compare new treatment efficacy to placebo and provide safety data for only common adverse effects. However, RCTs are often not powered to identify all important drug efficacy and safety endpoints and thus meta-analytic methods were developed. Meta-analysis is a statistical method that combines the results of two or more studies to evaluate the same intervention in comparison to a control such as placebo, to obtain a more precise estimate of the intervention’s effects relative to that control.1–3 The term meta-analysis was first coined by GV Glass in 1976, yet use of statistical methods to combine the results of multiple studies dates back to the early part of the 20th century, with early methodological techniques proposed by R Fisher and W Cochran in the 1930s.1 2
When completed using high-quality RCTs, meta-analyses are regarded as providing the highest level of evidence.4 However, traditional pairwise meta-analysis is limited by only being able to combine and estimate the benefits or harms of two treatments if they have been compared directly. In addition, meta-analysis cannot compare more than two treatments at a time.3 5 This presents a challenge to policymakers, clinicians and patients who often need to select the most optimal treatment from several competing options.6 Indirect comparisons have been made informally using point estimates and 95% CIs of treatments.7 However, this informal approach does not provide a precise estimate of the relative difference between two treatments because the relative effects are not measured.
In 1997, the adjusted indirect comparison method was proposed by Bucher et al, as an innovative meta-analytic approach that uses indirect evidence to estimate the relative benefits and risks between two treatments.8 Unlike traditional pairwise meta-analysis, adjusted indirect comparisons estimate the relative effects of two treatments that have not been compared directly by leveraging results from each treatment that has been compared with a common comparator, such as a placebo.6 8 9 However, the adjusted indirect comparison method ignores direct evidence, even when available. In 2002, network meta-analysis was proposed as an extension of the adjusted indirect comparison method that combines direct and indirect comparative data across several sets of pairwise treatment comparisons.5 10 The combination of direct and indirect data yields more precise effect estimates.6 A similar method, coined mixed treatment comparison, was proposed in 2004,11 and the term multiple treatment meta-analysis was also introduced to describe concepts of combining both direct and indirect evidence in 20055 (table 1).
Timeline of meta-analytic methodological innovations
Indirect comparison meta-analytic methods have become valuable tools in clinical and policy decision-making, and have thus been rapidly adopted since their introduction.7 12–14 However, application of these methodological innovations varies widely.6 12 15 Rogers’ Diffusion of Innovations Model defines diffusion as the process by which an innovation is communicated across individuals within a social system, particularly during the initial stages of its use.16 17 Our study sought to characterise the early diffusion of indirect comparison meta-analytic methods used to study drugs.16 We interpreted diffusion and uptake relative to the social system by creating coauthorship networks to examine the speed of uptake (number of publications) and spread of these methods (collaboration between authors, authors’ countries and across institutions) over time.
Materials and methods
We recently examined the diffusion of two confounder summary score methods and illustrate the importance of innovation attributes (relative advantage, compatibility, simplicity, trialability and observability) and seminal author engagement on the uptake of methodological innovations using Rogers’ Diffusion of Innovations Model.16 In addition to innovation attributes, Rogers’ Model identifies key aspects of the social system that may impact the rate of adoption.17 In particular, a methodological innovation will have a quicker rate of adoption if members within the social system (eg, researchers, clinicians and policymakers) share similar system norms. For example, regulatory agencies make decisions for drug approval and formulary coverage. Regulatory agencies are therefore well positioned to influence the uptake of methodological innovations that support the drug approval process. If novel methods become a requirement for drug approval, pharmaceutical companies, which share a vested interest in the drug approval process, may willingly adopt the methodological innovation in question. We examined the diffusion and early uptake of indirect comparison meta-analytic methods used to study drugs, and interpreted contributions by country and affiliation type using Rogers’ Diffusion of Innovations Model.
Systematic search
We completed a systematic literature search to identify all papers that used indirect comparison meta-analytic methods to study drug effects in humans. We searched the Cochrane Database of Systematic Reviews, EMBASE and MEDLINE from their dates of inception to 31 December 2013 using keywords based on a recent search (online supplementary appendix table A).18 We then used SCOPUS and Web of Science to perform a citation search to identify papers that referenced key seminal papers,8 10 major methodological contributions19–21 and reviews7 13–15 22 23 on indirect comparison meta-analytic methods.18
Supplementary file 3
All English language papers that used indirect meta-analytic methods to compare the clinical efficacy or safety of three or more interventions among humans were eligible if at least one intervention was a drug. We excluded abstracts, letters, commentaries, cost-effectiveness studies, overviews of systematic reviews, protocols and papers with no identifiable authors. Papers that used informal indirect comparisons (eg, simply compared point estimates with 95% CIs) or did not clearly describe the techniques used to perform the indirect comparison in the title, abstract, introduction or methods sections were also excluded. Two authors (JKB and MT) independently searched and screened all titles and abstracts for eligibility. Discrepancies following full-text review were resolved by a third author (SMC).
The number of papers and cumulative authors was plotted by calendar year and type: methodological contribution, review paper or empirical application; and important social system events (eg, publication of seminal papers) were added to the graph. We then focused exclusively on empirical applications. A proportional Venn diagram was used to illustrate the yield of each database search strategy that contributed to the identification of eligible empirical applications. We abstracted: author(s), journal, year of publication, area of study, primary outcomes (efficacy, safety or both), first and last author institutional affiliations, terminology used to describe methods, and presence and details of network diagrams. If no primary outcome was explicitly stated, all outcomes were considered primary. When multiple diagrams were present, the total number of unique comparators across all network diagrams was taken. Two authors (JKB and EAC) abstracted all the data, and another (MT) verified the data.
Coauthorship network of empirical applications
An Excel macro was used to generate a coauthorship matrix from author names downloaded into Microsoft Excel 2010 from Endnote X5 (Thomson Reuters, 2011). Names of authors presented in multiple forms were collapsed into the most common presentation or, in the event of a tie, the one with more initials. Publication (authors and order) and paper characteristics (country and institutional type ascribed) were imported into R, V.3.3.1 (R Foundation for Statistical Computing, 2016), leveraging RStudio, V.0.99.887 (RStudio, 2009), to generate directed coauthorship networks, and identify components. Coauthorship networks depict authors as ‘nodes’ with ‘ties’ between nodes denoting coauthorship. Directed coauthorship networks clarify network structure by sending ‘ties’ depicted as arrows, from first authors to coauthors. A component is a group of authors connected directly as coauthors on the same paper, or indirectly through a mutual coauthor on separate papers. A disconnected coauthorship network is based on the total number of components. The more components found in a coauthorship network, the more disconnected authors are from each other as a result of isolated publishing. Institutional affiliations and corresponding countries of the first and last authors of each empirical application were used to ascribe credit to each application and the network.16 Institutions were categorised by country and type (academia, government, industry or other). Node size was created proportional to the number of publications by that author. Node colour was created, first based on country affiliation attributed to each paper, and second based on institutional type. The networks were animated by calendar year of publication to visualise growth in application and contributions over time.
Patient and public involvement
No patients or the public were involved in the development and design of this research.
Results
Systematic search
We identified 477 eligible papers: 74 methodological contributions (online supplementary appendix B), 42 review papers (online supplementary appendix C) and 361 empirical applications (online supplementary appendix D) (figure 1); published by 1691 distinct authors between 1997 and 2013. A steady increase in the number of eligible papers was seen over time, and proportionally more were published in recent years (figure 2). Focusing exclusively on the 361 empirical applications, the keyword search strategy identified most applications (n=314, 87%; 30% unique). EMBASE identified the most (n=282, 78%; 6% unique), followed by MEDLINE (n=239, 66%; 3% unique), and relatively few were identified by the Cochrane Database of Systematic Reviews (n=20, 6%; <1% unique) (online supplementary appendix A figure A). The citation search identified an additional 47 (13%) papers outside keyword searches (online supplementary appendix A figure B).
Flow diagram of systematic search results.
Number of publications on indirect comparison meta-analytic methods by year of publication, n=477. Methodological contributions (chequered bar), review papers (horizontal stripes) and empirical applications (solid). Cumulative number of unique authors represented by the solid grey line, n=1689. †Innovators by seminal publication8: Bucher et al 1997 (Canada)8; Lumley 2002 (USA)10; Lu and Ades 2004 (UK).11 Early adopters: §government-sponsored academic groups and health technology and reimbursement assessment agencies (National Institute for Health and Clinical Excellence Guidelines Technical Support Unit 2002 (UK)30; Pharmaceutical Benefits Advisory Committee 2005 (Australia)37 38; Canadian Agency for Drugs and Technologies in Health 2009 (Canada)19; Haute Autorité de Santé 2009 (France)35; Institute for Quality and Efficiency in Health Care 2013 (Germany)36); #independent research organisations: Indirect Treatment Comparisons Good Research Practices Task Force 2011 (Canada, The Netherlands, USA, UK).6 15
The indirect comparison meta-analytic applications were published in 188 different journals. The most common areas of study were cardiovascular disorders (22%), cancers (12%), musculoskeletal disorders (12%), infectious diseases (10%) and psychiatry (9%) (table 2). Sixty-nine per cent of primary outcomes assessed therapeutic efficacy, 25% assessed efficacy and drug safety, and 6% assessed drug safety alone. Of the 361 empirical applications, only 161 (45%) published network diagrams illustrating the direct or indirect comparisons. The median number of interventions compared was 7 (IQR 5–10, min=3, max=145). The most common terminology used was network meta-analysis (38%), followed by mixed treatment comparison (26%), Bucher’s method (24%) and adjusted indirect comparison (21%). The sum of these percentages is greater than 100% due to an overlap in the terminology used. More specifically, 18% (n=65) of all eligible empirical applications used two or more terms to describe the methods used.
Characteristics of empirical indirect comparison meta-analytic applications in the study of drugs, n=361
Coauthorship network of empirical applications
Figure 3 (A: country, B: affiliation) summarises the final coauthorship networks, and online supplementary appendices E and F map the growth of each network by country affiliation and institution type over time. The largest component included 143 (40%) papers and 567 (37%) authors, including innovators Guyatt GH, Lu G and Ades AE (online supplementary appendix D1-143). Of the remaining 128 components, 90 (70%) included only a single paper (25% of all applications made up single-paper components), demonstrating a relatively disconnected network.
Directed coauthorship network of the 361 indirect comparison meta-analytic applications, 129 components, 1513 authors, 2000–2013. The lines represent the relationships (coauthorship) between authors, with arrows directed from first author to coauthors of each paper. Node size is proportional to the number of published articles. (A) Colour based on country: Canada (red), the USA (blue), the UK (yellow), all other Europe (light yellow) and all other regions (white). Authors publishing on papers with more than one country affiliation were coloured based on combinations of the primary colours and white. For example, authors on papers with affiliations from Canada and the USA were coloured purple (a combination of red and blue), while authors on papers affiliated with Canada, the USA and the UK were coloured grey (a combination of red, blue and yellow). (B) Colour based on affiliation type: academic (red), government (yellow), industry (blue) and all other affiliation types (white). Authors publishing on papers with more than one affiliation type were coloured based on combinations of the primary colours and white. For example, authors on papers with affiliation types from academia and government were coloured orange (a combination of red and yellow), while authors on papers affiliated with academic, industry and other were coloured light purple (a combination of red, blue and white).
Early application of these methods started in 2000, with three papers published by 200224–26; and each referencing the innovator paper.8 Authors were from Canada (red) and the USA (blue), and published in isolation of each other (online supplementary appendix E). In 2003, five papers were published in isolation of each other, with two credited to the USA (blue), and three credited to the UK (yellow). The majority referenced innovator Bucher et al,8 yet one paper referenced innovator Lumley.10 By 2004, an increase in collaboration between authors from different countries was noted, with the first multipaper component (France) published in 2004, and the first single-paper component with institutional affiliations from two countries (USA and Belgium) published in 2005. By 2006, another 13 papers were published: 11 papers referenced innovator Bucher with institutional affiliations credited to many countries worldwide (Belgium, Canada, France, Germany, India, USA), and two papers referenced two innovator papers,10 11 with one paper credited to the USA, the UK and Greece, and the other credited to the UK. From 2007 to 2013, we noted an increase in the number of applications published over time, with fastest uptake noted in 2011, and an increase in authors publishing from a broad range of countries depicted by the increase in colours observed in the animated networks (online supplementary appendices E and F and online supplementary files 1 and 2). In particular, a rapid increase in collaboration between authors was noted in 2009, as demonstrated by the merging of smaller components into larger components. Europe led the diffusion with node colours of yellow (UK), light yellow (all other Europe) and combinations of yellow with other primary colours comprising the majority of nodes in the coauthorship network.
Overall, institutional credit was given to 358 unique institutions around the world: 77% of contributions came from academic institutions, 18% from industry, 1% from government and 4% from other institutions (table 3). Europe led the diffusion with 49% of credited papers (22% UK, 27% other); 37% were credited to North America (26% USA, 11% Canada), and 14% to other regions.
Institutional affiliations by country (n=35) and institution type (n=7) for the entire indirect comparison meta-analytic applications network
Discussion
Indirect comparison meta-analytic methods are an important methodological innovation that has become valuable in providing comparative drug effect data in the absence of head-to-head trials. In this paper, we found that uptake was concentrated primarily in Europe (49%) with further contributions from North America (37%). Despite initial development from Canada (1997) and the USA (2002),8 10 our results are not surprising given that refined methods were published by core innovators from the Universities of Bristol and Washington.10 11 Early use of indirect comparison meta-analytic applications predominated from the UK, and may have been the result of an increase in demand by the UK government for more comparative effectiveness research to assist with clinical practice guideline development and to guide drug funding decisions. Indeed, the need for clinical practice guideline development was one of the major reasons for the establishment of the National Institute for Health and Clinical Excellence (NICE) in 1999,27 which has since become a world leader in providing guidance on the clinical and cost-effectiveness of new and established health technologies (including drugs). NICE decisions are made by independent committees of researchers, clinicians, industry and lay representatives; and have included innovator Ades, and early adopters from the NICE Guidelines Technical Support Unit, University of Bristol.5 20 28–30
The steady increase in the use of indirect comparison meta-analytic methods, and effective diffusion to Europe and North America, may also be partially explained by consideration of the five key innovation attributes described in Rogers’ Diffusion of Innovations Model (relative advantage, compatibility, simplicity, trialability and observability).16 The Multi-Parameter Evidence Synthesis (MPES) Research Group (from which the NICE Guidelines Technical Support Unit is based) has offered introductory short courses and workshops to facilitate understanding and application of these methods to health economists, statisticians and policymakers worldwide in collaboration with other academic institutions in the UK (Universities of Sheffield and York) since 2002 (observability, simplicity, trialability).30 Active workshops demonstrating the use of this methodological innovation likely provided a vehicle for peer observation to occur, so that the results and benefits of using this innovation were visible to potential adopters (observability). The provision of sample data sets and statistical code, as well as the integration of these methods into established software and software packages, may have also eased the use of these methods (simplicity), and allowed potential adopters the chance to try using these methods with direct guidance from the innovators and early adopters themselves (trialability). In addition, the MPES Research Group published tutorials and case studies highlighting the advantages of using pairwise, indirect comparison, and network meta-analyses for evidence synthesis (advantage), and highlighting the validity of these methods to inform clinical and policy decision-making (compatibility).28 31–34 We noted rapid uptake since 2011, coinciding with the publication of guidelines and reviews on these methods by health technology assessment and reimbursement agencies (eg, Canadian Agency for Drugs and Technologies in Health (CADTH), Haute Autorité de Santé, Institute for Quality and Efficiency in Health Care, Pharmaceutical Benefits Advisory Committee and Scottish Medicines Consortium) from many countries around the world.19 35–41
Given the economic pressure on payers to better allocate healthcare resources, many regulatory agencies have been calling for the use of comparative effectiveness research to assist in drug accessibility and reimbursement decisions.40 41 In addition, applications focused on drug efficacy tie into payer demands for more cost-effectiveness analyses of newly marketed drugs in comparison with competing or existing therapies. For example, the CADTH has published guidance documents to facilitate best practices in the use of indirect comparison meta-analytic methods to assess clinical and economic value of drugs and other health technologies in Canada, including how to best incorporate these methods to inform clinical parameters in these types of evaluations.19 42 Consequently, many pharmaceutical companies and contract research organisations have started to apply these methods. For example, the International Society for Pharmaceutical Outcomes Research Indirect Comparisons Good Research Practice Task Force adopted methods and statistical code from the MPES Research Group to publish a two-part report to guide researchers, clinicians and policymakers on good research practices for indirect comparisons; given its value and increasing acceptance by regulatory agencies.6 15 Coauthors mainly comprised research experts from pharmaceutical companies and contract research organisation (including JP Jansen who collaborated with innovator AE Ades and coauthors from the MPES Research Group), which may have helped disseminate use of these methods into industry. In addition, publication of this report may partially explain rapid and large uptake from 2011 since coauthors from the two-part report were from multiple countries (Belgium, Canada, the Netherlands, the UK, the USA). We believe that this observation may have been a response to requests by these agencies, as we noted collaboration with core innovators from academia, and an increase in the number of industry-sponsored applications published from 2009.
Our findings demonstrated rapid increase in the use of indirect comparison meta-analytic methods in recent years, with contributions increasing worldwide. With 70% (n=90) of the coauthorship network comprising single-paper components, and 81% (n=1121) of authors having published only a single paper, use of indirect comparison meta-analytic methods has indeed spread to many distinct research groups. However, uptake of these methods has been diffuse and highly disconnected when compared with the diffusion and early uptake of other methodological innovations,16 since many authors are publishing in isolation of each other (ie, smaller, single-paper components). In a prior study that examined the diffusion and early uptake of two confounder summary scores (the disease risk score and high-dimensional propensity score), only 19% and 11% of all eligible applications made up single-paper components in their respective coauthorship networks in comparison with 25% of all indirect comparison meta-analytic applications.16 Rapid and widespread use by academics, and more recently, government and industry, suggests that use of these methods has become diffuse and are no longer in the early stages of adoption, but rather, mainstream and accepted methods. As we also noted a lack of standardisation in the terminology used to describe the indirect comparison meta-analytic methods used, we encourage use of the term, network meta-analysis, as it is clearer than mixed treatment or multiple treatments meta-analysis, which may be assumed to indicate the concomitant administration of two or more drug therapies (eg, adjuvant therapy).
Our results are subject to some limitations. First, our analysis limited the coauthorship of empirical applications to English language papers identified in select bibliographic databases: the Cochrane Database of Systematic Reviews, EMBASE, MEDLINE, Scopus and Web of Science. The limitation of our search to these databases may have resulted in missed articles that were published in other languages, or identifiable in other bibliographic databases, such as Google Scholar, JSTOR, PubMed and RevMan5. Articles that did not clearly describe the techniques used to perform these methods were also excluded, since we could not assume that these methods were used. While we acknowledge that this may have resulted in the exclusion of some applications, we included articles that clearly described these methods in the title, abstract, introduction or methods section to allow for as much inclusion as possible. Consequently, we believe that our systematic search is both comprehensive and robust, as this is the largest and only search completed to date that examines the diffusion of indirect comparison meta-analytic methods in the study of drugs. However, it is worth noting that the term matching-adjusted indirect comparison, an extension of the adjusted indirect comparison which was introduced in 2010 and uses individual patient data from single-comparator RCTs to adjust for differences in patient characteristics across studies, was not considered in our analysis.43 However, six eligible papers were published using this term, and we expect to see an increase in the future.
Second, our study only ascribed country and institutional credit to the first and last authors of each paper. Although the first (principal) and last (often senior) authors traditionally contribute the most, and thus receive the most credit, for papers in the biomedical sciences, other coauthors in the authorship order may also help drive use of novel methods. Consequently, mapping contributions based on first and last authors may have resulted in missed contributions by other coauthors. Nonetheless, inclusion of the second authors in a previous study that examined the diffusion of two confounder summary scores found little impact on country and institutional credit.16
Finally, our work did not examine the quality of eligible empirical articles, or explore the correlation and impact of early diffusion on the quality of indirect comparison meta-analytic methods. Given the large number of authors who published in isolation of each other, it is possible that the degree of interconnectedness between authors in the network may have influenced the quality of eligible applications, as inconsistencies in methodological and reporting quality of indirect comparison meta-analytic methods have been documented.18 Similar to traditional pairwise meta-analysis, limitations related to the quality of the search conducted, quality and heterogeneity of studies included and publication bias can all influence the quality of the study. Uniquely, indirect comparison meta-analytic methods have additional limitations that should be accounted for, such as issues with transitivity and inconsistency of networks, as well as the presentation of results.44 A recent systematic review of network meta-analyses in clinical research demonstrated improvement in methodological and reporting quality over time.45 However, we acknowledge that this is an important area of future research that should be explored.
In conclusion, prior research identified challenges with integrating new statistical methods into practice.46 47 We recently identified the importance of considering the five innovation attributes from Rogers’ Diffusion of Innovations Model to facilitate knowledge translation of new methods for rapid integration.16 In this paper, we used indirect comparison meta-analytic methods to examine the impact of social systems on the diffusion of novel methods. We demonstrated rapid adoption by effective consideration of innovation attributes by innovators, and rapid adoption due to collaboration between innovators from the UK and a large number of early adopters from many countries around the world. Although speculative, and while there are likely multiple reasons for the relatively rapid adoption of these methods, we believe that adoption by government agencies may have contributed to more rapid uptake, and is worth noting; though further research should be explored. We believe that the social system can play a major role in facilitating the adoption of innovative methods, here through regulation, and by the increase in demand by government for more comparative effectiveness research. As many health technology assessment and regulatory agencies have started to call for more evidence synthesis methods to assist in drug accessibility and reimbursement decisions,41 use of indirect comparison meta-analytic methods has become more widely accepted, and will likely continue to be a key tool for policy decision-making. We encourage authors of novel methods to consider the five innovation attributes when integrating new methods into practice (relative advantage, compatibility, simplicity, trialability and observability), with emphasis on early collaboration with potential adopters, such as government regulatory bodies.
Supplementary file 1
Supplementary file 2
Acknowledgments
The authors thank Dave Dubins, PhD, for initial work in creating Excel macros for coauthorship matrix generation; and Gina Matesic, MLIS, for helping with search strategies.
References
Footnotes
Contributors JKB, MT and SMC were responsible for the conception and design of this study. JKB, EAC and MT collected the data and verified the results with SMC. Data analysis was performed by JKB and AXL. JKB led the drafting of the manuscript in consultation with MT and SMC. All authors provided critical feedback and helped shape the research, analysis and manuscript.
Funding JKB was supported in part by the Canadian Institute of Health Research (CIHR) Drug Safety and Effectiveness Cross-Disciplinary Training Program. MT was supported in part by a CIHR Frederick Banting and Charles Best Canada Graduate Scholarship (GSD-11342). EAC was supported in part by a CIHR Health Professional Student Research Award. Participation at CAHSPR was supported by a CAHSPR Student Travel Bursary, and participation at ICPE was supported by an ICPE Travel Scholarship, the Leslie Dan Faculty of Pharmacy Student Experience Fund, and a University of Toronto School of Graduate Studies Conference Grant.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Online technical appendix and supplemental materials are available at [/doi: 10.1136/bmjopen-2017-019110]
Presented at This research was presented at the Canadian Association for Health Services and Policy Research (CAHSPR) Annual Conference in Toronto, ON, Canada, May 2016, and at the International Conference on Pharmacoepidemiology and Therapeutic Risk Management (ICPE) in Dublin, Ireland, August 2016.