Applying the PRECIS-2 tool for self-declared ‘pragmatic’ acupuncture trials: protocol for a systematic review
=================================================================================================================

* Jinwoong Lim
* Hyeonhoon Lee
* Yong-Suk Kim

## Abstract

**Introduction** The pragmatic design has received much attention in the field of acupuncture clinical trials because of insufficient information about the specific effects of acupuncture. However, pragmatism in pragmatic acupuncture trials has not been comprehensively investigated. The PRECIS-2 tool was developed and has been gradually used to design pragmatic trials; therefore, we will apply the PRECIS-2 tool to investigate the pragmatism of pragmatic acupuncture trials in this study.

**Methods and analysis** In this systematic review, self-declared ‘pragmatic’ randomised clinical trials (RCTs) or protocols of self-declared ‘pragmatic’ RCTs investigating acupuncture will be searched and included to be reviewed. MEDLINE, EMBASE, the Cochrane Central Register for Controlled Trials, CINAHL, Allied and Complementary Medicine Database (AMED), China National Knowledge Infrastructure, VIP, WANFANG, Taiwan Periodical Literature Database, KoreaMed, KMbase, Research Information Service System, Oriental Medicine Advanced Searching Integrated System, CiNii and ClinicalTrials.gov for registered trials will be electronically searched from inception to March 2022. Protocols of published RCTs or secondary analysis of RCTs will be excluded. Additionally, no language restriction will be applied. Two authors will independently extract descriptive information and assess the pragmatism of pragmatic acupuncture trials using nine domains of the PRECIS-2 tool and one additional domain—control. Descriptive statistics will be reported for each domain and the overall score, and a one-sample t-test will be used to statistically analyse whether the score is greater than 3 (equally pragmatic and explanatory). The wheel diagrams of the nine domains of the PRECIS-2 tool will be used to demonstrate the pragmatism of the included studies.

**Ethics and dissemination** Ethical approval is not warranted as this study will obtain data from previously reported articles. The results will be disseminated through peer-reviewed journals and conferences.

**PROSPERO registration number** CRD42021236975.

*   COMPLEMENTARY MEDICINE
*   STATISTICS & RESEARCH METHODS
*   Clinical trials

### Strengths and limitations of this study

*   This protocol will be the first to assess the pragmatism of self-declared pragmatic acupuncture trials.

*   The pragmatism of trials will be evaluated using PRECIS-2 tool.

*   Any type of acupuncture, including electroacupuncture and microsystem acupuncture, will be included.

*   Assessing the risk of bias and the quality of reporting of trials is not included in this protocol.

*   Trials with pragmatic intentions will be excluded unless they are self-declared as ‘pragmatic’ in titles, abstracts or manuscripts.

## Introduction

Acupuncture is a treatment modality used in traditional East Asian medicine. It stimulates acupuncture points on the body with acupuncture needles to manage various symptoms and diseases.1 Scientific clinical trials have been conducted to assess the effect of acupuncture on several diseases and address the mechanism of acupuncture treatment.2–4 However, the specific efficacy and placebo effect of acupuncture have not been clearly revealed. Consequently, explanatory clinical trials have been unable to establish the exact efficacy of acupuncture.5 The methodology of acupuncture clinical trials has been continuously discussed, and researchers have tried to report reliable results for decision-makers.6 7 Alternatively, pragmatic acupuncture trials designed to evaluate the effectiveness of acupuncture treatments in real-world practice conditions have been conducted, and several studies have tried to show clinical benefits of acupuncture over conventional treatments even though the mechanism and specific efficacy could not be verified.8

To methodologically assess the pragmatism of trials, the PRagmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) tool has been recently developed,9 and it consists of nine domains: eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome and primary analysis. The tool has been shown to have sound reliability and validity,10 and when it is used retrospectively to assess clinical trials, one additional domain—control—has been suggested.11 Unfortunately, pragmatic acupuncture trials have not been comprehensively assessed with this tool and other tools to investigate the extent of their pragmatic design. In the field of acupuncture trials, the pragmatic design has received much attention; however, the assessment of relevance has not been studied. Therefore, this systematic review aims to investigate the methodological characteristics of pragmatic acupuncture trials using the PRECIS-2 tool and assess whether the trials are designed appropriately to be applied to the real-world environment.

## Methods and analysis

### Design

This study is a protocol of systematic review and follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Protocols guideline online supplemental file 1).12 The results will be a systematic review in accordance with the PRISMA guideline.13

### Supplementary data

[[bmjopen-2021-052861supp001.pdf]](pending:yes)

### Inclusion criteria of the studies in this review

#### Types of studies

Randomised clinical trials (RCTs) and RCT protocols that state it is a pragmatic design, published until March 2022, will be searched and included in this study. The inclusion criteria are (1) RCTs and RCT protocols self-declared as ‘pragmatic’ in title, abstract or manuscript, and (2) RCTs and RCT protocols of interventions that include acupuncture treatment. The exclusion criteria are (1) protocols of RCTs already published with results, (2) secondary analyses of published RCTs and (3) studies that use the word ‘pragmatic’ not in a methodological manner.

#### Type of participants

We will include participants with all the possible conditions or diseases; however, healthy participants will be excluded unless the study is a prevention study.

#### Type of interventions

Any type of acupuncture including manual acupuncture, electroacupuncture and microsystem acupuncture such as auricular acupuncture and acupoints acupressure will be included. RCTs investigating complex interventions without acupuncture will be excluded.

### Information sources and search strategy

MEDLINE, EMBASE, the Cochrane Central Register for Controlled Trials, CINAHL, Allied and Complementary Medicine Database (AMED), four Chinese databases (China National Knowledge Infrastructure, VIP, WANFANG and Taiwan Periodical Literature Database), four Korean databases (KoreaMed, KMbase, Research Information Service System, and Oriental Medicine Advanced Searching Integrated System), CiNii for Japanese literature and ClinicalTrials.gov for registered trials will be electronically searched from inception to March 2022. The research terms for each database are provided in online supplemental file 2. If necessary, appropriate articles will be manually retrieved. Additionally, no language restriction will be applied.

### Selection of studies

Duplicates will be removed before the screening. After reviewing the titles and abstracts, JL and HL will first select the studies and collect the manuscripts of relevant articles. Next, after indexing, the two reviewers will independently review the manuscripts of the articles and include or exclude the articles based on the inclusion/exclusion criteria.

### Data extraction and applying the PRECIS-2 tool to the included studies

General information about the studies, including the first author, publication year, country, intervention used in the experimental and control groups, and primary outcome measures will be extracted by JL and HL. The PRECIS-2 tool will be used to investigate the pragmatic characteristics of the included trials. Ten domains will be used, and two authors will first review 10% of the included articles and discuss the criteria. Based on the criteria, JL and HL will assess the other articles. The two reviewers will independently review the articles and discuss the scores of the PRECIS-2 tool for each article. The following descriptive information and rationale for the scores of 10 domains will be independently extracted and summarised: (1) eligibility criteria, (2) recruitment methods, (3) trial setting and number of centres, (4) organisational information—expertise and resources, (5) intervention delivery protocol and flexibility of the delivery, (6) methods to manage the adherence of participants, (7) follow-up features: the frequency and duration of follow-ups and additional data collection, (8) primary outcome measures, (9) primary analysis methods and (10) intervention in control groups. If there is inconsistency, a discussion will be held with Y-SK. If there is inconsistency after the discussion, JL’s and HL’s mean score will be used.

Basically, the PRECIS-2 tool consists of nine domains; however, Zwarenstein *et al* recommended one additional domain—control—when the PRECIS-2 tool is retrospectively applied to clinical trials in systematic reviews.11 We will, therefore, use nine domains based on the recommendation of the PRECIS-2 tool9 with the control domain. When the control group is sham-controlled and considered as completely explanatory, the score is 1, and when the control group involves usual care without any discipline of treatments and is considered as completely pragmatic, the score is 5. If there is uncertainty regarding a domain, the score will be left blank, as suggested by Loudon *et al*.9 The flow chart of this study is shown in figure 1.13

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/12/4/e052861/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/12/4/e052861/F1)

Figure 1 
Flow chart of the study.

### Data analysis plan

Scores for each domain and the overall score for each article will be summarised using descriptive statistics, including mean/median, measure of variance, interquartile range and percentage. The wheel diagrams of the nine domains of the PRECIS-2 tool will be used to show the extent of the domain’s pragmatic design (figure 2).9

![Figure 2](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/12/4/e052861/F2.medium.gif)

[Figure 2](http://bmjopen.bmj.com/content/12/4/e052861/F2)

Figure 2 
Wheel diagram of the nine domains of the PRECIS-2 (PRagmatic-Explanatory Continuum Indicator Summary 2) tool.9

According to Loudon *et al*,9 for the domains of flexibility: delivery, flexibility: adherence and control, if there are more than two groups, each group needs to be scored separately. Therefore, we will score each group separately; however, when it comes to data analysis, we will use the score of the group that was more related to acupuncture, and if all groups are related to acupuncture, we will use the highest score to reflect the potential pragmatism of the trial. A one-sample t-test will be applied to statistically analyse whether the score is greater than three (equally pragmatic and explanatory), and p<0.05 (null hypothesis: the score is not greater than three) will be considered statistically significant.

### Risk of bias assessment

Since this study is a systematic review on the methodology of trials using the PRECIS-2 tool and is not about the clinical outcome, risk of bias will not be assessed.

### Patient and public involvement

Patients and the public are not directly involved in this study as we will use data from already published articles.

### Ethics and dissemination

Since we will obtain data from already published articles, ethical approval is not required. We plan to publish the results of the study through peer-reviewed journals and conferences and share the findings with the relevant trialists and researchers.

## Discussion

The aim of this systematic review is not to investigate the efficacy or effectiveness of interventions but to investigate the methodological issue of acupuncture clinical trials. To the best of our knowledge, this study is the first systematic review protocol to comprehensively deal with the pragmatic design of acupuncture clinical trials. Previous systematic reviews used PRECIS-2 evaluating interventions of integrative medicine14 and Chinese herbal medicine15; however, this protocol will primarily focus on acupuncture and include various diseases or conditions. Furthermore, Dal-Ré *et al* reported that some self-labelled pragmatic trials showed explanatory features,16 and Neta and Johnson argued using PRECIS-2 tool to enhance ‘real-world’ evidence.17 This protocol will estimate the status of self-declared pragmatic acupuncture trials using PRECIS-2 tool.

As Loudon *et al* reported,9 defining a trial as pragmatic or explanatory is on a continuum rather than dichotomous. Trials having a pragmatic intention could be explanatory in some respects. PRECIS-2 has been developed considering the characteristics, and the results of this study will show the summary of the characteristics of pragmatic acupuncture trials and the sufficient and deficient components of pragmatic design in acupuncture trials on the continuum. Based on these results, although we will not be able to suggest the clinical advantages or disadvantages of acupuncture, we will be able to suggest the proper direction for future pragmatic trials, which will clearly reveal the advantages or disadvantages of acupuncture in the future.

The limitations of this study are as follows: (1) This study will not assess the risk of bias and reporting quality. Two previous studies reported the risk of bias and reporting quality of included trials with the results of PRECIS-2 assessments.18 19 However, since this study will evaluate the methodological features of trials in terms of pragmatism rather than reporting the clinical effect of interventions or quality of trials, assessing the risk of bias and reporting quality would be non-essential. (2) Search terms that could mean pragmatic intention are not included in this study. Previously, two studies assessed ‘self-labelled’ or ‘self-declared’ pragmatic trials, and used additional search terms including ‘practical’, ‘comparative effectiveness’ or ‘naturalistic’16 19; however, this study will include trials self-declared as ‘pragmatic’ and other terms will not be included in the search strategy.

## Ethics statements

### Patient consent for publication

Not applicable.

## Footnotes

*   JL and HL contributed equally.

*   Contributors JL and HL contributed equally to this work. JL and HL conceptualised this study and drafted the manuscript. Y-SK supervised this study. All authors approved the publication of the protocol.

*   Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

*   Competing interests None declared.

*   Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

[http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.  Kaptchuk TJ. Acupuncture: theory, efficacy, and practice. Ann Intern Med 2002;136:374–83.[doi:10.7326/0003-4819-136-5-200203050-00010](http://dx.doi.org/10.7326/0003-4819-136-5-200203050-00010)pmid:http://www.ncbi.nlm.nih.gov/pubmed/11874310
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-136-5-200203050-00010&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11874310&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000174095300005&link_type=ISI) 

2.  Kim SK, Bae H. Acupuncture and immune modulation. Auton Neurosci 2010;157:38–41.[doi:10.1016/j.autneu.2010.03.010](http://dx.doi.org/10.1016/j.autneu.2010.03.010)pmid:http://www.ncbi.nlm.nih.gov/pubmed/20399151
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.autneu.2010.03.010&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20399151&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

3.  Zhuang Y, Xing J-jing, Li J, et al. History of acupuncture research. Int Rev Neurobiol 2013;111:1–23.[doi:10.1016/B978-0-12-411545-3.00001-8](http://dx.doi.org/10.1016/B978-0-12-411545-3.00001-8)pmid:http://www.ncbi.nlm.nih.gov/pubmed/24215915
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/B978-0-12-411545-3.00001-8&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=24215915&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

4.  Kelly RB, Willis J. Acupuncture for pain. Am Fam Physician 2019;100:89–96.pmid:http://www.ncbi.nlm.nih.gov/pubmed/31305037
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

5.  Vase L, Baram S, Takakura N, et al. Specifying the nonspecific components of acupuncture analgesia. Pain 2013;154:1659–67.[doi:10.1016/j.pain.2013.05.008](http://dx.doi.org/10.1016/j.pain.2013.05.008)pmid:http://www.ncbi.nlm.nih.gov/pubmed/23707680
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.pain.2013.05.008&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23707680&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000323600000023&link_type=ISI) 

6.  Liu W, Cohen L. Overcoming barriers for clinical research of acupuncture. Med Acupunct 2020;32:348–51.[doi:10.1089/acu.2020.1480](http://dx.doi.org/10.1089/acu.2020.1480)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33362885
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

7.  Witt CM, Manheimer E, Hammerschlag R, et al. How well do randomized trials inform decision making: systematic review using comparative effectiveness research measures on acupuncture for back pain. PLoS One 2012;7:e32399.[doi:10.1371/journal.pone.0032399](http://dx.doi.org/10.1371/journal.pone.0032399)pmid:http://www.ncbi.nlm.nih.gov/pubmed/22389699
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1371/journal.pone.0032399&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22389699&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

8.  Cardini F, Wade C, Regalia AL, et al. Clinical research in traditional medicine: priorities and methods. Complement Ther Med 2006;14:282–7.[doi:10.1016/j.ctim.2006.07.003](http://dx.doi.org/10.1016/j.ctim.2006.07.003)pmid:http://www.ncbi.nlm.nih.gov/pubmed/17105698
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.ctim.2006.07.003&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17105698&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000242708500007&link_type=ISI) 

9.  Loudon K, Treweek S, Sullivan F, et al. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 2015;350:h2147.[doi:10.1136/bmj.h2147](http://dx.doi.org/10.1136/bmj.h2147)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25956159
    
    [FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTAvbWF5MDhfMS9oMjE0NyI7czo0OiJhdG9tIjtzOjI2OiIvYm1qb3Blbi8xMi80L2UwNTI4NjEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. Loudon K, Zwarenstein M, Sullivan FM, et al. The PRECIS-2 tool has good interrater reliability and modest discriminant validity. J Clin Epidemiol 2017;88:113–21.[doi:10.1016/j.jclinepi.2017.06.001](http://dx.doi.org/10.1016/j.jclinepi.2017.06.001)pmid:http://www.ncbi.nlm.nih.gov/pubmed/28603007
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

11. Zwarenstein M, Thorpe K, Treweek S, et al. PRECIS-2 for retrospective assessment of RCTs in systematic reviews. J Clin Epidemiol 2020;126:202–6.[doi:10.1016/j.jclinepi.2020.06.023](http://dx.doi.org/10.1016/j.jclinepi.2020.06.023)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32565215
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

12. Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350:g7647.[doi:10.1136/bmj.g7647](http://dx.doi.org/10.1136/bmj.g7647)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25555855
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1136/bmj.g7647&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=25555855&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

13. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.[doi:10.1136/bmj.n71](http://dx.doi.org/10.1136/bmj.n71)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33782057
    
    [FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE1OiIzNzIvbWFyMjlfMi9uNzEiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTIvNC9lMDUyODYxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

14. Chan KW, Lee PW, Leung CP-S, et al. Pragmatic clinical trial design of integrative medicine (practice): a focus group series and systematic review on trials of diabetes and kidney disease. Front Med 2021;8:668913.[doi:10.3389/fmed.2021.668913](http://dx.doi.org/10.3389/fmed.2021.668913)pmid:http://www.ncbi.nlm.nih.gov/pubmed/34513860
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

15. Lu L, Zhou L, Dong J, et al. The application of PRECIS-2 ratings in randomized controlled trials of Chinese herbal medicine. Oncotarget 2017;8:107002–10.[doi:10.18632/oncotarget.22204](http://dx.doi.org/10.18632/oncotarget.22204)pmid:http://www.ncbi.nlm.nih.gov/pubmed/29291006
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

16. Dal-Ré R, Janiaud P, Ioannidis JPA. Real-world evidence: how pragmatic are randomized controlled trials labeled as pragmatic? BMC Med 2018;16:49.[doi:10.1186/s12916-018-1038-2](http://dx.doi.org/10.1186/s12916-018-1038-2)pmid:http://www.ncbi.nlm.nih.gov/pubmed/29615035
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/s12916-018-1038-2&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=29615035&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

17. Neta G, Johnson KE. Informing real-world practice with real-world evidence: the value of PRECIS-2. BMC Med 2018;16:76.[doi:10.1186/s12916-018-1071-1](http://dx.doi.org/10.1186/s12916-018-1071-1)pmid:http://www.ncbi.nlm.nih.gov/pubmed/29783964
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

18. Robinson NB, Fremes S, Hameed I, et al. Characteristics of randomized clinical trials in surgery from 2008 to 2020: a systematic review. JAMA Netw Open 2021;4:e2114494.[doi:10.1001/jamanetworkopen.2021.14494](http://dx.doi.org/10.1001/jamanetworkopen.2021.14494)pmid:http://www.ncbi.nlm.nih.gov/pubmed/34190996
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom) 

19. Hohenschurz-Schmidt D, Kleykamp BA, Draper-Rodi J, et al. Pragmatic trials of pain therapies: a systematic review of methods. Pain 2022;163:21–46.[doi:10.1097/j.pain.0000000000002317](http://dx.doi.org/10.1097/j.pain.0000000000002317)pmid:http://www.ncbi.nlm.nih.gov/pubmed/34490854
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F12%2F4%2Fe052861.atom)