Article Text

Original research
Methodological standards in the design and reporting of pilot and feasibility studies in emergency medicine literature: a systematic review
  1. Onlak Ruangsomboon1,
  2. João Pedro Lima2,
  3. Mohamed Eltorki3,4,
  4. Andrew Worster2,5
  1. 1Department of Emergency Medicine, Faculty of Medicine, Siriraj Hospital, Mahidol University, Bangkok, Thailand
  2. 2Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
  3. 3Department of Pediatrics, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
  4. 4Department of Pediatrics, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
  5. 5Department of Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
  1. Correspondence to Onlak Ruangsomboon; doctor.mo{at}yahoo.com

Abstract

Objective Pilot and feasibility studies are intended to ensure that subsequent randomised controlled trials (RCTs) are feasible, economical and rigorous, especially in a challenging research environment such as emergency medicine (EM). We aimed to evaluate the methodological quality in conducting and reporting randomised pilot and feasibility studies in the EM literature and propose recommendations to improve their quality.

Design Methodological systematic review.

Data sources and eligibility We searched MEDLINE and Embase (2018–29 September 2023) for pilot or feasibility RCTs published as full texts in the five top-ranked and other first-quartile EM journals according to Scimago.

Data extraction and analysis We assessed their methodological features and reporting quality primarily based on the Consolidated Standards of Reporting Trials (CONSORT) extension.

Results A total of 24 randomised trials identified as pilot (n=13), feasibility (n=3) or both (n=8) were included. At least one feasibility outcome was assessed in 9 trials (feasibility trials), while 15 others only focused on treatment efficacy (efficacy trials). Only three (12.5%) studies progressed to the main trials. Among 12 feasibility trials, 55.6% reported their outcomes with uncertainty estimates, and 33.3% had clear progression criteria. Efficacy trials tended to draw clinical implications on their results. Studies from the five top-ranked journals had better methodological and reporting quality than those from other first-quartile journals.

Conclusion Main methodological concerns for pilot and feasibility studies in first-quartile EM literature include misconceptions, misuses and suboptimal design and reporting quality. These issues were more prominent in lower-ranked first-quartile journals. Our findings highlight the need for resources and training for researchers, journal editors and peer reviewers on the value, objectives and appropriate conduct of pilot and feasibility studies. The conceptual framework and standardised methodological components should be emphasised. EM journals should reinforce the reporting standards and support their publication. These actions can lead to more methodologically rigorous pilot and feasibility studies in EM.

PROSPERO registration number CRD42023468437.

  • STATISTICS & RESEARCH METHODS
  • ACCIDENT & EMERGENCY MEDICINE
  • Randomized Controlled Trial

Data availability statement

No data are available. Not applicable.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Robust systematic review methods are performed to addresses a major methodological issue.

  • Comprehensive aspects of the design, conduct and reporting of pilot and feasibility studies are assessed and evaluated.

  • The search and inclusion criteria are not very sensitive, though it should not affect the overall results of the review.

Introduction

Pilot and feasibility studies (PFSs) contribute significantly to the health research community by offering data necessary for the design and successful conduct of larger-scale studies, especially phase III randomised controlled trials (RCTs).1 PFSs can increase the chance of success of the main study and, most importantly, they can reduce the potential waste of time and resources.2 In evaluating complex interventions, it is recommended that the feasibility and acceptability of interventions should be assessed and the trial design should be evaluated before making decisions about the study progression.3 Many interventions evaluated in emergency medicine (EM) clearly meet the definition of complex intervention, with the complexity mainly arising from the interaction between the intervention’s components and the clinical emergency context in which it is being implemented.3 Consequently, PFSs are essential for the success of full-scale definitive RCTs in a challenging environment such as EM. Despite their notable importance, academic research training has not paid enough attention to them.2 Their methodological and reporting standards have not been widely implemented, and many misuses and misconceptions are still presented.4

Since a pilot and a feasibility study both aim at the same goal, they are often considered synonymous.5 However, a conceptual framework developed by distinguished methodologists suggests that pilot studies are a subset of feasibility studies with similar objectives but with a specific design feature and similar methods to the definitive trial, they thus agree that the two terms should not be considered mutually exclusive.5–7 PFSs performed prior to a full-scale RCT can be randomised or non-randomised, though a pilot study whose design matches that of the full trial would normally be randomised.6

In many other clinical areas, reviews have shown that the methodological standards and reporting quality of published PFSs were still suboptimal, with each research field having different issues and obstacles.8–13 No reviews, however, have been performed to assess such standards on published studies in EM. Therefore, this systematic review aimed to evaluate the methodological and reporting standards of randomised PFSs conducted in adults, in an emergency department (ED) setting and published in top-ranked EM journals. The ultimate objective was to propose recommendations based on the deficiencies identified in our review to improve the quality of PFSs.

Methods

Trial identification

The report of this study followed the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) 2020 reporting guidelines.14 Published literature in first-quartile (Q1) EM journals based on Scimago15 with one or both of the words ‘pilot’ and ‘feasibility’ in the title or abstract was identified by searching MEDLINE (2018‒29 September 2023) and Embase (2018‒29 September 2023) via Ovid. We started our search beginning in 2018 to allow for time to implement the conceptual framework for PFSs and the corresponding Consolidated Standards of Reporting Trials (CONSORT) extension in 2016.2 16 Based on the five top-ranked EM-based journals according to Scimago,15 Resuscitation, Academic Emergency Medicine (ACEM), Scandinavian Journal of Trauma, Resuscitation, and Emergency Medicine (SJTREM), Annals of Emergency Medicine (AnnalsEM) and Western Journal of Emergency Medicine (WJEM) were included as search terms. We also included studies published in other EM journals within the Q1 to be compared with the five top-ranked journals. We limited retrieval to those published between 2018 and 29 September 2023, written in English, and enrolled patients from the ED. The search strategy and results are presented in Appendix. Duplicates were removed using Covidence and manual deduplication. Only randomised PFSs involving adult patients in the ED were included, and those that were not original articles or that were conference abstracts without retrievable full texts were excluded. For this review, we did not delineate between PFSs despite the consensus definition, and we employed a more restricted definition focusing on only external pilot studies performed prior to full-scale RCTs.5 6 17 In searching for the corresponding definitive trials, we performed a literature search of the included PFSs’ titles, key terms, citations and authors’ names in the same databases and clinicaltrials.gov.

Study screening and data collection processes

Two reviewers (OR and JPL) independently and in duplicate screened abstracts and full texts, with discordances resolved by a third reviewer (ME). They also independently and in duplicate extracted trial characteristics, and their methodological and reporting aspects. Discordances at this stage were adjudicated by a senior reviewer (AW). We used Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia) for all the review processes.

Trial characteristics

We recorded general and specific trial characteristics related to PFSs. The study type was categorised based on the authors’ definition (pilot, feasibility or both). Feasibility outcomes were categorised according to Thabane et al2 into the four following domains.

Process

The feasibility of the study protocol; appropriateness of the inclusion and exclusion criteria, recruitment and consent rate; retention, adherence and follow-up rate; randomisation procedure and blinding; acceptability and feasibility of the intervention; selection of the primary outcome; preparation and appropriateness of the interventions and instruments used for outcome measurements.

Resources

The capacity of study centres and researchers, that is, willingness and capacity, length of time to obtain consent, apply study intervention, and collect study data, and the required training and number of researchers.

Management

Potential human and data management problems.

Scientific

Information to guide the sample size for the main trial; treatment safety and dose–response.

Methodological and reporting standards

We evaluated the methodological quality of the included trials using a list of components adapted from Arain et al5 and Shanyinde et al18 and developed using the CONSORT extension16 as a guide (table 1). We defined the authors’ conclusion about the feasibility of future definitive trials into three categories: proceed without changes, proceed with modifications and not proceed.

Table 1

Reporting and methodological standards for pilot and feasibility studies

Data analysis

We analysed all data using descriptive statistics. We categorised included trials into two groups: those from the five top-ranked journals and those from the other Q1 journals excluding the top five, in order to compare their standards. Furthermore, we defined trials that only assessed efficacy outcomes as efficacy trials and those with at least one feasibility outcome as feasibility trials. Some of the characteristics and methodological components were compared descriptively between these two categorisations.

Patient and public involvement

None.

Results

Search results

Of the 1745 citations retrieved, 1281 articles remained after the removal of duplicates. We assessed 345 full texts and included 24 pilot and feasibility RCTs in the final analysis.19–42 The majority of studies excluded at the title and abstract screening stage were published as conference abstracts (n=86). We also further excluded 81 adult trials in non-Q1 EM journals and 34 paediatric trials, as they were not the focus of this review. Details of study screening and reasons for exclusion are presented in figure 1.

Figure 1

The PRISMA flow chart of study selection and inclusion. ED, emergency department; PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analyses; Q1 EM, first-quartile emergency medicine; RCT, randomised controlled trial.

Trial characteristics

The summary of included trials’ characteristics with methodological and reporting standards is presented in table 2. Their individual characteristics are elaborated in online supplemental appendix table 1. Among 24 included trials, 13 were from the five top-ranked journals (Resuscitation (n=1), ACEM (n=9), SJTREM (n=1), AnnalsEM (n=1) and WJEM (n=1)), and 11 were from other Q1 journals (American Journal of Emergency Medicine (n=7), Emergency Medicine Australasia (n=2), Internal and Emergency Medicine (n=1) and International Journal of Emergency Medicine (n=1)). The majority were single-centred (87.5%), published after 2019 (70.8%) and from the USA (50.0%). Most defined their studies as ‘pilot’ (54.2%), especially those from other Q1 journals. The overall sample sizes ranged from 22 to 272. Counselling/education programme and drug/intravenous fluid were the most common types of intervention assessed (both 29.2%), especially among top-ranked journals. More trials from top-ranked journals reported blinding in their trials and registered their trials as pilot/feasibility than those from other Q1 journals (61.5% vs 18.2% and 76.9% vs 27.3%, respectively).

Table 2

Trial characteristics and methodological and reporting standards

Methodological and reporting standards

Most trials calculated their sample size based on efficacy outcomes (41.7%) or did not calculate or mention sample size calculation at all (37.5%). Of all included trials, 15 (62.5%) were efficacy trials that included no feasibility components other than hypothesis testing of efficacy outcomes, and 9 (37.5%) were feasibility trials assessing at least one feasibility outcome. Most of these feasibility trials were from top-ranked journals. Only two trials (8.3%) reported having endorsed the CONSORT extension statement, and three (12.5%) progressed to the main study, all of which were from the five top-ranked journals.

Among the nine feasibility trials, five (55.6%) reported the results appropriately with uncertainty estimates (such as CIs), and three (33.3%) had clear progression criteria with justification and appropriately made conclusions based on their criteria (table 3). Almost all these trials were from top-ranked EM journals. Four studies (44.4%) suggested that future trials should proceed without changes, while the other four (44.4%) recommended proceeding with some protocol modifications (on recruitment, eligibility criteria, engagement and delivery methods of intervention and outcome assessment) and one (11.1%) did not recommend proceeding because their aim was not to inform future trials. The most common primary feasibility outcome category was ‘process’ (66.7%). The full list of feasibility outcomes evaluated can be found in table 3. Overall, only the process and scientific domains were evaluated.

Table 3

Methodological and reporting standards of feasibility trials

A total of 20 trials (83.3%) performed hypothesis testing on efficacy outcomes; five were feasibility, and 15 were pure efficacy trials (table 4). The authors of efficacy trials tended to support the intervention even though the results were not significant. There were relatively equal number and proportion of trials that drew clinical implications to support the intervention among those from the five top-ranked and other Q1 journals.

Table 4

Methodological and reporting standards of pilot and feasibility trials with hypothesis testing of efficacy outcomes

Discussion

PFSs play a pivotal role in successfully completing phase III RCTs, as they provide necessary information to evaluate the adequacy of the planned methods and procedures to avoid the potential futility of conducting large, expensive, yet unfeasible RCTs.43 The key concepts of PFSs have been well established and disseminated, with guidelines for reporting and suggestions to improve their scientific rigour.16 43 Although there is a growing number of published PFSs in the literature, many reviews in multiple clinical areas have identified weaknesses in their conduct and reporting.8–13 We found concordant results in EM literature in this review, with mostly suboptimal methodological and reporting quality, even among Q1 EM journals.

Major issues for PFSs in EM literature

Publication bias

This review identified only 24 pilot/feasibility RCTs in Q1 EM journals during the past 5 years. This number is much lower than that of other clinical areas,9 10 12 44 which could have been because our inclusion was not sensitive enough, or there were merely fewer clinical trials in EM. In either case, we might face the same challenges in other clinical areas; that is, many of these PFSs never get published.2 5 17 In fact, many conference abstracts were excluded at the screening stage, with only less than half (44.2%) going on to have their full texts published. One possible reason behind this is that most PFSs are poorly designed with no clear feasibility objectives with an emphasis on statistical significance.2 Another possible reason was that EM journals do not support the publication of such trials. Most of the included trials were from two out of nine journals, and more PFSs were published from the five top-ranked (13 articles) compared with other Q1 journals (11 articles from 20 journals). More importantly, we found that the methodological and reporting quality of PFSs published in the five top-ranked journals was more robust than that of studies from other Q1 journals in many respects, highlighting the need to emphasise rigorous methods and quality in reporting among lower-ranked EM journals to improve the quality of the overall EM literature.

Recommendation

EM trialists should understand the rigorous design and conduct of PFSs. They should also follow established guidelines for PFSs reporting, such as the CONSORT extension statement, to improve the overall quality of published articles. Moreover, they should acknowledge that they, as researchers, have both ethical and scientific obligations to publish their studies. EM journals should also recognise the importance of these studies and encourage their publication. To guide authors, journals should implement clear PFS-specific instructions as they do with other study designs. In this way, both the quality and quantity of well-designed, well-executed, and well-reported PFSs can be enhanced among the whole EM research community.

Hypothesis testing of treatment efficacy

The most common misuse of pilot trials is to perform hypothesis testing for treatment efficacy and conclude whether an intervention is effective or not.4 This analysis is inappropriate, especially when no formal power calculations were carried out, such statistical analyses are thus most likely underpowered.2 17 45 In our review, the majority (83.3%) of all included trials performed statistical comparisons of treatment effects. Furthermore, all efficacy trials drew clinical implications on their efficacy results, with some even promoting the use of the intervention when statistical or clinical significance was not met. These efficacy results may lead to misleading interpretations, especially when no formal power calculation was performed. Additionally, a relatively equal proportion of trials from both top-ranked and lower-ranked Q1 journals drew clinical implications on intervention efficacy hypothesis testing, suggesting that this issue probably extends throughout the EM literature.

Recommendation

PFSs should only report group estimates or effect estimates with CIs. Hypothesis testing using inferential statistics should not be performed, and these results should not be interpreted based on p-values from underpowered analyses.46 Researchers may use the variance around the effect estimates to inform the sample size of the main trial. Data should be presented or shared in sufficient detail that would allow future systematic reviews and meta-analysis to extract. Also, the authors should not draw strong implications, such as superiority or no effects, especially in the conclusion. ‘Potential efficacy’ may be declared when the CI around the treatment effect estimate covers the predefined minimal clinically important difference (MCID).8 Moreover, the authors should clearly state that these analyses were exploratory in nature and mandate future confirmatory trials.

Assessment of safety and tolerability

PFSs, with often small sample sizes, cannot provide definitive information on the safety and tolerability of the intervention, especially when none is demonstrated.4 They may be able to help detect serious adverse events should they arise, but the rate always needs to be reported with CIs.4 In this review, 13 studies (54.2%) reported safety outcomes, but less than half stated the relevant limitations. Some concluded their safety profile as ‘no difference in adverse effects’, which is misleading.

Recommendation

Adverse effects should be monitored in PFSs, the same as any RCTs. However, authors should explicitly declare that the trial is underpowered to detect between-group differences or any rare adverse effects. On the contrary, if the CI around the harm effect estimate lies beyond the upper limit for safety, the authors should only report ‘potential harm’ instead of addressing certainty in their results, as any estimate from an underpowered study can be unreliable.8

Trial registration and reporting

Pilot trials, specifically randomised PFSs, should follow the same guidelines and requirements as full-scale RCTs, including trial registration, to minimise publication bias.8 In this review, however, not all published pilot trials provided trial registration information. This was surprising as these unregistered RCTs were published in Q1 EM journals, in which strict CONSORT standards should have been implemented. Furthermore, some trials were not registered as ‘pilot’ or ‘feasibility’ studies. Moreover, only a few reported having complied with the CONSORT extension for PFSs, even though the extension was published almost 2 years before our search start date. One clear consequence is that feasibility outcomes were not reported and interpreted appropriately. As mentioned in the previous topic, results should be reported with uncertainty estimates, and this uncertainty should be reflected in the final report. The reporting of the results has implications for the design of future trials and the generalisability of the feasibility results to other settings.16 43 In this review, only about 50% of feasibility trials reported uncertainty estimates of their feasibility outcomes. Furthermore, even a smaller number of trials reported having clear progression criteria. Appropriate and transparent use of progression criteria could offer clarity in delivering unbiased decisions on whether to proceed to a definitive trial or identify feasibility issues that can be modified. However, we found trials that concluded to proceed without changes (n=3) and proceed with changes (n=2) that did not have predefined progression criteria, so it was unclear how the authors made those decisions and if there was any bias associated with their conclusions.

Recommendation

Randomised PFSs should be registered and indicate that they are pilot or feasibility in nature. The design should also be clearly described in publications, ideally in the title, for greater transparency.44 The CONSORT extension checklist should be implemented in the journals’ editorial guidelines for submission specifically for PFSs. This would reinforce the authors to adhere to these criteria and set the appropriate standards for PFSs published in EM literature. As stated in the CONSORT extension, PFSs should have feasibility objectives always accompanied by uncertainty estimates and clear a priori progression criteria with conclusions made accordingly. The progression criteria should involve all, not just one, important trial components assessed, with decisions made considering both point estimates and uncertainty estimates of feasibility outcomes.

Sample size calculation

Generally, a justification for the sample size chosen is required for PFSs. Some may employ a CI approach for feasibility outcomes.2 47 48 Several simulation studies recommended different rules-of-thumb based on varying precisions of effect sizes of the expected outcomes.49 50 A formula has also been proposed for sample size estimation of PFSs aiming to detect problems in a trial, such as inclusion/exclusion criteria.51 Although effect sizes and uncertainty estimates derived from PFSs can be used to guide sample size calculation of the main trial,2 implications on these estimates should be made with extreme caution due to their considerable variability.2 3 17 45 Also, PFSs are usually too small to estimate parameters required for the definitive trial if its design is a clustered RCT.52 In this review, not all trials justified how they defined their sample size and many drew strong implications on their efficacy estimates.

Recommendation

A pilot study should have a clear justification for the sample size chosen. If preliminary estimates from PFSs are used to calculate the sample size for the main trial, MCID should be considered and the sample size estimation method that takes into account the uncertainty of the effect estimates should be endorsed.2 4 45 46 53

Misconceptions about pilot studies

Many researchers generally identify their studies as ‘pilot’ when they have limited resources available.2 7 Some considered designating their trials as ‘pilot’ would increase their chance of being funded or successfully publish their studies.5 11 Others redefined their trials a posteriori because the journal demanded so to caution readers of the uncertainty in the results.54 In EM literature, similar issues exist. We retrieved more pilot than feasibility trials, similar to previous reviews.5 10 18 Most trials named ‘pilot’ by the authors were standalone efficacy trials. In contrast, most feasibility trials appropriately employed feasibility objectives, suggesting little misuse of this particular term. Overall, these findings imply that EM researchers, and perhaps journals, might have a misconception of the definition of pilot RCTs. It seems that most consider a pilot RCT as a smaller and underpowered version of the definitive trial or the first trial evaluating that particular research question.

In addition, the primary objective of conducting PFSs should be to inform the design and ensure the feasibility of the full-scale RCTs. Nevertheless, despite most included trials stating that future definitive trials were feasible or should be performed, only three (12.5%) led to full-scale trials (one published and two ongoing). One of the possible reasons explaining the small proportion of progressing trials was that this review was conducted shortly after the included PFSs were published; therefore, it may be unlikely that the authors could have initiated or completed the definitive trials by this time. Also, despite multiple measures adopted to search for subsequent full-scale trials, we could have still missed some unidentified definitive trials since we did not contact the trial authors. Future comprehensive reviews of published definitive trials should be performed to evaluate their quality and assess whether appropriate PFSs were carried out beforehand and how they influenced the design of the definitive trials. Nonetheless, a similar low rate of progression to definitive trials was also seen in reviews of other clinical fields.5 9 44 55 Other issues reported by authors of pilot studies were the lack of funding, recruitment inadequacies, personnel change-over and that the pilot trials had sufficiently answered the research questions.9 55 The authors could also have refrained from publishing the pilot trials but instead added the pilot sample into the main trial to save time and cost, a situation easily detected and potentially preventable by public trial registration. Similar issues may also be presented in EM, although authors’ survey should be performed to better understand the problem.

Recommendation

Misconceptions about the true definition and objectives of PFSs, especially a ‘pilot’ study, should be addressed to the EM research community. The primary objective of PFSs should be to inform the main trial. Trialists should not believe in efficacy outcomes from PFSs and should be encouraged to proceed to definitive trials to confirm the results, unless feasibility issues do not allow.

Under-recognised trial aspects

It is interesting and surprising to observe that the resources and management domains of feasibility objectives were not evaluated in any of the included trials, unlike in other clinical areas,10 56 even though several included PFSs involved complex procedures and interventions, where resources and management may be an issue for successful full-scale trials.

Another crucial trial aspect that might not have been emphasised enough is the process of informed consent and how researchers disclose the nature of PFSs to potential participants.2 45 These ethical issues are challenging in EM, where obtaining informed consent often occurs in a time-dependent and busy environment that usually involves patients with emergency conditions. In this review, we found that some trials employed deferred consent, and no trials explicitly stated the nature of their study accordingly in their informed consent forms.

Recommendation

We encourage EM trialists to further evaluate all relevant feasibility domains, especially in resource-demanding trials and where data management may be a concern, such as those involving life-saving procedures. EM trialists should understand and abide by the highest ethical standards when performing PFSs, as with any other research study. Informed consent should be obtained whenever possible, and the process should be transparent with clear definitions of the study’s nature, rationale, objectives and criteria for success.2 The consent process should make it clear that the study being proposed is not one looking at treatment efficacy, in order to correct possible false expectations from those being asked to participate.57 A deferred or implied consent should be considered only in conditions that, without such a procedure, recruitment might not be possible and RCTs cannot be performed; it should not be implemented to expedite trial duration and process. PFSs in EM research involving emergency or life-threatening conditions may be required to prove if such a scenario is applicable.

Limitations

This systematic review, though robust in its methods, has limitations. First, we searched for articles with the term ‘pilot’ or ‘feasibility’ only in the title or abstract since we wanted to focus on those explicitly addressed as such. Second, we only included Q1 EM journals and only added the five top-ranked journal names in the search term among over 100 EM-related journals.15 Third, we excluded paediatric trials and those not conducted in the ED as we aimed to focus specifically on ED adult trials, and paediatric trials and paediatric EM literature may have other distinct aspects not considered nor included in this review. Therefore, our search results might not have been representative of Q1 journals or the whole EM literature. Moreover, the full methodological and reporting quality was not performed. We could have employed other tools, such as the Cochrane Risk of Bias V.2.0 tool and the full CONSORT extension checklist, but they were not the main objectives of this review and might not be relevant to all PFSs. Also, we preferred to focus on specific essential elements. Nonetheless, a more comprehensive review might provide a better picture of the trials’ overall quality and reporting completeness.

Conclusion

PFSs play an important role in health research in providing information for the planning and justification of full-scale RCTs. We found that the methodological and reporting quality of randomised PFSs published in high-ranking Q1 EM journals was below standard, as many still primarily focused on clinical efficacy with low-quality reporting of objectives and PFS-specific outcomes. Therefore, our review highlights the need for resources and training for researchers, journal editors, peer reviewers and research ethics boards on the value, objectives and appropriate conduct of PFSs. The conceptual framework and standardised methodological components should be widely disseminated and emphasised. Also, EM journals should acknowledge the importance of pilot and feasibility work, reinforce the reporting standards and support the publication of these studies. These actions can lead to more methodologically rigorous PFSs that will inform feasible, successful and rigorous future definitive RCTs in the EM literature.

Data availability statement

No data are available. Not applicable.

Ethics statements

Patient consent for publication

Ethics approval

Not applicable.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors OR, ME and AW conceived and designed the study. OR, JPL, ME and AW collected, managed and analysed the data. OR drafted the article. AW supervised the conduct of the study and provided critical insights for the review. OR takes responsibility for the paper as a whole and is the guarantor of the study.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.