Eligibility criteria

Eligible studies will randomise adults with long COVID, defined by WHO as symptoms 3 or more months following laboratory-confirmed, probable or suspected COVID-19 infection that persist for at least 2 months, to pharmacological or non-pharmacological interventions, placebo, sham, usual care or alternative pharmacological or non-pharmacological interventions, with no restrictions on the date or language of publication.26 This definition, although broad, is consistent with the most recent definition published by the National Academies of Sciences, Engineering, and Medicine and reflects the limitations in current scientific knowledge about long COVID.54 55

Given the diverse manifestations of long COVID, our review will not further restrict eligibility based on diagnostic criteria.26 However, subgroup analyses will test how diagnostic criteria influence the effectiveness of interventions.

We anticipate that trials will largely include patients who meet the WHO criteria for long COVID but some trials may not report the time since the acute COVID-19 infection or the duration of long COVID symptoms. We will include such trials in our primary analysis but will also perform additional sensitivity analyses excluding these trials to test the robustness of our findings.

Based on empirical evidence that preprint and published reports of randomised trials generally provide consistent results,56–58 we will include both preprint and published trial reports, but will also perform sensitivity analyses excluding preprints.56 59

At this time, there is too little known about long COVID to anticipate which interventions may be effective. Trials are investigating many types of interventions, including drugs (eg, colchicine, sulodexide, beta-blockers), behavioural and physical therapies (eg, cognitive behavioural therapy, aerobic exercise training), dietary supplements and medical devices (transcranial direct current stimulation).33 40 60 61 We will not restrict eligibility based on the type of intervention and anticipate including trials addressing a diverse range of therapies.

We will exclude pseudorandomised trials, trials involving animals, trials investigating treatments for acute COVID-19, trials testing interventions to prevent long COVID and trials including patients who do not meet criteria for long COVID.26 62 We will also exclude randomised trials with fewer than 25 participants in each arm. We anticipate that smaller trials are unlikely to meaningfully contribute to meta-analyses, are more likely to include unrepresentative samples and arms that are prognostically imbalanced and are at higher risk of publication bias.63

While there are hundreds of long COVID trials underway, trials have progressed at a slower pace than anticipated.64 Depending on feasibility, for select interventions of high potential interest to evidence users, in the first year of the living systematic review, we will consider including indirect evidence from trials addressing interventions for ME/CFS. Clinical experts and evidence users will help determine whether to include this indirect evidence. Subgroup analyses will investigate differences between the effects of interventions between long COVID and ME/CFS. If we find evidence of inconsistency, we will only present results of trials addressing long COVID. Similar to long COVID, we will include trials that recruit patients according to published diagnostic criteria for ME/CFS but perform subgroup analyses investigating potential differences in the effects of interventions based on these diagnostic criteria. Table 1 lists our eligibility criteria.

Data sources and search strategy

An experienced medical research librarian developed a comprehensive search strategy for MEDLINE, EMBASE, CINAHL, PsycINFO, AMED, CENTRAL and preprint servers (MedRxiv and ResearchSquare), from inception. Our search strategy (online supplemental file 1) combines terms related to long COVID with a filter for randomised trials. Additionally, we plan to supplement our search using the Epistemonikos COVID-19 Repository—a living catalogue of COVID-19 research—and by reviewing the references of relevant systematic reviews.37 40 65

The medical research librarian will update the searches at minimum biannually to facilitate biannual updates of the living review. We aim to update the search no more than 3 months before the publication of each iteration of the review.

To ensure our review remains up-to-date, for select interventions identified by our network of evidence users as highly important with great potential for efficacy, we will integrate methods for prospective systematic reviews—systematic reviews that include unpublished and interim data from ongoing and completed trials.66 During the COVID-19 pandemic, trialists reported trouble publishing trials due to null results, inability to achieve the target sample size and diminishing interest from journals. In response, the WHO REACT Working Group performed prospective systematic reviews for select topics.67 68 These reviews sourced ongoing trials via trial registries and invited trialists to share their interim or completed data. This model proved successful and also mitigated the influence of publication bias on review results.67 68 We plan to emulate a similar model for critical interventions of long COVID. Our experience with other living systematic reviews and the experience of other research groups suggest that some trial groups are receptive to sharing unpublished trial data.67–69

For interventions for which we consider evidence from ME/CFS trials, we will work with an experienced research medical librarian to devise additional search strategies specific to the interventions for which we will consider indirect evidence.

Study selection

Following training and calibration exercises to ensure sufficient agreement, pairs of reviewers will work independently and in duplicate to screen the titles and abstracts of search records and subsequently the full-texts of records deemed partially eligible at the title and abstracts screening stage using Covidence (https://www.covidence.org), an online systematic review software for screening titles and abstracts and full-text articles. Reviewers will resolve disagreements by discussion, or, if necessary, through adjudication by a third reviewer.

Data extraction

Following training and calibration exercises to ensure sufficient agreement, pairs of reviewers will work independently and in duplicate to collect data from eligible trials using a pilot-tested data collection form in an Excel spreadsheet (Microsoft Office Excel 2019). Reviewers will resolve disagreements by discussion or by consultation with a third reviewer, if necessary.

Reviewers will collect data on trial characteristics (eg, trial design, country of origin, funding, diagnostic criteria for long COVID), patient characteristics (eg, age, sex and gender, employment and education status, receipt of COVID-19 vaccination, method of acute COVID-19 diagnosis, severity of acute COVID-19 infection, duration of long COVID, number of infections, equity-related characteristics, long COVID symptoms), characteristics of interventions and comparators (eg, type of intervention, treatment duration) and patient-important outcomes.

For dichotomous outcomes, reviewers will extract the number of patients and events in each arm, and, for continuous outcomes, the number of patients, a measure of central tendency (mean or median) and a measure of variability (eg, SD, SE, 95% CI, p value) for each arm. For continuous outcomes, reviewers will prioritise extracting changes in the outcome from baseline, and if not reported, the outcome at follow-up. For each outcome, reviewers will preferentially extract the results from intention-to-treat analyses without any imputations for missing data.70

We will also prioritise outcome data at the longest reported timepoint at which the intervention is still being administered to allow for potential cumulative effects of interventions to emerge without effects dissipating due to termination of the intervention. When trials report data at timepoints at which the intervention is no longer being administered but randomised groups are still maintained, we will consult experts who are blind to the results of the trial about the duration of time the interventions being tested are expected to exert effects. If appropriate, we will consider extracting and analysing data at the longest reported point of follow-up even if the intervention is no longer administered. We anticipate that rehabilitation and behavioural interventions are designed to teach patients coping mechanisms they can independently apply after completing the programme and so results at the longest follow-up may better reflect the sustained effects of the intervention.

Our outcomes of interest are informed by a published core outcome set for long COVID and will include at a minimum fatigue, pain, postexertional malaise, changes in education or employment status, cognitive function, mental health, dyspnoea, quality of life, patient-reported physical function, recovery and serious adverse events (as defined by each trial).71 72 We will extract data for all validated instruments measuring our outcomes of interest. Table 2 presents examples. We will re-evaluate and potentially revise our choice of outcomes annually by discussion with patient partners and knowledge users and by considering emerging evidence. If we find important reasons to include additional outcomes, we will revisit previously included trials to collect data on these additional outcomes.

Given the relapse and remission patterns associated with long COVID and the potential for interventions to have long-term effects, for crossover trials, we will only collect data for the first phase of the trial before washout and crossover.

For trials initially available as a preprint that are subsequently published, we will rely on the results of the published trial report unless the preprint reports additional outcome data not included in the published trial report.

In response to growing concerns about the prevalence of potentially fabricated or falsified research,73 74 reviewers will use the Trustworthiness in RAndomised Clinical Trials (TRACT) checklist to evaluate each trial for the risk of data falsification, fabrication or major errors in the conduct of the trial or analysis of data that could seriously undermine trial conclusions.75 This checklist includes 19 items in seven domains (governance, author group, plausibility of intervention, timeframe, dropouts, baseline characteristics and outcomes) addressing the integrity and trustworthiness of trials. Each item is rated as no concerns, some concerns and major concerns.

The checklist does not include a cut-off at which a trial is considered suspicious and there is currently limited experience using the checklist in systematic reviews. Therefore, the authorship group will discuss all trials that are flagged as raising concerns for one or more domains. We will perform sensitivity analyses excluding trials that are deemed suspicious. We are also aware of another instrument to assess the risk of falsified or fabricated data in trials currently under development (INSPECT-SR).76 On its publication, we will evaluate the instrument and consider integrating it into our workflow, either as a replacement for or in addition to the TRACT checklist.

We anticipate that the effects of interventions may depend on diagnostic criteria for long COVID or ME/CFS, severity of acute COVID-19, time since infection, number of infections, vaccination status and SARS-CoV-2 variant.22 When reported, we will extract stratified data based on these factors to facilitate subgroup analyses.

Risk of bias assessment

Following training and calibration, reviewers will work independently and in duplicate to assess risk of bias (RoB) using a modified version of the Cochrane-endorsed RoB 2.0 tool—the gold standard for assessing limitations in trials that may bias results.77 The RoB 2.0 assesses the RoB across five domains: bias due to randomisation (eg, random sequence generation, allocation concealment), bias due to deviations from the intended intervention (eg, lack of blinding leading to imbalances in co-interventions across trial arms), bias due to missing outcome data (eg, attrition), bias due to measurement of the outcome (eg, unblinded outcome assessment) and selective reporting (eg, selective reporting of outcome measures based on results).

To assess the RoB due to deviations from the intended intervention, we will consider the effect of assignment rather than adherence to the intervention—since this effect is likely to be the observed effect in clinical settings and of the greatest interest to evidence users. Our modified version of the tool includes the same domains, but with revised response options (ie, definitely low RoB, probably low RoB, probably high RoB and definitely high RoB) and guidance tailored to issues relevant for the present review (eg, removing guidance for assessing RoB of adhering to the intervention, listing important cointerventions).

Unless we encounter compelling reasons to do otherwise, we will consider trials without blinding of patients, healthcare providers and investigators at high RoB due to deviations from intended intervention. This decision is based on the potential for unequal distribution of potentially effective cointerventions (eg, physical activity, social engagement, energy management) across trial arms. We will make an exception for trials that compare two or more interventions that are matched in terms of the level of interaction between trial participants and healthcare providers.78 79 Patients might expect interventions with higher levels of interaction to be more effective, potentially influencing their perception of outcomes and their likelihood of pursuing additional beneficial activities. When interventions are matched in terms of interaction, patients are less likely to have strong preconceptions about their comparative effectiveness.

Reviewers will resolve disagreements by discussion or by consultation with a third reviewer, if necessary.

Data synthesis and analysis

To describe trials and participants, we will present descriptive characteristics. Means and medians and associated measures of variability (eg, 95% CI, SD) will describe continuous variables and counts and proportions dichotomous and categorical variables.

Given the heterogeneity in the definition of long COVID and evidence indicating that it may comprise several distinct phenotypes,21 we anticipate that some interventions may be better suited for patients with certain phenotypes of long COVID. For example, pulmonary rehabilitation will likely only be effective for patients who are experiencing pulmonary sequelae related to COVID-19,80 and interventions targeting cognitive function will likely only be effective for patients experiencing neurocognitive sequelae related to COVID-19.81 Other interventions may be suitable for patients experiencing general symptoms related to long COVID, such as fatigue, postexertional malaise and headaches. We will perform separate syntheses when there are compelling clinical reasons to suspect that certain interventions are only applicable or effective for a subtype of long COVID. Clinical experts in our authorship group will lead these decisions and regularly revisit them based on new evidence.

To summarise the comparative benefits and harms of interventions, we will perform network and pairwise meta-analyses. Network meta-analyses compare three or more interventions, grouped into nodes, by pooling direct and indirect evidence.66 To facilitate network meta-analysis, we will classify interventions into ‘nodes’ considering the drug class for pharmacological interventions, class of therapy (eg, cognitive behavioural therapy, aerobic exercise) for behavioural and physical interventions and characteristics of the therapy for other non-pharmacological interventions. If we find evidence that the effects of interventions are different based on their mode (group vs individual, online or in-person), intensity or dose of delivery, we will create separate nodes. We will group placebo and sham interventions and standard care in the same node unless there is evidence that suggests that there are differences in the effect of interventions when compared against placebo/sham and standard care.78

We anticipate that trials may measure the same constructs using different instruments. To enhance interpretation, reviewers may convert effects measured by different instruments assessing the same construct into the most commonly used or familiar instrument.82 83 We will avoid converting effects across instruments due to potential differences in the range of constructs covered by each instrument. We will also avoid standardised mean differences due to their potential to be influenced by differences in variability across trial populations.83

For each outcome and outcome measure, random-effects network meta-analysis using a frequentist framework with the restricted maximum likelihood (REML) heterogeneity estimator—a conservative approach to network meta-analysis—will be used to pool the results of trials.84 85 Our choice of a frequentist over a Bayesian framework is motivated by evidence that there are seldom important differences between the results of Bayesian and frequentist networks and the computational complexity associated with analysing large networks under a Bayesian framework.84

Relative risks (RRs) will summarise the results of dichotomous outcomes—except for serious adverse events that will be summarised using risk differences due to the propensity for trials to frequently report zero events, precluding calculation of RRs and ORs—and mean differences for continuous outcomes, along with associated 95% CIs. When network meta-analysis is not possible, we will perform pairwise random-effects meta-analysis with the REML heterogeneity estimator.85

We will use I² statistics to summarise the magnitude of heterogeneity in meta-analyses and interpret an I² value of 0%–40% as not important, 30%–60% as moderate heterogeneity, 50%–90% as substantial heterogeneity and 75%–100% as considerable heterogeneity.86 87 The I² value, however, is prone to misinterpretation since even small degrees of unimportant inconsistency may translate to high I² values if estimates from studies are highly precise.88 89 Hence, we will also consider the absolute magnitude of differences in effect estimates across studies. For analyses that include 10 or more studies, we will test for publication bias using Egger’s test and qualitatively review funnel plots for evidence of asymmetry.90 91

A feature of network meta-analyses is their ability to generate treatment rankings but we will avoid using treatment rankings to interpret our results based on theoretical and empirical considerations that they may be misleading.92–94

Diverse trial outcomes or outcome measures may lead to disconnected networks or preclude network formation. In such situations, consistent with established guidance, we will present the results of disconnected networks separately.66 We will refrain from model-based approaches to link networks because of their novelty and limited evidence supporting their reliability.95 If networks cannot be formed, we will present pairwise meta-analyses. With the emergence of more trial evidence, we anticipate that disconnected networks will eventually combine to become connected.

To enhance interpretability of results, we will transform relative effects (eg, RRs) to absolute effects (eg, number of events per 1000 patients), using the median risk of the outcome reported across the control groups of eligible trials.96

We will test for incoherence using node-splitting.97 In case of incoherence, we will investigate potential sources considering our a priori defined hypotheses for effect modification: diagnostic criteria for long COVID, time since infection, number of infections, vaccination status, severity of acute COVID-19 and SARS-CoV-2 variant.22 Should we choose to include evidence from ME/CFS trials, we will also consider ME/CFS as a potential source of incoherence. In the event that tests for effect modification are unable to identify a credible source of incoherence, we will downgrade the certainty of evidence. Conversely, if we confidently identify the source of incoherence, we will perform separate analyses stratified by the source of incoherence.

We have also generated similar a priori factors to explain potential heterogeneity in results between trials: diagnostic criteria for long COVID, time since infection, number of infections, vaccination status, severity of acute COVID-19 and SARS-CoV-2 variant.22 98 Meta-regressions and subgroup analyses will test for subgroup effects based on these factors. Notably, we will avoid pooling trials rated at low and high RoB indiscriminately. Instead, we will test for differences between the results of these trials, and when we detect important differences, rely on trials at low RoB. If we choose to include evidence from ME/CFS trials, we will also perform subgroup analyses comparing the results of trials addressing long COVID and ME/CFS.

Inferences of subgroup effects often prove spurious.99–108 Such spurious claims may be attributed to testing many factors, leading to apparent but inaccurate evidence of effect modification due to chance, selective reporting or improper statistical analysis.108–114 To avoid misleading claims, we will assess the credibility of subgroup effects using the ICEMAN tool—the gold standard tool for evaluating effect modification in trials and systematic reviews.115

We will perform all analyses using the meta and netmeta packages in R (V.4.1.2, Vienna, Austria) and make all code to reproduce our results freely accessible on Open Science Framework.116 117

Certainty of evidence and reporting

Results from studies may appear impressive, but we may have low confidence in results due to limitations of the evidence. The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach will guide our assessment of the certainty (quality) of evidence.118–120 The GRADE approach rates the certainty of evidence as either high, moderate, low or very low certainty based on considerations of RoB (ie, study limitations), inconsistency (ie, heterogeneity in results between trials), indirectness (ie, differences between the questions addressed in studies and the question of interest), publication bias (ie, the tendency for studies with statistically significant results or positive results to be published, published faster or published in journals with higher visibility), imprecision (ie, random error), intransitivity (ie, violation of joint randomisability) and incoherence (ie, differences between direct and indirect estimates). High certainty evidence indicates situations in which we are confident that the estimated effect represents the true effect and low or very low certainty evidence indicates situations in which the estimated effect may be substantially different from the true effect.

A minimally contextualised approach will guide our judgements related to imprecision.121 The minimally contextualised approach does not consider statistical significance as the only indicator of whether an intervention is effective. An estimate may not be statistically significant but may still have evidence of moderate certainty for benefit or harm. Conversely, an intervention may produce results that are statistically significant but that indicate no important benefit or harm. The minimally contextualised approach considers only whether the effect estimates exceed the minimum important difference (MID)—the smallest difference in an outcome that patients find important—and does not consider whether the effect is small, moderate or large. If the point estimate meets or exceeds the MID, we will assess the certainty of there being an effect and use the null as the threshold for judgements about imprecision. If the point estimate falls below the MID, we will assess the certainty in the absence of an important effect, and downgrade the certainty if the confidence interval crosses the MID, indicating that the intervention may potentially confer an important effect.

We will source MIDs either from the literature, or, when not available, survey our review authors and patient partners. MIDs of patient-reported outcomes are typically determined either using anchor-based or distribution-based methods.122 Anchor-based methods use an external ‘anchor’ to interpret the magnitude of change in a measure or outcome. Distribution-based methods rely on the distribution of the data to interpret the importance of change in a measure. We will prioritise anchor-based MIDs over distribution-based MIDs, since anchor-based estimates reflect patients’ direct experiences. We anticipate that guideline producing organisations will fully contextualise the results to formulate recommendations.123 Finally, should a published MID be unavailable for any of the outcomes of interest, particularly for dichotomous outcomes like recovery and return to work/education for which MIDs are typically not derived, we will survey patient partners and our partner evidence users on reasonable ranges using a previously established procedure.124

To make judgements about intransitivity, we will consider the potential effect modifiers within the network, including the credibility of effect modification, the strength of effect modification and the distribution of effect modifiers across direct comparisons. There is currently limited evidence on potential effect modifiers of interventions for long COVID. We anticipate that the effects of interventions may vary based on diagnostic criteria, severity of acute COVID-19, time since infection, number of infections, vaccination status and SARS-CoV-2 variant. If we find credible evidence of effect modification based on these or other factors, we will consider rating down for intransitivity when appropriate. For comparisons that include evidence from ME/CFS trials, we will additionally rate down for indirectness.

Reporting of results

We will report our living review according to the PRISMA checklist for network meta-analyses and PRISMA extension for living systematic reviews.52 53 125 PRISMA flow diagrams will illustrate the total number of search records, the number of records excluded, reasons for exclusion and the total number of trials included in the review. Network and forest plots will present network geometry and results from meta-analyses, respectively. GRADE Evidence Profiles will summarise effect estimates and the certainty of evidence.96

We will describe our results using GRADE plain language summaries (ie, describing high certainty evidence with declarative statements, moderate certainty evidence with ‘probably’, low certainty evidence with ‘may’ and very low with ‘very uncertain’).126

For each outcome, we will place interventions in categories from best to worst.127 128 First, we will classify interventions according to whether they are more or less effective than placebo or standard care. If the treatment effect of the intervention vs placebo or standard care is less than the MID, the intervention will remain in the same group as placebo or standard care. If, on the other hand, the effect meets or exceeds MID, the intervention can be classified as more or less effective than placebo or standard care, depending on the direction of the effect. Subsequently, we will compare the interventions classified as more effective than placebo or standard care against each other by examining whether the differences in effects between them exceed the MID. In the final step, we will further categorise interventions according to the certainty of evidence.

Each iteration of the living review will also be accompanied by a plain language summary (each <800 words) that describe our findings in lay language for healthcare providers and patients who may not be familiar with network meta-analysis methods. We will draft these summaries according to established guidance by Cochrane: they will describe the types of interventions tested and describe the benefits and harms of interventions supported by moderate or high certainty evidence.66 We anticipate these plain language summaries will reduce the opportunity for misinterpretation of our findings by healthcare providers, patients and other decision-makers who may not be familiar with network meta-analysis methods or the interpretation of evidence according to the GRADE approach.

Updating and triggers for retirement

We will update our living systematic review and network meta-analysis every 6 months or sooner in the event of publication of practice-changing evidence (eg, publication of a seminal trial, when the certainty of evidence or the magnitude or direction of the effect of an intervention importantly changes). Preprints and journal publications will communicate the results of each iteration of our review. This approach adheres to best practices in updating living reviews: it balances the need for up-to-date evidence with the time needed to ensure that the review is sufficiently rigorous, focuses our efforts on disseminating critical findings and maximises the feasibility of the project.50 129

We will retire our living systematic review when the evidence base becomes stable with few to no new trials, if we reach high or moderate certainty evidence for all interventions and outcomes suggesting that new evidence is unlikely to change current estimates, when our network of evidence users suggest that the findings of the living systematic review are no longer relevant to them, if we deplete our funding, or if we can no longer maintain the personnel needed to continue the living review.130 At this time, given the timeline of planned trials and the funding available to us, we intend to continue to maintain the living systematic review for 3 years.33 34

Our Open Science Framework repository dedicated to the living review will inform evidence users of the status of the review (whether it is active or retired), the anticipated date of the next update and the results of the most recent iteration of the review (osf.io/9h7zm).

Conflicts of interest

Systematic reviews necessitate subjective judgements about the magnitudes of benefits and harms of interventions and the certainty of evidence. To ensure such judgements are not unduly influenced, we will screen all coauthors and members of our team for financial and intellectual conflicts of interest using a standardised procedure developed by the BMJ.131

Financial conflicts will include stocks, grants, research contracts, royalties and speaking fees and travel accommodations and intellectual conflicts will include academic publications or statements on social or traditional media that could make reviewers attached to a particular intervention or point of view. We will exclude individuals with financial conflicts and restrict intellectual conflicts to no more than 25% of the team. Only reviewers completely free of both financial and intellectual conflicts of interest will be involved with screening search records, data extraction, RoB assessments, data analysis and assessment of the certainty of evidence.

Patient and public involvement

As part of our funding application, patients were involved from the study’s inception, reviewing and offering feedback on the protocol, provided by the Long Covid Web Patient Advisory Council.

When interpreting our results, we will rely on patients’ judgements about whether they consider benefits of a particular therapy to outweigh harms. To do this, we will perform semi-structured interviews and discussions with purposively selected groups of patient partners, aiming for diversity in demographics (eg, age, sex, under-represented racial or ethnic groups, income, abilities) and experiences of long COVID (eg, severity, duration). These interviews will be intended to offer explanations for why certain therapies may be preferential to others. This feedback ensures that this study directly aligns with patient priorities.

For each iteration of the review, three to four patient partners will also review plain language summaries that describe our findings using language that will be accessible to the general public for readability and acceptability.132 133 Patients will be involved in dissemination of results to ensure accessibility.

We anticipate that this living systematic review will become a trusted reference point for national and international professional associations and authoritative organisations that intend to produce guideline recommendations on the management of long COVID. We will prioritise engaging organisations that involve patients in their guideline development process and consider patient values and preferences—consistent with standards for producing trustworthy guidelines.134–137