Randomised controlled trials: missing data

Answers

Statements a, b, and c are all true for the method of baseline observation carried forward.

The aim of the trial was to assess the effectiveness of a range of weight management programmes for weight loss. A randomised controlled trial study design was used. Outcome measurements and baseline characteristics were available for all participants, whereas follow-up data were available for 658 (88.9%) participants at the end of the programme (12 weeks) and 522 (70.5%) at one year.

It is common for trial participants’ outcome measurements to be missing. Referred to generally as “missing data,” this can occur for a variety of reasons. Participants may not start the treatment that they are randomised to, they may be lost to follow-up, or they may not provide outcome measurements when required during follow-up. How missing data are dealt with in the analysis of a trial can have a major effect on the results and therefore the conclusions.

In the study above, the most straightforward method for dealing with missing data would have been to exclude those participants who did not provide all outcome measurements during follow-up. Such an approach is referred to as “complete case analysis.” However, if the occurrence of missing data is associated with a treatment or the outcomes, this approach may introduce confounding in the comparison of treatment groups. Participants were randomised to treatment groups. The aim of randomisation, if the sample size for a trial is large enough, is to produce groups of patients similar in baseline characteristics. Such factors include demographics, prognostic factors, and characteristics that influence participants to take part in a trial. Therefore, any differences between treatment groups in outcome would be due to differences in treatment received, not differences in baseline characteristics, and the trial would have internal validity. Described in a previous question,2 internal validity is the extent to which the observed treatment effects can be ascribed to differences in treatment and not confounding. However, the exclusion of those participants with missing data from the analysis would introduce confounding because the balance in baseline characteristics would no longer exist. As a result, internal validity would be threatened. Furthermore, the exclusion of participants would ultimately lead to reduced statistical power.

Imputation is an alternative approach to deleting participants with missing data. This involves replacing a missing observation with a plausible data value, which may be an existing value or one that is predicted from the participant’s available data. Missing data were imputed in the trial above using the method of “baseline observation carried forward.” Using this method, participants’ outcome measurements at baseline were used for all time points during follow-up where the data were missing. The method is appealing because it is straightforward. In particular, it ensures that all randomised patients are included in the analyses, thereby maintaining comparability between treatment groups at baseline (a is true) and minimising confounding.

It is difficult to predict the weight at 12 weeks or one year of participants with missing data. No doubt all participants would have experienced some weight change—whether a gain or decrease—during follow-up. Use of the method of baseline observation carried forward meant that participants’ outcome measurement at baseline was used at all times during follow-up when data were missing. The method assumed that the weight of participants who did not have their weight measured at 12 weeks and one year did not change during follow-up from baseline (b is true). This is unlikely, however, so the method may have provided biased outcome measurements and hence biased sample estimates.

In addition to using baseline observation carried forward to impute missing data, the researchers also undertook analyses with missing data imputed using the method of last observation carried forward. The two methods are similar approaches to the imputation of missing data. Use of the method of last observation carried forward meant that participants’ last outcome measurements were used for all subsequent missing data. As for the method of baseline observation carried forward, the method of last observation carried forward assumes that participants’ weights remain unchanged from when they last had their weight recorded. However, this does not seem plausible. The method of last observation carried forward was used as part of a sensitivity analysis. Because these methods of imputation of missing data may bias the sample estimates, the two methods were used to assess whether they produced similar results. However, both methods of imputation probably underestimate the standard deviation of the outcome of weight change for a treatment in a trial. This is likely to lead to confidence intervals that are too narrow and to provide estimates that are too accurate (c is true). Because both methods of imputation would probably lead to biased sample estimates for treatment groups, any comparisons between treatment groups would be inaccurate.

Although the methods of baseline observation carried forward and last observation carried forward have some advantages, they are generally discouraged as methods of imputation for the reasons described above. Much methodological work has recently been undertaken on the development of approaches to imputation. More sophisticated methods, including regression mean imputation and multiple imputation, have been developed that are thought to give more plausible and unbiased values. These methods for imputation will be explored in future endgames.