Operative treatment versus nonoperative treatment of Achilles tendon ruptures: systematic review and meta-analysis

I read the study by Ochen et al. with great interest (1). After pooling 10 RCTs and 19 observational studies they conclude: “However, re-rupture rates are low and differences between treatment groups are small (risk difference 1.6%).” Related editorial states: “[…] the difference in re-rupture rate between operative and non-operative management is small and not clinically relevant when examined at population level” (2).

I have three questions about the study by Ochen et al. regarding its methodology and conclusions:
1) How was the risk difference calculated?
2) Is it valid to include a large registry-based study in the analysis?
3) Is it valid to base the conclusion on an estimate which is calculated using studies with poor quality?

1) The authors say that the risk difference in rerupture rate between groups is 1.6%. Since they do not report any CIs or I2 values; I assume this was a crude estimate. In the operative group the prevalence was 215/9375=0.02933 and in the nonoperative group 252/6487=0.038847, resulting in a difference of 0.015914, ie. 1.6%. Hence I assume that the main finding is not a meta-analytic estimate. For the meta-analyses, the authors do not disclose whether they used continuity corrections which is of importance because data for reruptures is sparse, ie. many operative treatment arms have zero count for reruptures. Based on re-calculations, I assume they used a correction of 0.5. With this methodology the pooled risk difference using random effects model is 4.5% (95% CI: [2.4; 6.6], I2=57.1%). For fixed effects model the same estimate is 1.5% (95% CI: [0.9; 2.0]). Therefore, there is uncertainty around the primary outcome reported in the study.

2) The study by Wang et al. (3) was included in the final analyses. Their study includes 12570 patients which constitute almost 80% of all patients included in this meta-analysis. The study by Wang et al. was a registry study based on an insurance billing records from the United States. Registry studies have their advantages, but they do not, as in this case, include any relevant clinical data except age and gender. No clinical follow-up data is available either. Length of follow-up and details on the treatment protocols are not provided. Like index injuries the re-ruptures were assessed using a database search. The data used in the study do not even include the side of the injury and, as the authors write, their data lacks all information on nonoperatively treated reruptures. The study was not population based. Considering that rerupture is a rare event (<10% prevalence) the estimate for rerupture rate in the study by Wang et al. is not robust or unbiased. Combining registry data with RCT and other clinical cohort studies does not seem feasible.

If the study by Wang et al is excluded from the analyses the pooled estimate for RR using the same methodology reduces to 0.37 (95% CI: [0.27; 0.51], I2=0%). Fixed effects model results in an estimate of 0.34 (95% CI: [0.25; 0.47]). The pooled risk difference using random effects model increases to 4.7% (95% CI: [2.8; 6.7], I2=26%). With a fixed effects model the same estimate is 5.9% (95% CI: [4.2; 7.6]). These CIs do not include the point estimate of 1.6%.

3) In addition to the study by Wang et al., eight other observational studies had a MINORS score of less than 12 indicating poor quality. In other words, in seven studies baseline equivalence was not reported and in another eight baseline characteristics were not comparable. Moreover, in 18 out of 19 observational studies loss to follow-up was more than 5% and in eight studies the inclusion/exclusion criteria were unclear or not reported. Hence the validity of including these studies to the primary analysis is questionable.

In Table 3 the authors report the results from secondary sensitivity analysis including only high-quality studies. Again, it is unclear whether the estimate of risk difference is a crude or pooled estimate. The pooled estimate for risk difference using fixed effects model is 5.2% (95% CI: [3.2; 7.2], I2=8.2%). For a random effects model the estimate is 4.3% (95% CI: [2.3; 6.3]). Again the “not clinically relevant” point estimate of 1.6% is excluded.

To conclude, I think there is some uncertainty regarding the primary outcome, ie. the risk difference between groups. Moreover, I think that the primary results by Ochen et al. are not robust and the results do not support the editorial conclusion when comparing operative and nonoperative treatment in general in Achilles tendon ruptures. Instead I suggest that the secondary sensitivity analysis of this study and the additional analysis presented here are more robust and less biased estimates for the primary outcome.

References:
1) Ochen et al. BMJ 2019;364:k5120. doi: https://doi.org/10.1136/bmj.k5120
2) Maffulli et al. BMJ 2019;364:k5344. doi: https://doi.org/10.1136/bmj.k5344
3) Wang et al. Foot Ankle Surg2015;21:250-3. doi:10.1016/j.fas.2015.01.009