Vitruvian plot: a visualisation tool for multiple outcomes in network meta-analysis

Introduction

The impact of scientific findings on real-world clinical practice is determined not only by their intrinsic value but also by how effectively they can be communicated to clinicians and patients.1 This has been the case for more than 150 years. In 1854, nurse and statistician Florence Nightingale arrived in Turkey with a group of 37 nurses to attend the wounded of the Crimean War. While there, she recorded more casualties from epidemic diseases, malnutrition, poor sanitation and other modifiable factors than from battlefield wounds.^e1 Her initial attempt to communicate how a large proportion of these deaths was avoidable used complex statistics. Despite her efforts, her findings were viewed with scepticism and disregarded.^e1 Realising that the main pitfall was the format used to communicate her findings, she developed the Diagram of the Causes of Mortality in the Army of the East, now known also as Nightingale’s rose diagram or coxcomb.2 ^,e2 The success of this communication strategy promoted the implementation of preventative public health measures and ultimately contributed to the reduction of death rates in hospitals.^e1

Available literature on behavioural decision research and communication on benefit–risk information indicates that the format in which findings are presented may significantly affect the comprehension and behaviour of end users.3–7 ^{,e3, e4} Well-designed patient decision aids—either visualisation tools or conversation aids—have been shown to materially improve patients’ understanding of the effects of outcomes and risks compared with control interventions.^{3, e4} First, bars and pictographs have been identified as the best strategies to visually communicate complex statistics and numerical results from multiple comparative alternatives.6 ^{, e3} Second, a common timeframe and a consistent reference class (denominator) and format should be employed when comparing the chance of occurrence of two or more independent events, favouring absolute risks, either percentages or frequencies, over relative ones.6 Third, 6textual and graphical communication should acknowledge the average numeracy and literacy skills of the target audience.5 6 Fourth, the implications of communicating the inherent uncertainty in the results should also be considered.6 7 Finally, available information should be accompanied by narrative statements to inform both gain and loss frames (ie, people experiencing and not experiencing the event of interest)6 7

Network meta-analysis (NMA) is a useful tool for summarising available evidence about the effects of three or more interventions for the same condition.8 Typically multiple outcomes are analysed separately, aiming to capture the comparative profile among treatment alternatives. The combination of multiple interventions and outcomes further increases the complexity of presenting benefit–risk findings from NMAs. Pooled estimates concerning an outcome are usually presented using forest plots or league tables.^e5 These methods, however, are only fit to simultaneously show results regarding a small number of outcomes, limiting the overall interpretation of findings.9 10 Alternative methods to visualise multiple outcomes from NMAs have been proposed,11–14 ^,e6-e10 with only limited consistency with good practice guidelines for communicating benefit–risk information.6 15–18 For instance, some tools prioritised the textual information over the graphical communication, with the latter limited to the use of numbers in coloured tables.12 ^,e6-e8 A visual representation would allow patients and clinicians to quickly appraise and compare the available interventions across the considered outcome. On the other hand, other tools employed alternative visualisation display (ie, rank heat plot, bubble plot, spie chart, and scatter plot) to communicate risk information from advanced metrics.11 13 ^{,e9, e10}

The development of an effective and user-friendly communication tool would be a critical step forward in the implementation of an evidence-based approach in real-world shared decision-making processes. We aimed to create a tool to facilitate communication of findings from multiple outcomes in NMA to patients and clinicians. We developed this tool following recommendations from the available literature and collecting feedback from researchers, statisticians, methodologists, clinicians and people with lived experiences of physical and mental health issues. We named it a ‘Vitruvian plot’ after Leonardo da Vinci’s Vitruvian Man, which simultaneously represents multiple human proportions. This article illustrates two alternative visualisation strategies to communicate results from multiple NMAs and the evaluation of confidence in these findings. The data sets, the R code and related instructions to further customise the script are freely available on GitHub (https://github.com/EGOstinelli/Vitruvian-plot). In the online supplemental material, we provide two competing visualisation methods using the same data sets.

Methods

How to build the plot

We developed the Vitruvian plot in R as a radial bar plot, which is a bar plot shifted from a cartesian coordinate system (where coordinates are defined as distances from two fixed perpendicular axes) to a polar one (where coordinates are defined as the distance from a polar point and an angle from a polar axis). Each Vitruvian plot corresponds to a single intervention and shows the NMA results for multiple outcomes.

For binary outcomes, the impact of each intervention is shown on an absolute event rate scale, which is easier to understand for patients and clinicians.6 In our examples, we used the ORs from the outcome-specific NMAs and a fixed event rate for placebo (control event rate, CER) to estimate the event rate of each intervention19:

For continuous outcomes, we can follow a similar procedure if the NMA is performed on the natural scale, that is, using mean difference. In this case, we would simply use the mean difference for each intervention versus placebo estimated from the NMA, and a fixed mean outcome for placebo. If, however, the NMA is performed using the standardised mean difference (SMD), these can be converted in ORs as follow, for instance using the Hasselblad and Hedges’ method:20 ^{, e11}

When estimating CER from continuous data, the number of events in the population of patients allocated to the common comparator can be imputed using the baseline and endpoint continuous values and after setting a meaningful threshold (eg, 50% improvement from baseline).21 In all cases, the absolute estimate for the common comparator can either be derived from the data used in the NMA or supplied from an external source (ie, ‘real-world’ study).

In the following analyses, we used a mock data set to analyse five outcomes with corresponding NMAs for demonstration purposes. Figure 1 shows a plot where the overall profile of each active intervention and placebo across five outcomes is depicted as wedges and expressed as absolute estimates: response, remission, all-cause drop-out (acceptability), nausea and headache. Absolute estimates of both the intervention of interest and the common comparator are reported as grey squares and light blue circles, respectively, to contextualise the absolute estimates. The Vitruvian plots can then be merged in a synoptic chart to simultaneously visualise and navigate among all the available interventions.

Figure 1

Request permission

Anatomy of the Vitruvian plot. The key elements of the Vitruvian plot and related terminology are hereby illustrated.

We applied the recommendations on the communication of benefit–risk information from the available literature.3–7 ^{,e3, e4} Each radial bar represents the magnitude of effect for an outcome and also displayed as a percentage, using both textual and graphical communication strategies. Uncertainty over the relative performance against the common comparator is reported as the strength of statistical evidence (figure 2) or as the confidence in the evidence (figure 3). Finally, each visualisation is accompanied by a supportive statement to facilitate the interpretation of the Vitruvian plots.

Figure 2

Request permission

Synoptic chart showing strength of statistical evidence. The profiles of placebo and treatments 1 and 2 are compared across five domains: clinical response, clinical remission, acceptability (all-cause drop-out), nausea, and headache. A dark green colour indicates there is strong statistical evidence that the corresponding intervention performs better than placebo. A dark red colour indicates strong evidence that the intervention performs worse than placebo. Light green/red colours indicate weaker evidence of a beneficial/detrimental effect. Colours closer to yellow indicate an increasing lack of evidence on whether the intervention performs better or worse than placebo. The light blue colour identifies placebo as the common comparator. When presenting the Vitruvian plots to patients, we strongly suggest including both probabilities of events and non-events as well as the time frame. A suggested reporting statement for the ‘remission’ outcome and treatment 1 is: ‘Of 100 people with your clinical diagnosis, 15 may remit (1 more than with placebo) and 85 may not remit with treatment 2 over 8 weeks’.

Figure 3

Request permission

Synoptic chart showing confidence in the evidence. The profiles of placebo and treatments 1 and 2 are compared across five domains: clinical response, clinical remission, acceptability (all-cause drop-out), nausea and headache. Darker hues of purple indicate increasing confidence in the evidence for the comparison of the corresponding treatment vs placebo (categorised in four levels, that is, very low, low, moderate, high quality). The light blue colour identifies placebo as the common comparator. When presenting the Vitruvian plots to patients, we strongly suggest including both probabilities of events and non-events as well as the time frame. A suggested reporting statement for the ‘nausea’ outcome and treatment 1 is: ‘Of 100 people with your clinical diagnosis, 14 may experience nausea (2 more than with placebo) and 86 may not experience nausea with treatment 2 over 8 weeks’.

Results

Discussion

In this article, we have illustrated a novel way of presenting information for NMAs in a transparent, accessible and useful manner that can inform and support the decision-making process between patients and clinicians. This is instrumental to take full advantage of the available evidence .6 7 For instance, in our first example (figure 2), treatment 1 may be considered a strong candidate due to its response profile. Nonetheless, after considering the confidence in evidence (figure 3), treatment 2 might be a good candidate due to its high confidence ratings.

One of the strengths of the Vitruvian plot is the active involvement of people with lived experience of physical or mental diseases, and clinicians. This approach, combined with the involvement of multidisciplinary experts and recommendations from available literature, allowed us to overcome some limitations of the existing modalities to present results from NMAs. For instance, although the Vitruvian plot shares visual similarities with the spie chart with equally weighted outcomes (see section 3 of online supplemental file), the latter did not include a measure of confidence in the evidence when presenting the magnitude of effect, something that Daly and colleagues expressed interested in as a possible future development.11 The spie chart implemented ranking metrics, such as the surface under the cumulative ranking curve (SUCRA) values, as metrics of choice,11 similarly to the radar graph developed by Seide and colleagues to visually compare multiple drugs over an outcome.14 The feedback we received from patients and clinicians was instrumental to develop the Vitruvian plot. For instance, we implemented the level of confidence in the evidence and favoured absolute estimates over relative ones or ranking metrics. The use of absolute metrics is in line with currently available recommendations for the comparison of the chance of occurrence of two or more independent events.6 7

Absolute estimates have been used in the Kilim plot, together with the colour scheme we adopted in our first example (ie, strength of statistical evidence).12 Our approach differs from the Kilim plot in how we presented the metric of choice, combining textual information with corresponding visual elements. The interpretation of the magnitude of effect is conveyed by both textual and graphical formats, an optimal communication strategy when targeting populations with varying levels of numeracy and literacy skills.4–6 Although visual aids are not free from bias, such as an increase or reduction of risk-avoidant behaviour, well-developed visual formats have been proven effective to communicate health risks. For instance, Garcia-Retamero & Cokley found that adding visuals to written messages was more effective than written health information alone in improving accuracy in judgements and promoting changes in health behaviour related to sexually transmitted diseases in a high-risk population.24

Communication and understanding of uncertainty are ‘arguably essential for informed decision- making’.6 This is in line with the increasing support to avoid dichotomising findings according to the statistical significance (ie, statistically significant or not), especially for NMAs.25 26 Nonetheless, how to effectively implement this in the shared decision-making framework is a challenge yet to be solved. Only a small number of studies have evaluated the potential impact of communicating uncertainty in healthcare, suggesting that it might be associated with pessimistic risk perceptions and avoidance of reaching a decision (ie, ‘ambiguity aversion’).6 27 28 ^,e13-e17 Communication of uncertainty may exert an effect on how people understand it, feel about it, and ultimately trust the information.29 A careful assessment of the impact on these domains is essential not only to prevent detrimental effects but also to reinforce and enhance behavioural changes and confidence with the tool. For instance, reporting uncertainty may be perceived as an example of transparent communication in research, thus increasing trustworthiness.30 Findings from a recent study on the impact of communicating uncertainty suggest that people adequately recognised and perceived uncertainty, with only a limited impact on the perceived reliability of numbers.30

The current version of the Vitruvian plot has some limitations. First, outcomes are not differentiated in terms of weight. Daly and colleagues explored non-equal weighting of outcomes.11 In their example, they used the regression coefficients from a multivariate regression model to tailor areas and ranks for each treatment to sex assigned at birth and age group. While this approach could be implemented in the current version of the Vitruvian plot, we aim at developing an interactive version in which outcomes can be optionally weighted according to the individual preferences of users. Second, the current version of the Vitruvian plot works best with a limited number of concurrent independent outcomes. Implementation of highly correlated outcomes is discouraged, as these will impact the overall profile of the intervention. Moreover, the number of outcomes presented should be carefully considered and balanced to present a comprehensive and informative synthesis of the available evidence in supporting practical decision-making. Third, continuous outcomes require conversion into dichotomous effect sizes when SMDs are used in the analysis. Finally, it is yet unclear whether this and competing methods of visualisation for multiple outcomes NMA provide a real practice benefit. The performance of these visualisation strategies in improving comprehension and better engaging users in decision-making should be carefully assessed.

As the interplay between patients, clinicians and evidence continues to rapidly evolve, the development of accessible formats to communicate critical information is of increasing importance. We hope that the implementation of the Vitruvian plots in real-world practice will support patients and clinicians in evidence-based shared decision-making. Future steps would be to develop an interactive version of the Vitruvian plot and to properly assess this new communication strategy to explore the relative performance and preferences of different components of the Vitruvian plots, such as format (ie, static vs interactive) and specific approaches (eg, statistical strength vs confidence in the evidence), across different fields of medicine and healthcare.

References e1–e17 are provided in online supplemental file.