Monitoring progress with national and subnational health goals by integrating verbal autopsy and medically certified cause of death data

Introduction

All countries have committed to monitor progress towards national and international health goals, including the Sustainable Development Goals (SDGs) and the goals and targets set forth in the WHO’s Noncommunicable Diseases Global Monitoring Framework (NCD GMF).1 2 Global and local goals and targets are a key component of any health and development strategy, providing vital intelligence on the effectiveness of targeted interventions and the outcomes of significant investments in health systems. Many of these health goals require accurate and timely cause of death (COD) and all-cause mortality data for the country, disaggregated by age and sex.3 Indeed, 7 of the 17 SDGs, and 17 of their corresponding indicators, require cause-specific mortality data from a Civil Registration and Vital Statistics (CRVS) system for their measurement.4 A good example is SDG 3.4 which aims, by 2030, to reduce premature mortality from NCDs by one-third.1

The COD data to measure progress with these goals should ideally come from a complete and timely CRVS system where the cause of every death is ascertained using the International Medical Certificate of Cause of Death (MCCOD) completed by a physician, adhering to the certification and coding rules of the International Classification of Diseases Revision in force (currently ICD-10).5 However, the ability of national governments to reliably measure indicators for these, and other health goals, is affected by incomplete death registration and often poor-quality COD data. Only about two in three deaths worldwide are registered, while close to 70% of the world’s population live in a country where less than 90% of deaths are registered or otherwise known to health authorities, overwhelmingly low/middle-income countries (LMICs) where the burden of premature mortality is greatest.6 Reliable COD data are even less available for policy and planning.7 In many countries, the only available COD data are from hospitals, but these are often misdiagnosed or otherwise of poor quality and are unrepresentative of national COD patterns because they are biased towards younger ages at death and acute diseases.8 9 Data on community (ie, non-hospital) deaths, which comprise the majority of deaths in LMICs, are commonly either unavailable or contain vague or ill-defined diagnoses due to poor-quality data collection and diagnostic practices.10

Typically, the guidance offered to countries to assist them to compute national values of these indicators does not adequately specify what methods to apply when faced with incomplete or suboptimal quality data. For example, SDG Indicator 23 (pertaining to SDG 3.4 mentioned above) measures the probability of dying between exact ages 30 and 70 years from either a cardiovascular disease, cancer, diabetes or chronic respiratory disease. The SDG guidance states that the most reliable data source for this indicator should be based on death certificates and administrative data from health facilities, but without any mention of the intrinsic biases in health facility data.11 The guidance offered goes on to state that where coverage of death certification is poor, household surveys can be used to measure NCD mortality, but without any specific guidance on potential quality issues or biases commonly associated with survey data on CODs, nor on the feasibility of using this source, or how to use them to calculate the indicator.11 Similarly, the WHO’s NCD GMF specifies that, for the same indicator, the preferred data source is CRVS systems of sufficient completeness to calculate all-cause mortality rates, with other possible sources being sample registration systems and verbal autopsy (VA).2 Again, there is no guidance about the potential limitations of these data nor how to use or interpret them. For many countries, these challenges in calculating mortality-based indicators mean there is a heavy reliance on estimates produced by the Global Burden of Disease (GBD) Study and other international efforts using complex statistical models, often largely based on other countries’ COD data, with very little direct evidence on the local epidemiological situation.12–14 Further, these estimates are not generally available for subnational populations for the vast majority of countries. Recent research has suggested methods to estimate population-level CODs by extrapolating from in-hospital cause patterns, but it is unclear how widely applicable this approach might be.15

For many, if not most, LMICs, complete death registration and widespread use of medical certification to ascertain CODs are unlikely to be achieved in the foreseeable future, in part because there is limited experience and knowledge about cost-effective ways to notify community deaths to official authorities and typically low physician to population ratios.16 17 Increasingly, VA is being used or trialled in LMICs to ascertain the cause of community deaths in rural populations, together with more effective data management practices with the potential to integrate them into routine death registration and reporting systems.10 VA predicts the probable COD based on an interview of family members of the deceased using a standardised questionnaire about the signs and symptoms experienced by the deceased prior to death; these responses are then applied to automated diagnostic algorithms or reviewed by a physician, to ascertain the probable COD. For example, in Myanmar, since 2018 VAs have been collected from 42 townships that comprise a representative sample of the national population.10 Routinely collected VA data on all, or a sample of, community deaths, diagnosed using an established, validated diagnostic method, together with COD data from hospitals ascertained using MCCOD, can, in principle, lead to more policy-relevant, representative COD statistics than either source alone. Yet, there has been relatively little scientific debate about issues in the integration of the two data sources, despite the fact that this process has significant potential to fill key data gaps that currently exist in many LMICs and enable countries to more reliably monitor progress with health goals.

This integration is not straightforward, however, and needs to consider not only the number of reported VAs and hospital deaths from each cause, but also the extent to which the reporting of the fact of death is incomplete in hospitals and the community, by age and sex. As a result, there have been few attempts to integrate VA and MCCOD data; previous studies to generate national cause-specific mortality rates by integrating VA and MCCOD data have been conducted in Malaysia and Thailand but have relied on complete or close to complete death registration systems where there is reasonable certainty about the true numbers of deaths by age and sex.18 19

To our knowledge, there are no published studies, with replicable methods, on the use of incomplete death reporting and the integration of VA and MCCOD data to calculate national cause-specific mortality rates, despite the policy utility of this information. National Statistics Offices and Ministries of Health should be empowered to calculate these indicators themselves using available data, in order to be able to continuously assess progress towards national and subnational health goals. In this paper, we propose a method for local analysts to apply to estimate national and subnational cause-specific mortality rates for populations where both MCCOD and VA data are available, and where death registration or reporting is incomplete. We then provide an example of the application and interpretation of the method to calculate SDG Indicator 23, the probability of dying between exact ages 30 and 70 years from any of cardiovascular disease, cancer, diabetes and chronic respiratory disease, for a nationally representative population of Myanmar.

Methods

Results

In Myanmar, completeness of VA reporting was estimated to be 70% in 2019, being substantially higher in males than females (81% males, 63% females). This was assessed as being sufficient to proceed with integration of the data, recognising that about one-third of community deaths do not have a cause, more so for females. This analysis resulted in a total of 67 426 estimated deaths at all ages in 2019, of which 89% were estimated to have occurred in the community; similar to GBD and UN World Population Prospects (WPP) estimates (according to the total estimated deaths, the probability of dying from age 15 to 60 years in 2019 was 251 per 1000 for males and 134 per 1000 for females, compared with 226 for males and 128 for females as estimated by the GBD, and 246 for males and 149 for females as estimated by the UN WPP).22 23 The percentage of VAs at ages 30–69 years for which the COD could not be determined in 2019 was 12% for men and 10% for women. The percentage of MCCODs at ages 30–69 years that were ill-defined (R00–R99) was 6% for both men and women. In 2019, 22% of MCCODs had unusable CODs (ie, of limited use for policy) at very high, high or medium severity levels.29 Further analysis of the quality of the VA data using the VIPER tool demonstrated that the CSMFs from VA were plausible, based on a comparison with GBD 2019 data and an analysis of the age–sex pattern of specific causes.20 For hospital deaths coded to chronic respiratory diseases (J30–J98) in 2019, 49% used codes that mapped to chronic respiratory for SmartVA (J40–J46); this figure was 61% if J96 ‘Respiratory failure, not elsewhere classified’ and J98 ‘Other respiratory disorders’ were excluded from the denominator (because many of these deaths would likely be in codes J40–J46 if correctly certified). Almost all (95%) hospital deaths included in the SmartVA’s cancer codes (C00–D48) were within the WHO’s codes for this cause group (C00–C97).

Table 1 summarises the data inputs and outputs in the integration of the two data sources. As expected, a higher proportion of younger adult deaths occurred in hospitals compared with older adults. The CSMF of the four causes combined increased with age and was consistently higher in community than hospital deaths. For all (ie, total) deaths, the CSMF of the four causes combined was comparatively high, as expected, being 0.676 for men and 0.749 for women in the age group of 65–69 years. When the four causes were calculated as age-specific death rates and probabilities of dying, they were higher among men than women, especially in younger age groups. The final result shows that the probability of dying of any of cardiovascular disease, cancer, diabetes or chronic respiratory disease between ages 30 and 70 years in 2019 was 0.265 for men and 0.216 for women, independent of other causes and assuming current mortality conditions.

Online supplemental table S2 presents the CSMFs for each of the four causes by age group and sex. Cardiovascular diseases were the most common of the four causes, with the CSMFs for each increasing with age. The CSMFs were higher for women in the younger ages because a much higher proportion of men in these ages die from external causes and cirrhosis. The CSMFs for diabetes were consistently and significantly higher for women across all age groups.

Figure 1 compares CSMFs for specific causes according to whether the data were reported in hospitals only with that resulting from the integration of data sources. For each cause, the CSMF derived from the integration data was higher than that in hospitals, with the largest differences among women. For all four NCDs combined, for example, the CSMF for women based on the integrated data was 65% vs 41% in hospitals, while the respective figures for men were 43% and 28%. In other words, NCDs are likely to be 50% higher based on integrating data sources compared with hospital data alone. Other large differences can be observed for cardiovascular diseases (10 percentage points higher for women) and cancers (8 percentage points higher).

Discussion

The measurement of progress towards the attainment of SDGs and other health goals assumes complete reporting of deaths and reliable COD data, something that is, and will continue to be challenging for many LMICs where good quality mortality data are typically not available. The introduction of routine VA in an increasing number of LMICs to measure COD patterns in the community, where the majority of deaths occur, should, in principle at least, substantially improve countries’ ability to monitor health goals and, more generally, generate evidence on local COD patterns.

This manuscript has described and demonstrated the use of a practical method to estimate summary cause-specific health indicators based on the integration of MCCOD for hospital deaths with routine VA data for community deaths, taking into account the common situation of incomplete reporting of the fact of death. We demonstrate the applicability of the method in a nationally representative sample of 42 townships in Myanmar to produce estimates of the SDG premature mortality indicator pertaining to NCDs. The higher CSMFs for these causes based on the integrated data, compared with what is suggested from hospital data alone (which only comprise a minority of all deaths in the country), demonstrate the importance of collecting routine VA data to ascertain the causes of community deaths and the policy benefits of integrating community CSMFs with hospital CSMFs; otherwise, using just hospital data would underestimate the magnitude of premature NCD mortality in the country. The resultant estimates of the probability of dying from any of the four NCDs included in the SDG indicator between the ages of 30 and 70 years in 2019 of 0.265 for men and 0.216 for women are similar to those made by the WHO for 2016 for Myanmar of 0.27 for men and 0.21 for women, but lower than the 0.304 for men and higher than the 0.199 for women calculated using GBD mortality and COD estimates.22 30 31 Importantly, rather than relying on indirect estimates using the complex methods of the GBD or WHO, largely based on covariates with little or no local data, the method is straightforward enough to enable local authorities to use the country’s own data to monitor progress with national, and crucially, subnational health goals. The method also overcomes the vague and generally unhelpful guidance currently available for countries about how to estimate indicators required for monitoring progress with global health and development goals.

A strength of the integration method is that it is appropriate for countries where there is incomplete death reporting, because it employs the empirical completeness method to reliably estimate the extent of incompleteness. In many LMICs, incomplete death reporting is unfortunately likely to be the reality for the foreseeable future, and hence the ability to reliably estimate the volume of mortality both in and outside hospitals is key to facilitating the integration of COD estimates from different sources, in particular VA for home deaths. In the case of Myanmar, cost-effective approaches using automated VA have been demonstrated to work well in a nationally representative sample of the population, yielding important information on community CODs that did not exist before.10 It is important though that users of the integration method are able to understand the quality and plausibility of the COD data that result using the ANACONDA and VIPER tools, and when data quality and completeness is too poor for the integration to proceed.20 24 For example, in Myanmar VA diagnoses of the COD are available for an estimated 70% of community deaths; if the cause pattern for the 30% of deaths without a VA is significantly different, this will bias community cause patterns. Similarly, if the completeness of VA reporting is below about 50%, or if the fraction of undetermined (ill-defined) CODs exceeds about 20%, then the integration should probably not proceed. Users should also be careful about making inferences about national level cause patterns if the data used for the integration are not from a nationally representative sample.

There are some limitations of the integration method presented here, many of which are related to the quality of the available data. Where the reporting of the fact of death is incomplete then, despite the use of a reliable method to estimate completeness, there will still be some residual uncertainty about the total number of deaths and how they are distributed by age and sex, and also between hospitals and the community. Local knowledge of the quality of hospital reporting and plausibility of the proportion of total deaths that occur in hospitals is important to identify any clear errors in estimates. In the example of Myanmar, the low proportion of deaths that occur in hospitals implies that any error in the estimation of this number would likely only have a small impact on the resultant COD estimates. That would not be the case for errors in the levels and cause structure of community deaths, however. Incomplete death reporting or high levels of ill-defined or undetermined causes can also bias the quality of COD estimates, and in particular will affect the representativeness of results if these data issues are more acute for certain subpopulations of a country. Additionally, small numbers of deaths may create uncertainty about results, which is particularly relevant when applied to subnational populations. Another limitation of any integration is that it will be reliant on the cause list employed in VA. As in the case of the Myanmar example, the ICD codes suggested by the WHO for use in the calculation of the premature mortality indicator do not directly map to the VA cause list; however, analysts can use their hospital data to gauge what impact this will have on calculation of this indicator. Despite the fact that integration with a lower number of causes from the VA will necessarily reduce the list of causes for which mortality can be monitored, current VA tools can identify COD from the major conditions likely to be of highest policy relevance for LMICs, some more reliably than others.32 33

Conclusion

To fully realise the benefits of the integration of VA and MCCOD data to calculate national and international health indicators, there is an urgent need to strengthen capacity in National Statistics Offices and Ministries of Health to apply these methods confidently and competently. Analysts in Myanmar from the Central Statistical Organization and Ministry of Health and Sport have been trained remotely in these methods and have successfully estimated nationally representative CSMFs that have recently been published in their Statistical Yearbook.34 This demonstrates the feasibility of widespread application of this integration method by country staff, thus reducing reliance on the indirect GBD or WHO estimates, enabling subnational estimation, building local analytical capacity and enhancing national ownership of the estimates. Such multisource integration methods also demonstrate the need and benefits of investing in nationally representative routine-automated VA systems, especially in populations with a high proportion of deaths in the community, and for strengthened practices to correctly certify and code deaths that occur in hospitals. These strategies will yield more reliable mortality and COD information which in turn will enable countries to measure progress towards health goals and the avoidance of major causes of premature death.