Rod Jackson

It has been a long journey getting cardiovascular disease (CVD) risk prediction recognised as relevant to clinical decision-making. The Framingham Heart Study investigators put CVD risk prediction on the map over 30 years ago with their Framingham CVD risk scores, and Bill Kannel’s seminal 1976 paper advising clinicians to inform their risk management decisions using predicted CVD risk rather than individual risk factor levels has stood the test of time.1 There is now a wealth of supporting evidence that major CVD risk factors like blood pressure or blood lipid levels are not only individually poor predictors of a patient’s CVD risk but also of a patient’s potential to benefit from treatment, when compared with multifactor CVD risk prediction estimates.2 Disappointingly, estimating CVD risk remains the exception rather than the rule in routine clinical practice.3 As a result, CVD risk factor management is poorly targeted4 because risk prediction, like your tax return, is difficult to do in your head.5

CAUSE FOR OPTIMISM?

But there is cause for optimism. Many clinical guidelines on the management of CVD risk factors now incorporate risk prediction scores. Moreover, separate guidelines for managing hypertension and dyslipidaemia—which should not be considered independently—are being replaced by more clinically relevant CVD prevention guidelines.6 7 However, there are several barriers to the widespread acceptance of risk prediction as a prerequisite to high-quality clinical management of CVD risk. First, there are continuing arguments on how and for whom to apply prediction scores and second, the accuracy of existing risk prediction scores is quite modest. Before turning to the problem of accuracy—the subject of a study published in this issue of the journal (see article on page 34)8—some of the other barriers need to be considered.

WHAT RISK PERIOD SHOULD BE USED?

A topic that continues to be debated is the choice of the time period over which risk scores should be estimated—for example, 5 years, 10 years or even over a lifetime. Choosing between 5- or 10-year risk is not a major problem as the latter is just a little more than twice the former and the only important difference is that an equivalent treatment threshold for 10-year risk is about twice that for 5-year risk. Also, those arguing for lifetime risk estimation have simply misunderstood the purpose of clinical risk prediction. Lifetime risk of death is 100% (1 per person per lifetime) and about half of all deaths are cardiovascular in origin in high-income countries. So knowledge of one’s lifetime CVD risk, which is high for almost everyone, informs us that we should all try hard to reduce our risk, but is not helpful in differentiating between patients at low and high risk. For example, after excluding the 3.2% of men with “perfect” risk profiles, lifetime risk of CVD among 50-year old men in the Framingham Heart Study varied between 36.4% and 68.9%.9

Estimating a patient’s probability of a CVD event over the next 5–10 years is much more clinically relevant. It varies far more than a lifetime probability of between one in three and two in three and it is done to help the patient and practitioner decide what they should do today. If 5- or 10-year risk is high, then intensive interventions that lower risk quickly and significantly should be considered now. If 5- or 10-year CVD risk is low, there is more time to try lower-intensity “lifestyle” interventions that may take longer to reduce risk. Moreover, most of a person’s risk, compared with someone else of the same age, appears to be reversible within a few years.10 So among patients at low risk, there is limited gain from the early initiation of intensive expensive interventions with their associated adverse effects.

Are there any good reasons to chose a 5 rather than 10-year risk period? As most trials of CVD risk factor interventions are based on at most 6 years’ follow-up,10 11 5-year estimates of risk and treatment benefit are probably more accurate than 10-year estimates.

AT WHAT LEVEL OF RISK SHOULD INTERVENTIONS BE RECOMMENDED?

Another contentious issue is the level of predicted risk at which intensive interventions, like drug treatment, should be recommended. Despite the heated debates among clinician about appropriate thresholds, it is not actually a clinical decision. Almost all middle-aged and older adults would benefit from lipid-lowering or blood pressure-lowering drugs but the degree of benefit and the cost effectiveness of treatment is directly proportion to the pretreatment level of risk.2 Therefore the threshold at which treatment should be recommended depends on how much the payer is willing to spend and is not clinically generalisable.

RISK PREDICTION IN THE YOUNG AND OLD

Perhaps the most controversial issue on the application of risk prediction scores relates to their use in people in their 30s and 40s or over 75–80 years. Younger people typically have a low predicted 5- or 10-year CVD risk even with multiple “abnormal” individual risk factors, so there is usually plenty of time to try low-intensity interventions without putting them at significant risk, particularly as most of the risk is reversible in a few years.10 But many clinicians find this approach difficult to accept given their traditional training on treating single risk factors. While it is appropriate to fully inform younger patients with raised levels of individual risk factors that if they do not make positive lifestyle changes soon, their longer-term risk will be high, it is not appropriate to recommend intensive treatments if their short-term CVD risk is low.7

The more important age–risk question is treatment of the elderly. Whatever their CVD risk factor profiles, most people over 75–80 years meet recommended risk-based treatment thresholds. However, there is limited trial evidence, particularly in people over 80 years, and some investigators are concerned that treatment simply changes the cause of death rather than prolonging life.12 As treatment benefits are directly proportional to the pretreatment CVD risk, elderly people have the greatest potential to gain from treatment and this issue needs to be resolved. With a rapidly ageing population, trials of CVD risk factor lowering in people over 80 years must be made a high priority.

ACCURACY OF RISK PREDICTION

While the controversies discussed above are likely to be resolved mainly through a more informed debate, the accuracy of risk prediction could be a greater challenge. If risk prediction scores are not sufficiently accurate, they may be more trouble than they are worth, as the traditional treatment approach based on single risk factors is certainly easier.

In this issue of Heart, Hippisley-Cox and colleagues compare the accuracy of their QRISK CVD risk prediction score with a commonly used Framingham score.8 They recently published the QRISK score derived from the world’s largest ever CVD risk prediction study13 and now they present the world’s largest independent validation study. QRISK appeared to be more accurate than the Framingham score and new Scottish ASSIGN score14 when validated against a one-third random sample of their original cohort; the remaining two-thirds of the cohort were used to derive the QRISK score.13 However, the authors were concerned that because both cohorts came from the same population, QRISK might have had a “home advantage”. Their independent validation published here, using a database based on an alternative UK electronic primary care practice system, confirms the previous validation.

THE QRISK PROJECT

The QRISK project is significant for several reasons. First, all previous risk scores have been derived from predefined cohorts, which are complex and costly to establish, whereas QRISK is based on routinely collected data. This accounts for the much greater size of QRISK, compared with previous risk prediction studies, as data can be relatively cheaply extracted from electronic primary care records generated as part of usual practice. Using routine data from a large population also means that the QRISK score is likely to be more generalisable than scores derived from cohorts based on volunteers.

Second, despite a substantial amount of missing data—for example, 60–70% of patients had missing blood lipids, the QRISK score performed better than scores based on studies with much more complete and accurate data. This signals the huge potential of routinely collected data for generating clinically relevant evidence. Until recently this incredibly rich source of data has been effectively inaccessible because it was stored in paper records in thousands of individual practices. The computerisation of records has the potential to revolutionise much clinical research but will require more standardised and more complete recording of data. The quality and completeness of CVD risk factor data in primary care records will certainly improve with the documentation required for initiatives like the current UK primary care pay for performance programme.

While the QRISK investigators have demonstrated that their score is better than Framingham, just how good is it? The accuracy of a risk prediction score can be judged on two main components—calibration and discrimination. A well-calibrated score is one in which the predicted risk is similar to the observed risk and QRISK is better calibrated to the UK population than the Framingham score. Overall, the predicted QRISK score was 10% lower than the observed risk in the validation cohort for both men and women, whereas the Framingham score was 16% higher than the observed risk for women and 28% higher for men. But calibration doesn’t require a new cohort study and the Framingham investigators have previously described a method of calibrating the Framingham score for different populations.15

The more important component of accuracy is discrimination or the ability of a score to differentiate between people who will have an event from those who will not, over a defined period of time. As discussed, for common conditions like CVD, discrimination is only meaningful over relatively short periods of time, say 5–10 years. At the extreme it is possible to discriminate perfectly between people who will and will not die over a lifetime! There are two types of discrimination measures—summary measures and threshold measures. Receiver operator characteristic curves are summary measures of discrimination. They plot all possible thresholds for discriminating between those who will and will not have a CVD as a continuous curve. However, in clinical practice typically just one drug treatment threshold is chosen, such as the 20% 10-year CVD risk recommended threshold for offering patients statins in UK guidelines.16 Therefore the more clinically relevant measure of discrimination is specific to this threshold.

Unfortunately the QRISK investigators do not report this important measure of discrimination. An early Framingham CVD risk score was reported to be able to identify the 10% of the asymptomatic population at highest risk who accounted for about one-fifth of the CHD events and one-third of the stroke and peripheral vascular events over the following 8 years.17 In a recent Scottish study both Framingham and the ASSIGN scores were able to identify 20% of the asymptomatic population accounting for approximately 45% of CVD events over the following 10 years.14 Given the relatively small differences between summary discrimination measures reported for Framingham and QRISK in the independent validation study, it is likely that the accuracy of discrimination using QRISK, at the 20% 10–year CVD risk treatment threshold, is quite similar to that for the Framingham and ASSIGN scores described above.

CONCLUSION

So are we there yet with CVD risk prediction? Probably not, although compared with the tradition treatment thresholds based on blood pressure or cholesterol levels,18 we are well on the way. But whether one uses a Framingham, ASSIGN or QRISK score, more than 50% of CVD events in the next 10 years among asymptomatic adults in the UK will occur in people below the current guideline drug treatment threshold.

What level of discrimination is possible using readily accessible predictors applicable in a busy primary care setting? A score derived from the prospective cardiovascular Munster Study19 in Germany used a new neural network modelling approach and was able to identify approximately 8% of men aged 35–65 years who accounted for almost 75% of CVD events over the following 10 years. This is probably too good to be true as neural network models are not as generalisable as more standard logistic regression models and no independent validation was done. Nevertheless, the study suggests there is significant potential to improve on current scores, so the journey continues. In the meantime UK primary care practitioners should adopt the QRISK score as well as fully documenting the relevant risk factors and outcomes in their electronic records, so that the next-generation QRISK score is even more accurate. They should also make sure they treat their patients with symptomatic CVD who account for only about 10% of the population but about half of all future CVD events.