You are here

  • Home
  • Archive
  • Volume 9, Issue 8
  • Predicting the risk of stroke among patients with type 2 diabetes: a systematic review and meta-analysis of C-statistics

Article Text

Article menu

Download PDFPDF

Diabetes and endocrinology
Research
Predicting the risk of stroke among patients with type 2 diabetes: a systematic review and meta-analysis of C-statistics
  1. Mohammad Ziaul Islam Chowdhury1,
  2. Fahmida Yeasmin2,
  3. Doreen M Rabi1,
  4. Paul E Ronksley3,
  5. Tanvir C Turin4
  1. 1 Department of Community Health Sciences, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
  2. 2 Department of Mathematics and Statistics, University of Calgary, Calgary, Alberta, Canada
  3. 3 Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
  4. 4 Department of Family Medicine, University of Calgary, Calgary, Alberta, Canada
  1. Correspondence to Dr Tanvir C Turin; chowdhut{at}ucalgary.ca

Abstract

Objective Stroke is a major cause of disability and death worldwide. People with diabetes are at a twofold to fivefold increased risk for stroke compared with people without diabetes. This study systematically reviews the literature on available stroke prediction models specifically developed or validated in patients with diabetes and assesses their predictive performance through meta-analysis.

Design Systematic review and meta-analysis.

Data sources A detailed search was performed in MEDLINE, PubMed and EMBASE (from inception to 22 April 2019) to identify studies describing stroke prediction models.

Eligibility criteria All studies that developed stroke prediction models in populations with diabetes were included.

Data extraction and synthesis Two reviewers independently identified eligible articles and extracted data. Random effects meta-analysis was used to obtain a pooled C-statistic.

Results Our search retrieved 26 202 relevant papers and finally yielded 38 stroke prediction models, of which 34 were specifically developed for patients with diabetes and 4 were developed in general populations but validated in patients with diabetes. Among the models developed in those with diabetes, 9 reported their outcome as stroke, 23 reported their outcome as composite cardiovascular disease (CVD) where stroke was a component of the outcome and 2 did not report stroke initially as their outcome but later were validated for stroke as the outcome in other studies. C-statistics varied from 0.60 to 0.92 with a median C-statistic of 0.71 (for stroke as the outcome) and 0.70 (for stroke as part of a composite CVD outcome). Seventeen models were externally validated in diabetes populations with a pooled C-statistic of 0.68.

Conclusions Overall, the performance of these diabetes-specific stroke prediction models was not satisfactory. Research is needed to identify and incorporate new risk factors into the model to improve models’ predictive ability and further external validation of the existing models in diverse population to improve generalisability.

  • stroke
  • risk
  • prediction model
  • systematic review
  • meta-analysis

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2018-025579

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • The breadth of the comprehensive systematic literature search is a strength of this study.

    • To our knowledge, this is the first study where a meta-analysis and study quality assessment was performed on stroke prediction models in patients with diabetes.

    • We were only able to use C-statistics to compare the model performance, which might be insensitive to identify differences in models’ ability to accurately risk-stratify patients into clinically meaningful risk groups.

    Introduction

    Stroke, also known as a cerebrovascular accident, is the third leading cause of disability and accounted for over 6 million deaths worldwide in 2015.1 2 Diabetes mellitus, characterised by chronic hyperglycaemia due to an absolute or relative deficiency in insulin, is a major risk factor for stroke. People with diabetes are at a twofold- to fivefold increased risk for stroke compared with people without diabetes.3–7 Large clinical trials performed in people with diabetes supports the need for targeted cardiovascular risk reduction strategies to prevent the onset, recurrence and progression of acute stroke.8

    Risk prediction models are statistical tools to estimate the probability that an individual with specific risk factors (eg, diabetes mellitus) will develop a future condition, such as stroke, within a certain time period (eg, 5 years).9 Such tools for the estimation of stroke risk are frequently used to assist in decisions about clinical management for both individuals and populations. Accurate risk prediction of stroke is thus necessary to provide patients with accurate information on the expected benefit from a therapy or intervention. The importance of well-performing prediction models is increasingly being recognised and health researchers continue to develop parsimonious risk prediction models under different scenarios to meet this demand. Model performance statistics, such as C-statistic or AUC (area under the receiver operating characteristic curve) are indicators frequently used to identify models with the best predictive ability. These metrics can be compared and assessed through a formal systematic review and meta-analysis. Performing a systematic review and meta-analysis can also provide a comprehensive quantitative summary of the predictive ability of these models and evaluate their predictive performance within the available literature.

    Risk factors for stroke include lifestyle-related factors,10 11 predisposing medical conditions,10 12 specific genetic diseases,13 14 as well as sociodemographic factors.11 12 Over the past decade, a number of prediction models (or risk scores) have been developed incorporating these risk factors to predict a person’s risk of developing stroke.15 Prediction of stroke is important for a number of reasons: to detect or screen high-risk subjects to prevent developing stroke through early interventions, to facilitate patient–doctor communication based on more objective information and to help patients to make an informed choice regarding their treatment. While multiple stroke prediction models have been proposed in patients with diabetes, little is known about which is the most accurate one. There has also been a lack of consistency in estimating risk across these different models. With this in mind, we aimed to systematically identify all prediction models for stroke that have been applied to patients with diabetes. We characterised the study populations in which these models were derived and validated. We also assessed the predictive performance and generalisability of these stroke prediction models so that the selection of models for clinical implementation can be informed.

    Methods

    Data sources and searches

    Similar to previously employed methodology,16 we searched MEDLINE, EMBASE and PubMed (from database inception to 22 April 2019) for studies predicting the risk of stroke among patients with diabetes. We also searched the reference lists of all identified relevant publications. The search strategy focused on three key elements: diabetes, risk prediction with specific names of known risk scores and stroke. Only studies published in English were considered. The detailed search strategy is given in online supplemental table S1.

    Supplemental material

    Study selection

    Eligible articles were identified by two reviewers independently using a two-step process. First, an initial screen of titles and abstracts was performed. Abstract were retained if they reported data from an original study and reported on the development and/or validation of a stroke risk prediction model for patients with type 2 diabetes. We defined a stroke risk prediction model as one combining two or more independent variables to obtain estimates of the predicted risk for developing stroke. We considered any clinical-based or laboratory-based definition of stroke. Selected abstracts were further screened based on a full-text review. We used broad inclusion criteria to provide an extensive systematic review of the topic. There were no restrictions on study design (eg, cohort study, case–control study), geographic region or age ranges. Studies that developed prediction models for stroke in populations with type 2 diabetes and in the general population were included; however, models that were developed in the general population but did not validate their model in a type 2 diabetes population or models developed on a type 1 diabetes population were excluded. A study was included if the outcome of the prediction model was any type of stroke or stroke that was part of a composite cardiovascular disease (CVD) outcome, but excluded if the outcome was any other cardiovascular conditions (eg, coronary heart disease (CHD), coronary artery disease (CAD), heart failure). We excluded studies that did not predict stroke. Studies on recurrent stroke or other vascular conditions (eg, patients with hypertension) were also excluded. Studies that focused only on the added predictive value of new risk factors to an existing prediction model without reporting the performance of the existing model were excluded. Studies on score-based tools, such as risk charts were also excluded. Agreement between reviewers at the full-text stage was quantified using the kappa statistic. Any disagreement between reviewers was solved through consensus.

    Data extraction

    Data were extracted from the finally selected studies using a standardised form by two reviewers. Information collected from each study included, outcome of the prediction model, location where the model was developed, predictors included in the model, age and gender of the study participants, number of events, duration of follow-up, modelling method used, measures of discrimination and calibration of the prediction model and the external validation of the prediction model. For the external validation studies, a different data extraction sheet was used. The collected information included specifics of the validation population, number of events, type of outcome, statistical tests and measures of discrimination, and calibration of the prediction model. Study quality was assessed using the CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies) Checklist.17 The following items were evaluated for each study: Was inclusion/exclusion criteria for study participants specified?; Was there non-biased selection of study participants?; Did the authors discuss or consider missing values/information?; Was there blinded assessment of the outcome?; Was duration of follow-up adequate?; Were modelling assumptions satisfied?; Was the model externally validated? and Was the potential clinical utility of the model discussed in light of study limitations?

    Data analysis

    The selection process for this systematic review and meta-analysis is summarised using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.18 Discrimination is defined as any assessment of the ability of the model to differentiate between subjects who will develop stroke from those who will not. The discrimination of a prediction model is often assessed using the concordance or C-statistic (also known as AUC). Calibration is defined as any report of the agreement between predicted probabilities and observed probabilities. Calibration is assessed using goodness-of-fit tests (eg, Hosmer-Lemeshow test), calibration slopes, tabular or graphical comparisons of predicted versus observed values within groupings of predicted risk or calibration plots. In studies that only provided a C-statistic but no measure of its variance, the SE and 95% CI of the AUC (C-statistic) was calculated using the formula:

    Embedded Image

    where Embedded Image the number of patients with stroke and Embedded Image the number of patients without stroke and the upper 95% CI Embedded Image , and lower 95% CI Embedded Image .19 The summary statistic from the individual studies was the C- statistic or AUC. We grouped studies based on the outcome of the risk prediction models developed in diabetes populations, whether stroke was the primary outcome of the model or stroke was a part of composite CVD outcome. Random effects meta-analysis was used to obtain the pooled weighted average C-statistic with 95% CIs for common groups of models using the DerSimonian and Laird method.20 Heterogeneity was assessed using the Cochran Q and the I2 statistic and was explored using meta-regression and stratified analyses according to model outcomes. Small study effects were examined using funnel plots and Begg’s test. The analyses were performed using Stata version 13.1 (Stata, College Station, Texas, USA) using the metan, metareg, metabias and metafunnel commands.

    Patient and public involvement

    There was no direct patient or public involvement in this review.

    Results

    The search retrieved 21 797 citations (after duplicate removal) with an additional 63 potentially relevant papers retrieved from our grey literature search. After title and abstract screening, 262 studies were selected for full-text screening. After examining the full-text papers, 56 studies remained (reasons for exclusion stated in figure 1), describing 38 models predicting stroke in patients with diabetes. Agreement between reviewers on the final articles eligible for inclusion in the systematic review was good (κ=0.83). Of these 38 models, 34 were specifically developed in patients with diabetes and 4 were developed in the general population but later externally validated in patients with diabetes. Among the models developed in patients with diabetes, nine models reported their outcome as stroke and presented a corresponding performance measure (C-statistic) for the models. Twenty-three models reported their outcome as a composite CVD outcome where stroke was one of the components and presented the model’s performance measure (C-statistic) for the composite CVD outcome. Among the models developed in the general population, one model reported its outcome as stroke and three models reported a composite CVD outcome, which included stroke. Of these 38 prediction models, 17 were validated by 33 studies (some studies validated more than one model in the same study), of which 10 models had multiple validations, 7 models had a single validation and 21 models were not validated. Among the models with multiple validations, eight models were developed in a diabetes population (validated by 31 studies) and two were developed in the general population (validated by four studies). United Kingdom Prospective Diabetes Study (UKPDS) Risk Engine for Stroke by Kothari et al 21 was the most validated risk score (validated by 12 studies). Figure 1, describes the systematic selection process of studies presenting a stroke prediction model applicable to patients with diabetes.

    Figure 1
    Figure 1

    PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram for systematic review of studies presenting stroke prediction models developed or validated in individuals with diabetes.

    Predicting the risk of stroke within models developed in populations with diabetes

    Table 1 describes the study characteristics of the nine risk prediction models developed in diabetes populations and presented a corresponding performance measure. The number of participants ranged from 1748 to 26 140 in the model development. The outcome of most models was stroke regardless of type. Duration of follow-up (total/median/mean) ranged from 501 days to 10.5 years with six models having ≥5 years of follow-up (defined as long duration) and three models with <5 years of follow-up. Most of the prediction models were developed using Cox proportional hazards modelling techniques. The number of predictors included in the prediction models ranged from 4 to 29 with an average of 11 predictors per model. Several predictors were common to multiple models including age, sex, duration of diagnosed diabetes, systolic blood pressure and haemoglobin A1c (HbA1c). Only two models were externally validated after their development and four of them had never been validated in an external population. Calibration of the prediction model was reported by six studies (most commonly using the Hosmer-Lemeshow test). Discrimination was assessed using the C-statistic (or AUC) and reported by six models with values ranging from 0.64 to 0.80. The median C-statistic of the models was 0.71 with a large amount of unexplained heterogeneity in the discriminative performance of these models (I2=94.6%; Cochran Q-statistic p<0.001; figure 2). Stratifying pooled results by sample size (small vs large, p=0.19), follow-up time (short vs long, p=0.60), variables included in the model (few vs many, p=0.24) and geographic location (Asia vs others, p=0.60) did not explain the observed heterogeneity in the discriminative performance of these models. The discriminative ability of the model by Kiadaliri et al 22 was highest (C-statistic=0.80). The funnel plot and Begg’s test (p>0.999) suggested the absence of small study effects, with no correlation between studies of smaller cohorts reporting higher C-statistics (online supplemental figure S1).

    Figure 2
    Figure 2

    Forest plot of C-statistics, with 95% CIs of risk prediction models when outcome was reported for stroke.

    View this table:
    Table 1

    Characteristics of prediction models when outcome and corresponding performance measure (C-statistic) were reported for stroke

    A set of nine criteria was used to assess the quality of the studies and was summarised in table 2. All the studies specified inclusion/exclusion criteria. Non-biased selection of study participants was clear in all studies except the study by Palmer et al.23 Handling of missing values was reported in four (44%) studies, modelling assumptions was satisfied by two studies and model external validation was performed in two studies. Stevens et al was the only study to mention whether the outcome was assessed without knowledge of the candidate predictors.24 Duration of follow-up was long (≥5 years) in six models (67%). The clinical utility of the models was discussed in six (67%) studies and almost all studies reported their study limitations.

    View this table:
    Table 2

    Study quality assessment of prediction models when outcome and corresponding performance measure (C-statistic) were reported for stroke

    Predicting risk of stroke (as part of a composite CVD outcome) in populations with diabetes

    We identified 23 models developed in diabetes populations that reported their outcome as a composite of CVD. A summary of the characteristics of these prediction models is described in table 3. The number of participants considered in the model development ranged from 132 to 1 81 619 with an average age of >50 years. Duration of follow-up (mean, median, maximum) ranged from 11 months to 11.8 years with 14 models with ≥5 years and nine models with <5 years of follow-up. The number of predictors included in prediction models ranged from 4 to 18 with an average of 11 predictors per model. The most common predictors included in the models were age, sex, systolic blood pressure and HbA1c, smoking and high-density lipoprotein-cholesterol. Four models were externally validated after its development and 17 of them had never been validated in an external population. Calibration of the prediction models was reported by 13 studies while discrimination reported by almost all studies with C-statistics ranging from 0.60 to 0.92. The median C-statistic of the models was 0.70 with a large amount of unexplained heterogeneity in the discriminative performance of these models (I2=93.7%; Cochran Q-statistic p<0.001; figure 3). Sample size (small vs large, p=0.46), models’ external validation (externally validated vs not externally validated, p=0.71), variables included in the model (few vs many, p=0.21) and geographic location (Europe vs others, p=0.08) were not identified as significant sources of heterogeneity in the discriminative performance of these models. The discriminative ability of the model by Ofstad et al 25 was highest (C-statistic=0.92) when novel risk markers were added to their standard model. The funnel plot and Begg’s test (p=0.24) suggested the absence of small study effects, with no correlation between studies of smaller cohorts reporting higher C-statistics (online supplemental figure S2). Only four models were developed using logistic regression models while others were developed mostly using Cox proportional hazards models.

    Figure 3
    Figure 3

    Forest plot of C-statistics, with 95% CIs of risk prediction models when stroke was reported as part of a composite cardiovascular disease outcome. AD-ON, Action in Diabetes and Vascular Disease:Preterax and Diamicron Modified Release Controlled Evaluation-Observational; EPIC-NL,European Prospective Investigation into Cancer and Nutrition-Netherlands; HWNNs,hybrid wavelet neural networks; MACE, major adverse cardiovascular event; NZDCS, New Zealand Diabetes Cohort Study; SMART, second manifestations ofarterial disease; SOMs, self-organising maps.

    View this table:
    Table 3

    Characteristics of prediction models when stroke is reported as a part of composite CV outcome and performance measure (C-statistic) is presented for the composite CV outcome

    The study quality for this group of models is summarised in table 4. Similar to the models developed in diabetic populations that look at the outcome of stroke specifically, we found that study quality was similar.

    View this table:
    Table 4

    Study quality assessment of prediction models when stroke is reported as a part of composite CV outcome and performance measure (C-statistic) is presented for the composite CV outcome

    Validation studies of stroke prediction models developed in populations with and without diabetes

    Seventeen risk prediction models for stroke (developed both in patients with diabetes and in general populations) were validated in diabetes populations by 33 studies (table 5). Among the 17 validated models, 14 of them were externally validated in independent cohorts and 3 of them were internally validated in a test sample or separate data set from the same cohort. Three studies validated more than one risk model in the same cohort. Models with multiple validations (two or more) and reported C-statistics are provided in figure 4. Models that had only been validated once were excluded from meta-analysis. In addition, only those studies that provided enough information to estimate the variance of the provided C-statistic for meta-analysis were considered for analysis.

    Figure 4
    Figure 4

    Forest plot of C-statistics, with 95% CIs, of stroke prediction models that are externally validated in two or more independent cohorts. JHS, Jackson Heart Study; MESA,Multiethnic Study of Atherosclerosis; UKPDS, United Kingdom ProspectiveDiabetes Study

    View this table:
    Table 5

    Characteristics of the validation studies of the stroke prediction models

    UKPDS Risk Engine for Stroke by Kothari et al 21 was the most frequently externally validated model with a total of 12 studies reporting its performance in different diabetes cohorts. In 12 external validation studies, a total of 126 323 patients were included with considerable variations in sample sizes across the different studies. The pooled C-statistic for the model by Kothari et al 21 was 0.72 (95% CI, 0.68 to 0.75), with high heterogeneity identified (I2=95%; Cochran Q statistic p<0.001). Stratification by sample size (small vs large, p=0.69), geographic location (Asia vs others, p=0.09) and stroke type (fatal vs non-fatal, p=0.07) did not explain the observed heterogeneity in the discriminative performance of this model. UKPDS Risk Engine by Stevens et al 26 was the second most externally validated model with five validation studies including 2826 patients. One study did not report the number of participants and none of the studies reported C-statistics. As a result, a pooled C-statistic and heterogeneity was not possible to assess for this model. The UKPDS Outcomes Model by Clarke et al 27 was externally validated by four studies including 65 056 patients. The pooled C-statistic was 0.66 (95% CI, 0.61 to 0.71) with high heterogeneity between studies (I2=84.5%; Cochran Q statistic p=0.002). Similar to the UKPDS Risk Engine for Stroke,21 stratification across select study characteristics did not explain the observed heterogeneity. The Framingham risk score by Anderson et al 28 was externally validated in two studies including 8574 patients. The pooled C-statistic was 0.58 (95% CI, 0.54 to 0.61) with non-significant heterogeneity between studies (I2=56.1%; Cochran Q statistic p=0.102). The Framingham risk score by D’Agostino et al 29 was externally validated in two studies including 7604 patients. One study (Ataoglu et al 30) did not report the C-statistic for the model and one study (Kengne et al 31) reported two C-statistic values, one for major events and one for any event. The pooled C-statistic for these two values was 0.58 (95% CI, 0.55 to 0.60). Models by Mukamal et al.,32 Davis et al.,33 Kengne et al 34 and Zethelius et al 35 each were externally validated by two studies with pooled C-statistics of 0.67 (95% CI, 0.67 to 0.68), 0.75 (95% CI, 0.58 to 0.92), 0.67 (95% CI, 0.65 to 0.69) and 0.69 (95% CI, 0.63 to 0.75), respectively. Observed heterogeneity was high in models by Davis et al 33 and Zethelius et al 35 while low in models by Mukamal et al 32 and Kengne et al.34 The model by Basu et al 36 was externally validated by two studies in three different population yielding a pooled C-statistic of 0.71 (95% CI, 0.67 to 0.76) with moderate heterogeneity between studies (I2=56.8%; Cochran Q statistic p=0.099).

    Separate models by D'Agostino et al (Framingham Stroke Risk Score),37 Yang et al (Hong Kong Diabetes Registry for Stroke),38 Kiadaliri et al,22 Stevens et al (UKPDS 66),24 Hippisley-Cox et al (QRISK2),39 Elley et al (New Zealand Diabetes Cohort Study)40 and Alrawahi et al 41 were each validated in one external or separate cohort with sample sizes ranging from 178 to 1 81 399 patients. For the studies that reported discrimination, C-statistics ranged from 0.67 to 0.79. In addition, calibration assessed by calibration plots and Hosmer-Lemeshow tests found good calibration in most studies.

    The overall pooled C-statistic for all validation studies was 0.68 (95% CI, 0.67 to 0.70) with high heterogeneity between studies (I2=95.3%; Cochran Q statistic p<0.001). Models that were developed in diabetes population showed significantly higher C-statistics than models developed in general populations (meta-regression p=0.001). Models, where stroke was reported as the main outcome as opposed to part of a composite CVD outcome, did show borderline significantly higher C-statistics (meta-regression p=0.052), although the value of the C-statistic is still low. This observed difference in the two models makes sense as models that include stroke as part of a composite outcome are expected to be different from models where stroke is the only outcome. A summary describing the characteristics of the studies where prediction models were developed in general populations but validated in patients with diabetes is presented in table 5.

    Discussion

    This systematic review and meta-analysis provides an overview of all stroke prediction models that were specifically developed for, or validated in patients with diabetes to calculate future stroke risk. Thirty-four stroke prediction models were identified that were specifically designed for patients with diabetes and only 32% of these prediction models have been externally validated, with varying results. Overall, the pooled C-statistics were poor for most models. Four of the prediction models identified were originally developed in the general population but externally validated in diabetes populations. The most notable prediction model was the UKPDS Risk Engine for Stroke21 with 12 validation studies. Ten stroke prediction models had multiple validations, seven models had single validations and twenty-one had no validations at all. It is difficult to assess model performance for those with no validation or single validations. Additional validation studies on the performance of stroke prediction models in different diabetes populations are needed. Since stroke prediction models developed in the general population may not account for specific risk factors related to diabetes, using risk scores developed specifically in the diabetes population will help to estimate stroke risk among people with diabetes more accurately.

    None of the models showed good discriminative performance consistently when externally validated. The model by Kothari et al 21 where the stroke was the primary outcome showed moderate discriminative performance (pooled C-statistic=0.72). Since this model was externally validated multiple times, the performance of this model can be considered as consistent. The discriminative ability of stroke prediction models where stroke was the primary outcome and models where stroke was a part of composite CVD outcome were modest, with C-statistics often less than 0.70.42 Meta-analyses of the C- statistic suggests that there is significant between-study heterogeneity in the models where stroke is reported as the primary outcome and in those where stroke is reported as part of composite CVD outcome. Further, the possible sources of heterogeneity are unexplained. Perhaps the difference in patient characteristics in the different cohorts could be a potential source of heterogeneity; however, geographic location, sample size, follow-up time, external validation and variables included in the models were not significant sources of heterogeneity in meta-regression.

    The discrimination of the 17 models that were validated were generally comparable with those observed in the development cohorts. However, the performance of some models externally validated in multiple cohorts was heterogeneous and possible source for this heterogeneity remains unexplained. There was also variability in prediction model quality and the methodology used in developing them. Our study findings suggest that, from a large number of published models in patients with diabetes, very few well-validated models are available for stroke prediction. This is helpful to inform the determination of models for clinical uptake when risk stratification approaches for stroke are implemented.

    No evidence of small-study effects was detected, in which smaller studies reported better discrimination of models for predicting stroke. Study quality assessment shows many of the models failed to meet some key criteria: consideration of missing values, modelling assumptions, model validation and blinded outcome assessment, which is a concern. Many studies lacked standard reporting. This, to some extend, may be due to lack of guidelines for standards of reporting for risk prediction studies during that time. Many authors reported different aspects of prediction models, and in varying ways created difficulty in collecting information. The publication of new guidelines such as Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD)43 has been introduced and may help improve reporting standards in subsequent studies in this area.

    In prior reviews examining risk prediction models in adults with diabetes (Chamnan et al 44 van Dieren et al,45 and Chowdhury et al 16), all components of cardiovascular disease such as CHD, stroke, CAD, myocardial infarction, heart failure were considered as outcomes of the prediction model. Our review adds to knowledge on predicting risk of stroke in persons with diabetes in the following ways: (1) We only considered models where the primary outcome of the model was stroke or when stroke was part of a composite CVD outcome and corresponding C-statistic were provided; (2) We did not consider other components of CVD as outcomes of the model and therefore our estimates of model performance are more specific to stroke; (3) We have identified and included several recently derived models and conducted meta-analyses to explore reasons for variability in the discriminative performance across models and (4) We provide a detailed assessment of quality of studies among models developed in diabetes populations. Only one prior study16 in this area performed a meta-analysis of model performance statistics across multiple studies or assessed study quality.

    One of the major strengths of our study is the breadth of the systematic search, which included three different databases and extensive use of reference lists of the identified studies. Therefore, it is unlikely that any stroke prediction model-related studies have been missed. To best of our knowledge, this is the first study, where a meta-analysis and study quality assessment was performed on stroke prediction models in patients with diabetes. Nonetheless, there are few limitations in our study, which need to be kept in mind. In this paper, we only considered studies that developed or validated stroke prediction models within patients with diabetes. While prediction models for stroke have been developed for patients with other potential risk factors (eg, patients with hypertension), we felt that an exploration of a broad range of risk factors was outside the scope of this review. Though the inclusion of all stroke prediction models (regardless of the underlying risk factor(s)) could potentially improve the generalisability of our findings, it could have also increased the between-study heterogeneity, making the pooled estimates more difficult to interpret. We also did not consider non-English publications. Although, the English language is generally perceived to be the universal language of science, selection of research findings in a particular language can introduce language bias and may lead to erroneous conclusions. With this in mind, readers should to be cautious when interpreting the findings of our results. Finally, we were only able to use C-statistics to compare the model performance, which might be insensitive to identify differences in the ability of models to accurately risk-stratify patients into clinically meaningful risk groups.46 In addition, meta-analysis of calibration measures (eg, E/O ratio) along with C-statistics could give a comprehensive summary of the performance of these models.

    Our findings suggest that there is no significant difference between the discrimination of models where stroke was the primary outcome and stroke was part of composite CVD outcome. Models, particularly those that have never been validated or validated once need to undergo further external validation in which they will be used with or without recalibration or model updating to better understand the comparative performance of these models.

    Conclusions

    In conclusion, we have identified many models for predicting stroke in patients with diabetes and attempted to compare these models. Only a small number of models have undergone external validation and might provide generalisable predictions that would support their use in another clinical setting. It is difficult to choose one model over another as none of these models exhibited superior discriminative performance, and unfortunately, no single model appears to perform consistently well. It could be argued that risk prediction in patients with diabetes is not essential. Persons with diabetes are generally perceived to be at elevated risk of stroke and the current practice is to treat to common HbA1C, blood pressure and low-density lipoprotein targets based on diabetes status alone and not on calculated risk. This non-risk based approach may be leading to unnecessary overtreatment and the absence of high-quality validated risk prediction models which limits our ability to assess whether more targeted approaches are possible. Further research is warranted to identify new risk factors with high associated relative risk to improve the currently available prediction models.

    References

    1. 1.↵
      1. WHO
      . The top 10 causes of death. Available: http://www.who.int/mediacentre/factsheets/fs310/en/
    2. 2.↵
      1. Global Health Estimates
      . Geneva: World Health organization, 2012. Available: http://www.who.int/healthinfo/global_burden_disease/en/ [Accessed cited 2017 August 10].
    3. 3.↵
      Prevalence of small vessel and large vessel disease in diabetic patients from 14 centres. the world health organisation multinational study of vascular disease in diabetics. diabetes Drafting group. Diabetologia 1985;28(Suppl):615–40.
      OpenUrlCrossRefPubMed
    4. 4.↵
      2. Stamler J ,
      3. Vaccaro O ,
      4. Neaton JD , et al
      . Diabetes, other risk factors, and 12-yr cardiovascular mortality for men screened in the multiple risk factor intervention trial. Diabetes Care 1993;16:434–44.doi:10.2337/diacare.16.2.434
      OpenUrlAbstract/FREE Full Text
    5. 5.↵
      2. Folsom AR ,
      3. Rasmussen ML ,
      4. Chambless LE , et al
      . Prospective associations of fasting insulin, body fat distribution, and diabetes with risk of ischemic stroke. The Atherosclerosis risk in communities (ARIC) study Investigators. Diabetes Care 1999;22:1077–83.doi:10.2337/diacare.22.7.1077
      OpenUrlAbstract/FREE Full Text
    6. 6.↵
      2. Lehto S ,
      3. Rönnemaa T ,
      4. Pyörälä K , et al
      . Predictors of stroke in middle-aged patients with non-insulin-dependent diabetes. Stroke 1996;27:63–8.doi:10.1161/01.STR.27.1.63
      OpenUrlAbstract/FREE Full Text
    7. 7.↵
      2. Karapanayiotides T ,
      3. Piechowski-Jozwiak B ,
      4. van Melle G , et al
      . Stroke patterns, etiology, and prognosis in patients with diabetes mellitus. Neurology 2004;62:1558–62.doi:10.1212/01.WNL.0000123252.55688.05
      OpenUrlCrossRefPubMed
    8. 8.↵
      2. Idris I ,
      3. Thomson GA ,
      4. Sharma JC
      . Diabetes mellitus and stroke. Int J Clin Pract 2006;60:48–56.doi:10.1111/j.1368-5031.2006.00682.x
      OpenUrlCrossRefPubMedWeb of Science
    9. 9.↵
      2. Meads C ,
      3. Ahmed I ,
      4. Riley RD
      . A systematic review of breast cancer incidence risk prediction models with meta-analysis of their performance. Breast Cancer Res Treat 2012;132:365–77.doi:10.1007/s10549-011-1818-2
      OpenUrlCrossRefPubMed
    10. 10.↵
      2. O'Donnell MJ ,
      3. Xavier D ,
      4. Liu L , et al
      . Risk factors for ischaemic and intracerebral haemorrhagic stroke in 22 countries (the INTERSTROKE study): a case-control study. The Lancet 2010;376:112–23.doi:10.1016/S0140-6736(10)60834-3
      OpenUrl
    11. 11.↵
      2. Sacco RL ,
      3. Benjamin EJ ,
      4. Broderick JP , et al
      . Risk factors. Stroke 1997;28:1507–17.doi:10.1161/01.STR.28.7.1507
      OpenUrlFREE Full Text
    12. 12.↵
      2. Boehme AK ,
      3. Esenwa C ,
      4. Elkind MSV , et al
      . Stroke risk factors, genetics, and prevention. Circ Res 2017;120:472–95.doi:10.1161/CIRCRESAHA.116.308398
      OpenUrlAbstract/FREE Full Text
    13. 13.↵
      2. Tuttolomondo A ,
      3. Pecoraro R ,
      4. Simonetta I , et al
      . Neurological complications of Anderson-Fabry disease. Curr Pharm Des 2013;19:6014–30.doi:10.2174/13816128113199990387
      OpenUrlCrossRefPubMed
    14. 14.↵
      2. Tuttolomondo A ,
      3. Pecoraro R ,
      4. Simonetta I , et al
      . Anderson-Fabry disease: a multiorgan disease. Curr Pharm Des 2013;19:5974–96.doi:10.2174/13816128113199990352
      OpenUrl
    15. 15.↵
      2. Brownrigg JRW ,
      3. de Lusignan S ,
      4. McGovern A , et al
      . Peripheral neuropathy and the risk of cardiovascular events in type 2 diabetes mellitus. Heart 2014;100:1837–43.doi:10.1136/heartjnl-2014-305657
      OpenUrlAbstract/FREE Full Text
    16. 16.↵
      2. Chowdhury MZI ,
      3. Yeasmin F ,
      4. Rabi DM , et al
      . Prognostic tools for cardiovascular disease in patients with type 2 diabetes: a systematic review and meta-analysis of C-statistics. J Diabetes Complications 2019;33:98–111.doi:10.1016/j.jdiacomp.2018.10.010
      OpenUrl
    17. 17.↵
      2. Moons KGM ,
      3. de Groot JAH ,
      4. Bouwmeester W , et al
      . Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the charms checklist. PLoS Med 2014;11:e1001744.doi:10.1371/journal.pmed.1001744
    18. 18.↵
      2. Moher D ,
      3. Liberati A ,
      4. Tetzlaff J , et al
      . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097.doi:10.1371/journal.pmed.1000097
    19. 19.↵
      2. Hanley JA ,
      3. McNeil BJ
      . The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29–36.doi:10.1148/radiology.143.1.7063747
      OpenUrlCrossRefPubMedWeb of Science
    20. 20.↵
      2. Egger M ,
      3. Davey-Smith G ,
      4. Altman D
      , eds. Systematic reviews in health care: meta-analysis in context. John Wiley & Sons, 2008.
    21. 21.↵
      2. Kothari V ,
      3. Stevens RJ ,
      4. Adler AI , et al
      . UKPDS 60: risk of stroke in type 2 diabetes estimated by the UK prospective diabetes study risk engine. Stroke 2002;33:1776–81.doi:10.1161/01.str.0000020091.07144.c7
      OpenUrlAbstract/FREE Full Text
    22. 22.↵
      2. Ahmad Kiadaliri A ,
      3. Gerdtham U-G ,
      4. Nilsson P , et al
      . Towards renewed health economic simulation of type 2 diabetes: risk equations for first and second cardiovascular events from Swedish register data. PLoS One 2013;8:e62650.doi:10.1371/journal.pone.0062650
    23. 23.↵
      2. Palmer CNA ,
      3. Kimber CH ,
      4. Doney ASF , et al
      . Combined effect of inflammatory gene polymorphisms and the risk of ischemic stroke in a prospective cohort of subjects with type 2 diabetes: a Go-DARTS study. Diabetes 2010;59:2945–8.doi:10.2337/db09-1690
      OpenUrlAbstract/FREE Full Text
    24. 24.↵
      2. Stevens RJ ,
      3. Coleman RL ,
      4. Adler AI , et al
      . Risk factors for myocardial infarction case fatality and stroke case fatality in type 2 diabetes: UKPDS 66. Diabetes Care 2004;27:201–7.doi:10.2337/diacare.27.1.201
      OpenUrlAbstract/FREE Full Text
    25. 25.↵
      2. Ofstad AP ,
      3. Gullestad L ,
      4. Orvik E , et al
      . Interleukin-6 and activin A are independently associated with cardiovascular events and mortality in type 2 diabetes: the prospective Asker and Bærum cardiovascular diabetes (ABCD) cohort study. Cardiovasc Diabetol 2013;12:126.doi:10.1186/1475-2840-12-126
    26. 26.↵
      2. Stevens RJ ,
      3. Kothari V ,
      4. Adler AI , et al
      . The UKPDS Risk Engine: a model for the risk of coronary heart disease in type II diabetes (UKPDS 56). Clin Sci 2001;101:671–9.doi:10.1042/cs1010671
      OpenUrlCrossRefPubMed
    27. 27.↵
      2. Clarke PM ,
      3. Gray AM ,
      4. Briggs A , et al
      . A model to estimate the lifetime health outcomes of patients with type 2 diabetes: the United Kingdom Prospective Diabetes Study (UKPDS) outcomes model (UKPDS No. 68). Diabetologia 2004;47:1747–59.doi:10.1007/s00125-004-1527-z
      OpenUrlCrossRefPubMedWeb of Science
    28. 28.↵
      2. Anderson KM ,
      3. Odell PM ,
      4. Wilson PW , et al
      . Cardiovascular disease risk profiles. Am Heart J 1991;121:293–8.doi:10.1016/0002-8703(91)90861-B
      OpenUrlCrossRefPubMedWeb of Science
    29. 29.↵
      2. D’Agostino RB ,
      3. Vasan RS ,
      4. Pencina MJ , et al
      . General cardiovascular risk profile for use in primary care: the Framingham heart study. Circulation 2008;117:743e53.
    30. 30.↵
      2. Ataoglu HE ,
      3. Saler T ,
      4. Uzunhasan I , et al
      . Improvement of cardiovascular risk assessment in type 2 diabetes by the addition of carotid artery intima–media thickness to Framingham risk score: a retrospective cohort study. J Diabetes 2009;1:188–93.
      OpenUrlPubMed
    31. 31.↵
      2. Kengne AP ,
      3. Patel A ,
      4. Colagiuri S , et al
      . The Framingham and UK prospective diabetes study (UKPDS) risk equations do not reliably estimate the probability of cardiovascular events in a large ethnically diverse sample of patients with diabetes: the action in diabetes and vascular disease: Preterax and Diamicron-MR controlled evaluation (advance) study. Diabetologia 2010;53:821–31.doi:10.1007/s00125-010-1681-4
      OpenUrlCrossRefPubMedWeb of Science
    32. 32.↵
      2. Mukamal KJ ,
      3. Kizer JR ,
      4. Djoussé L , et al
      . Prediction and classification of cardiovascular disease risk in older adults with diabetes. Diabetologia 2013;56:275–83.doi:10.1007/s00125-012-2772-1
      OpenUrlCrossRefPubMed
    33. 33.↵
      2. Davis WA ,
      3. Knuiman MW ,
      4. Davis TME
      . An Australian cardiovascular risk equation for type 2 diabetes: the Fremantle diabetes study. Intern Med J 2010;40:286–92.doi:10.1111/j.1445-5994.2009.01958.x
      OpenUrlCrossRefPubMedWeb of Science
    34. 34.↵
      2. Kengne AP ,
      3. Patel A ,
      4. Marre M , et al
      . Contemporary model for cardiovascular risk prediction in people with type 2 diabetes. Eur J Cardiovasc Prev Rehabil 2011;18:393–8.doi:10.1177/1741826710394270
      OpenUrlCrossRefPubMedWeb of Science
    35. 35.↵
      2. Zethelius B ,
      3. Eliasson B ,
      4. Eeg-Olofsson K , et al
      . A new model for 5-year risk of cardiovascular disease in type 2 diabetes, from the Swedish national diabetes register (NDR). Diabetes Res Clin Pract 2011;93:276–84.doi:10.1016/j.diabres.2011.05.037
      OpenUrlCrossRefPubMed
    36. 36.↵
      2. Basu S ,
      3. Sussman JB ,
      4. Berkowitz SA , et al
      . Development and validation of risk equations for complications of type 2 diabetes (RECODe) using individual participant data from randomised trials. Lancet Diabetes Endocrinol 2017;5:788–98.doi:10.1016/S2213-8587(17)30221-8
      OpenUrl
    37. 37.↵
      2. D'Agostino RB ,
      3. Wolf PA ,
      4. Belanger AJ , et al
      . Stroke risk profile: adjustment for antihypertensive medication. The Framingham study. Stroke 1994;25:40–3.doi:10.1161/01.STR.25.1.40
      OpenUrlAbstract/FREE Full Text
    38. 38.↵
      2. Yang X ,
      3. So W-Y ,
      4. Kong APS , et al
      . Development and validation of stroke risk equation for Hong Kong Chinese patients with type 2 diabetes: the Hong Kong diabetes registry. Diabetes Care 2007;30:65–70.doi:10.2337/dc06-1273
      OpenUrlAbstract/FREE Full Text
    39. 39.↵
      2. Hippisley-Cox J ,
      3. Coupland C ,
      4. Vinogradova Y , et al
      . Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 2008;336:1475–82.doi:10.1136/bmj.39609.449676.25
      OpenUrlAbstract/FREE Full Text
    40. 40.↵
      2. Elley CR ,
      3. Robinson E ,
      4. Kenealy T , et al
      . Derivation and validation of a new cardiovascular risk score for people with type 2 diabetes: the New Zealand diabetes cohort study. Diabetes Care 2010;33:1347–52.doi:10.2337/dc09-1444
      OpenUrlAbstract/FREE Full Text
    41. 41.↵
      2. Alrawahi AH ,
      3. Lee P ,
      4. Al-Anqoudi ZAM , et al
      . Cardiovascular risk prediction model for Omanis with type 2 diabetes. Diabetes Metab Syndr 2018;12:105–10.doi:10.1016/j.dsx.2017.09.012
      OpenUrl
    42. 42.↵
      2. Hosmer DW ,
      3. Lemeshow S
      . Applied logistic regression. 2nd Edition. New York, NY: John Wiley & Sons, 2000.
    43. 43.↵
      2. Collins GS ,
      3. Reitsma JB ,
      4. Altman DG , et al
      . Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162:55–63.doi:10.7326/M14-0697
      OpenUrlCrossRefPubMed
    44. 44.↵
      2. Chamnan P ,
      3. Simmons RK ,
      4. Sharp SJ , et al
      . Cardiovascular risk assessment scores for people with diabetes: a systematic review. Diabetologia 2009;52:e14:2001–14.doi:10.1007/s00125-009-1454-0
      OpenUrl
    45. 45.↵
      2. van Dieren S ,
      3. Beulens JWJ ,
      4. Kengne AP , et al
      . Prediction models for the risk of cardiovascular disease in patients with type 2 diabetes: a systematic review. Heart 2012;98:360–9.doi:10.1136/heartjnl-2011-300734
      OpenUrlAbstract/FREE Full Text
    46. 46.↵
      2. Cook NR ,
      3. Ridker PM
      . Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med 2009;150:795–802.doi:10.7326/0003-4819-150-11-200906020-00007
      OpenUrlCrossRefPubMedWeb of Science
    47. 47.
      2. Wells BJ ,
      3. Roth R ,
      4. Nowacki AS , et al
      . Prediction of morbidity and mortality in patients with type 2 diabetes. PeerJ 2013;1:e87.doi:10.7717/peerj.87
    48. 48.
      2. Tanaka S ,
      3. Tanaka S ,
      4. Iimuro S , et al
      . Predicting macro- and microvascular complications in type 2 diabetes: the Japan diabetes complications Study/the Japanese elderly diabetes intervention trial risk engine. Diabetes Care 2013;36:1193–9.doi:10.2337/dc12-0958
      OpenUrlAbstract/FREE Full Text
    49. 49.
      2. Li T-C ,
      3. Wang H-C ,
      4. Li C-I , et al
      . Establishment and validation of a prediction model for ischemic stroke risks in patients with type 2 diabetes. Diabetes Res Clin Pract 2018;138:220–8.doi:10.1016/j.diabres.2018.01.034
      OpenUrl
    50. 50.
      2. Khalili S ,
      3. Hatami M ,
      4. Hadaegh F , et al
      . Prediction of cardiovascular events with consideration of general and central obesity measures in diabetic adults: results of the 8.4-year follow-up. Metab Syndr Relat Disord 2012;10:218–224.doi:10.1089/met.2011.0070
      OpenUrlCrossRefPubMed
    51. 51.
      2. Cederholm J ,
      3. Eeg-Olofsson K ,
      4. Eliasson B , et al
      . Risk prediction of cardiovascular disease in type 2 diabetes: a risk equation from the Swedish national diabetes register. Diabetes Care 2008;31:2038–43.doi:10.2337/dc08-0662
      OpenUrlAbstract/FREE Full Text
    52. 52.
      2. Looker HC ,
      3. Colombo M ,
      4. Agakov F , et al
      . Protein biomarkers for the prediction of cardiovascular disease in type 2 diabetes. Diabetologia 2015;58:1363–71.doi:10.1007/s00125-015-3535-6
      OpenUrlCrossRefPubMed
    53. 53.
      2. Paynter NP ,
      3. Mazer NA ,
      4. Pradhan AD , et al
      . Cardiovascular risk prediction in diabetic men and women using hemoglobin A1c vs diabetes as a high-risk equivalent. Arch Intern Med 2011;171:1712–8.doi:10.1001/archinternmed.2011.351
      OpenUrlCrossRefPubMedWeb of Science
    54. 54.
      2. Price AH ,
      3. Welsh P ,
      4. Weir CJ , et al
      . N-Terminal pro-brain natriuretic peptide and risk of cardiovascular events in older patients with type 2 diabetes: the Edinburgh type 2 diabetes study. Diabetologia 2014;57:2505–12.doi:10.1007/s00125-014-3375-9
      OpenUrl
    55. 55.
      2. Selby JV ,
      3. Karter AJ ,
      4. Ackerson LM , et al
      . Developing a prediction rule from automated clinical databases to identify high-risk patients in a large population with diabetes. Diabetes Care 2001;24:1547–55.doi:10.2337/diacare.24.9.1547
      OpenUrlAbstract/FREE Full Text
    56. 56.
      2. Zarkogianni K ,
      3. Athanasiou M ,
      4. Thanopoulou AC , et al
      . Comparison of machine learning approaches toward assessing the risk of developing cardiovascular disease as a long-term diabetes complication. IEEE J Biomed Health Inform 2018;22:1637–47.doi:10.1109/JBHI.2017.2765639
      OpenUrl
    57. 57.
      2. Price AH ,
      3. Weir CJ ,
      4. Welsh P , et al
      . Comparison of non-traditional biomarkers, and combinations of biomarkers, for vascular risk prediction in people with type 2 diabetes: the Edinburgh type 2 diabetes study. Atherosclerosis 2017;264:67–73.doi:10.1016/j.atherosclerosis.2017.07.009
      OpenUrl
    58. 58.
      2. Wan EYF ,
      3. Fong DYT ,
      4. Fung CSC , et al
      . Development of a cardiovascular diseases risk prediction model and tools for Chinese patients with type 2 diabetes mellitus: a population-based retrospective cohort study. Diabetes Obes Metab 2018;20:309–18.doi:10.1111/dom.13066
      OpenUrl
    59. 59.
      2. Young JB ,
      3. Gauthier-Loiselle M ,
      4. Bailey RA , et al
      . Development of predictive risk models for major adverse cardiovascular events among patients with type 2 diabetes mellitus using health insurance claims data. Cardiovasc Diabetol 2018;17:118.doi:10.1186/s12933-018-0759-z
    60. 60.
      2. van der Leeuw J ,
      3. Beulens JWJ ,
      4. van Dieren S , et al
      . Novel biomarkers to improve the prediction of cardiovascular event risk in type 2 diabetes mellitus. J Am Heart Assoc 2016;5:e003048.doi:10.1161/JAHA.115.003048
    61. 61.
      2. Alshehry ZH ,
      3. Mundra PA ,
      4. Barlow CK , et al
      . Plasma lipidomic profiles improve on traditional risk factors for the prediction of cardiovascular events in type 2 diabetes mellitus. Circulation 2016;134:1637–50.doi:10.1161/CIRCULATIONAHA.116.023233
      OpenUrlAbstract/FREE Full Text
    62. 62.
      2. Woodward M ,
      3. Hirakawa Y ,
      4. Kengne A-P , et al
      . Prediction of 10-year vascular risk in patients with diabetes: the AD-ON risk score. Diabetes Obes Metab 2016;18:289–94.doi:10.1111/dom.12614
      OpenUrl
    63. 63.
      2. Parrinello CM ,
      3. Matsushita K ,
      4. Woodward M , et al
      . Risk prediction of major complications in individuals with diabetes: the Atherosclerosis risk in Communities study. Diabetes Obes Metab 2016;18:899–906.doi:10.1111/dom.12686
      OpenUrlPubMed
    64. 64.
      2. Colombo M ,
      3. Looker HC ,
      4. Farran B , et al
      . Apolipoprotein CIII and N-terminal prohormone B-type natriuretic peptide as independent predictors for cardiovascular disease in type 2 diabetes. Atherosclerosis 2018;274:182–90.doi:10.1016/j.atherosclerosis.2018.05.014
      OpenUrl
    65. 65.
      2. Davis WA ,
      3. Colagiuri S ,
      4. Davis TME
      . Comparison of the Framingham and United Kingdom Prospective Diabetes Study cardiovascular risk equations in Australian patients with type 2 diabetes from the Fremantle Diabetes Study. Med J Aust 2009;190:180–4.
      OpenUrlPubMedWeb of Science
    66. 66.
      2. Jiao FF ,
      3. Fung CSC ,
      4. Wong CKH , et al
      . Effects of the multidisciplinary risk assessment and management program for patients with diabetes mellitus (RAMP-DM) on biomedical outcomes, observed cardiovascular events and cardiovascular risks in primary care: a longitudinal comparative study. Cardiovasc Diabetol 2014;13:127.doi:10.1186/s12933-014-0127-6
    67. 67.
      2. Lahoz-Rallo B ,
      3. Blanco-Gonzalez M ,
      4. Casas-Ciria I , et al
      . Cardiovascular disease risk in subjects with type 2 diabetes mellitus in a population in southern Spain. Diabetes Res Clin Pract 2007;76:436–44.doi:10.1016/j.diabres.2006.09.028
      OpenUrlCrossRefPubMed
    68. 68.
      2. Metcalf PA ,
      3. Wells S ,
      4. Scragg RKR , et al
      . Comparison of three different methods of assessing cardiovascular disease risk in new Zealanders with type 2 diabetes mellitus. N Z Med J 2008;121:49–57.
      OpenUrlPubMed
    69. 69.
      2. Bannister CA ,
      3. Poole CD ,
      4. Jenkins-Jones S , et al
      . External validation of the UKPDS risk engine in incident type 2 diabetes: a need for new type 2 diabetes-specific risk equations. Diabetes Care 2014;37:537–45.doi:10.2337/dc13-1159
      OpenUrlAbstract/FREE Full Text
    70. 70.
      2. Wu Y ,
      3. He J ,
      4. Sun X , et al
      . Carotid atherosclerosis and its relationship to coronary heart disease and stroke risk in patients with type 2 diabetes mellitus. Medicine 2017;96:e8151.doi:10.1097/MD.0000000000008151
    71. 71.
      2. Ipadeola A ,
      3. Adeleye JO
      . The metabolic syndrome and accurate cardiovascular risk prediction in persons with type 2 diabetes mellitus. Diabetes Metab Syndr 2016;10:7–12.doi:10.1016/j.dsx.2015.08.011
      OpenUrl
    72. 72.
      2. Leal J ,
      3. Hayes AJ ,
      4. Gray AM , et al
      . Temporal validation of the UKPDS outcomes model using 10-year posttrial monitoring data. Diabetes Care 2013;36:1541–6.doi:10.2337/dc12-1120
      OpenUrlAbstract/FREE Full Text
    73. 73.
      2. McEwan P ,
      3. Bennett H ,
      4. Ward T , et al
      . Refitting of the UKPDS 68 risk equations to contemporary routine clinical practice data in the UK. Pharmacoeconomics 2015;33:149–61.doi:10.1007/s40273-014-0225-z
      OpenUrl
    74. 74.
      2. Pagano E ,
      3. Gray A ,
      4. Rosato R , et al
      . Prediction of mortality and macrovascular complications in type 2 diabetes: validation of the UKPDS outcomes model in the Casale Monferrato survey, Italy. Diabetologia 2013;56:1726–34.doi:10.1007/s00125-013-2933-x
      OpenUrl
    75. 75.
      2. Tao L ,
      3. Wilson ECF ,
      4. Griffin SJ , et al
      . Performance of the UKPDS outcomes model for prediction of myocardial infarction and stroke in the ADDITION-Europe trial cohort. Value Health 2013;16:1074–80.doi:10.1016/j.jval.2013.06.001
      OpenUrlPubMed
    76. 76.
      2. Shivakumar V ,
      3. Kandhare AD ,
      4. Rajmane AR , et al
      . Estimation of the long-term cardiovascular events using UKPDS risk engine in metabolic syndrome patients. Indian J Pharm Sci 2014;76:174.
      OpenUrl
    77. 77.
      2. Moazzam A ,
      3. Amer J ,
      4. Rehan N
      . Estimating the risk of cardio vascular diseases among Pakistani diabetics using UKPDS risk engine. Journal of Ayub Medical College Abbottabad 2015;27:277–9.
      OpenUrl
    78. 78.
      2. Ezenwaka CE ,
      3. Nwagbara E ,
      4. Seales D , et al
      . Prediction of 10-year coronary heart disease risk in Caribbean type 2 diabetic patients using the UKPDS risk engine. Int J Cardiol 2009;132:348–53.doi:10.1016/j.ijcard.2007.12.005
      OpenUrlPubMed
    79. 79.
      2. Sun X ,
      3. He J ,
      4. Ji X-L , et al
      . Association of chronic kidney disease with coronary heart disease and stroke risks in patients with type 2 diabetes mellitus: an observational cross-sectional study in Hangzhou, China. Chin Med J 2017;130:57–63.doi:10.4103/0366-6999.196564
      OpenUrl
    80. 80.
      2. Pang X-H ,
      3. Han J ,
      4. Ye W-L , et al
      . Lower extremity peripheral arterial disease is an independent predictor of coronary heart disease and stroke risks in patients with type 2 diabetes mellitus in China. Int J Endocrinol 2017;2017:9620513–6.doi:10.1155/2017/9620513
      OpenUrl
    81. 81.
      2. Herder C ,
      3. Schöttker B ,
      4. Rothenbacher D , et al
      . Interleukin-6 in the prediction of primary cardiovascular events in diabetes patients: results from the ESTHER study. Atherosclerosis 2011;216:244–7.doi:10.1016/j.atherosclerosis.2011.01.041
      OpenUrlCrossRefPubMedWeb of Science
    82. 82.
      2. Costa B ,
      3. Cabré JJ ,
      4. Martín F , et al
      . [The Framingham function overestimates stroke risk for diabetes and metabolic syndrome among Spanish population]. Aten Primaria 2005;35:392–8.
      OpenUrlCrossRefPubMed
    83. 83.
      2. Read SH ,
      3. van Diepen M ,
      4. Colhoun HM , et al
      . Performance of cardiovascular disease risk scores in people diagnosed with type 2 diabetes: external validation using data from the National Scottish diabetes register. Diabetes Care 2018;41:2010–8.doi:10.2337/dc18-0578
      OpenUrlAbstract/FREE Full Text
    84. 84.
      2. Yao M-F ,
      3. He J ,
      4. Sun X , et al
      . Gender differences in risks of coronary heart disease and stroke in patients with type 2 diabetes mellitus and their association with metabolic syndrome in China. Int J Endocrinol 2016;2016:8483405–7.doi:10.1155/2016/8483405
      OpenUrl
    85. 85.
      2. Basu S ,
      3. Sussman JB ,
      4. Berkowitz SA , et al
      . Validation of risk equations for complications of type 2 diabetes (RECODe) using individual participant data from diverse longitudinal cohorts in the U.S. Diabetes Care 2018;41:586–95.doi:10.2337/dc17-2002
      OpenUrlAbstract/FREE Full Text
    86. 86.
      2. Alrawahi AH ,
      3. Lee P
      . Validation of the cardiovascular risk model developed for Omanis with type 2 diabetes. Diabetes Metab Syndr 2018;12:387–91.doi:10.1016/j.dsx.2018.01.004
      OpenUrl
    View Abstract

    Footnotes

    • Contributors All authors contributed to this work. MZIC and TCT contributed to the conception and design of the review. MZIC and FY read and screened abstracts and titles of potentially relevant studies. MZIC and FY read the retained papers and were responsible for extracting data and rating their quality independently. MZIC performed the data analysis. MZIC drafted the paper and PER, DMR and TCT critically reviewed it and suggested amendments prior to submission. All authors approved the final version of the manuscript and take responsibility for the integrity of the reported findings.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement No data are available.