Performance of prediction models for nephropathy in people with type 2 diabetes: systematic review and external validation study
BMJ 2021; 374 doi: https://doi.org/10.1136/bmj.n2134 (Published 28 September 2021) Cite this as: BMJ 2021;374:n2134
- Roderick C Slieker
, postdoctoral fellow12, - Amber A W A van der Heijden, assistant professor3,
- Moneeza K Siddiqui, principal investigator (tenure track)4,
- Marlous Langendoen-Gort, doctoral student3,
- Giel Nijpels, professor emeritus3,
- Ron Herings, professor15,
- Talitha L Feenstra, professor67,
- Karel G M Moons, professor89,
- Samira Bell, consultant nephrologist4,
- Petra J Elders, professor3,
- Leen M ’t Hart, associate professor1210,
- Joline W J Beulens, professor18
- 1Department of Epidemiology and Data Science, Amsterdam Public Health Institute, Amsterdam Cardiovascular Sciences Institute, Amsterdam UMC, Location VUmc, 1081 HV, Amsterdam, Netherlands
- 2Department of Cell and Chemical Biology, Leiden University Medical Center, Leiden, Netherlands
- 3Department of General Practice, Amsterdam Public Health Institute, Amsterdam UMC, Location VUmc, Amsterdam, Netherlands
- 4Division of Population Health and Genomics, School of Medicine, University of Dundee, Dundee, UK
- 5PHARMO Institute for Drug Outcomes Research, Utrecht, Netherlands
- 6Groningen Research Institute of Pharmacy, University of Groningen, Groningen, Netherlands
- 7Centre for Nutrition, Prevention and Health Services, Institute for Public Health and the Environment, Bilthoven, Netherlands
- 8Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- 9Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- 10Molecular Epidemiology section, Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands
- Correspondence to: R C Slieker r.slieker{at}amsterdamumc.nl (or @rcslieker on Twitter)
- Accepted 25 August 2021
Abstract
Objectives To identify and assess the quality and accuracy of prognostic models for nephropathy and to validate these models in external cohorts of people with type 2 diabetes.
Design Systematic review and external validation.
Data sources PubMed and Embase.
Eligibility criteria Studies describing the development of a model to predict the risk of nephropathy, applicable to people with type 2 diabetes.
Methods Screening, data extraction, and risk of bias assessment were done in duplicate. Eligible models were externally validated in the Hoorn Diabetes Care System (DCS) cohort (n=11 450) for the same outcomes for which they were developed. Risks of nephropathy were calculated and compared with observed risk over 2, 5, and 10 years of follow-up. Model performance was assessed based on intercept adjusted calibration and discrimination (Harrell’s C statistic).
Results 41 studies included in the systematic review reported 64 models, 46 of which were developed in a population with diabetes and 18 in the general population including diabetes as a predictor. The predicted outcomes included albuminuria, diabetic kidney disease, chronic kidney disease (general population), and end stage renal disease. The reported apparent discrimination of the 46 models varied considerably across the different predicted outcomes, from 0.60 (95% confidence interval 0.56 to 0.64) to 0.99 (not available) for the models developed in a diabetes population and from 0.59 (not available) to 0.96 (0.95 to 0.97) for the models developed in the general population. Calibration was reported in 31 of the 41 studies, and the models were generally well calibrated. 21 of the 64 retrieved models were externally validated in the Hoorn DCS cohort for predicting risk of albuminuria, diabetic kidney disease, and chronic kidney disease, with considerable variation in performance across prediction horizons and models. For all three outcomes, however, at least two models had C statistics >0.8, indicating excellent discrimination. In a secondary external validation in GoDARTS (Genetics of Diabetes Audit and Research in Tayside Scotland), models developed for diabetic kidney disease outperformed those for chronic kidney disease. Models were generally well calibrated across all three prediction horizons.
Conclusions This study identified multiple prediction models to predict albuminuria, diabetic kidney disease, chronic kidney disease, and end stage renal disease. In the external validation, discrimination and calibration for albuminuria, diabetic kidney disease, and chronic kidney disease varied considerably across prediction horizons and models. For each outcome, however, specific models showed good discrimination and calibration across the three prediction horizons, with clinically accessible predictors, making them applicable in a clinical setting.
Systematic review registration PROSPERO CRD42020192831.
Footnotes
Contributors: RCS, AAH, and JWB designed the study. RCS, AAH, JWB, and MLG screened citations for inclusion and were involved in risk of bias assessment and data extraction and interpretation. RCS collected, cleaned, and analysed the data. All co-authors were involved in the interpretation of the data. MKS and SB performed the external validation in GoDARTS. RCS wrote the draft manuscript with input from all co-authors. All authors approved the final version of this manuscript. RCS is the guarantor of this manuscript and accepts full responsibility for the work and the conduct of the study, had access to the data, and controlled the decision to publish. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding: This research was supported by the Dutch Diabetes Research Foundation (grant No 2014.00.1753). The funder had no role in the study design; collection, analysis, and interpretation of data; or preparation of the manuscript. GoDARTS is funded and supported by the Wellcome Trust Type 2 Diabetes Case Control Collection (072960/Z/03/Z, 084726/Z/08/Z, 084727/Z/08/Z, 085475/Z/ 08/Z, 085475/B/08/Z) and as part of the EU IMI-SUMMIT programme.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from Dutch Diabetes Research foundation Fund, IMI-RHAPSODY, and ZorgInstitute Netherlands (Dutch Healthcare Institute) for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
The lead authors (RCS, AAH, JWB) affirm that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Dissemination to participants and related patient and public communities: To disseminate our results we aim to target a broad audience, including health professionals, scientists, and members of the public through written communications, social media, and the cohort’s website.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement
The steering committee of the Hoorn studies will consider reasonable requests for the sharing of deidentified patient level data. Requests should be made to the corresponding author.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.