Risk models and scores for type 2 diabetes: systematic review

Riskmodels and scores for type 2 diabetes: systematicreview

OPEN ACCESS

Douglas Noble lecturer 1, Rohini Mathur research fellow 1, Tom Dent consultant 2, Catherine Meadssenior lecturer 1, Trisha Greenhalgh professor 1

1Centre for Primary Care and Public Health, Barts and the London School of Medicine and Dentistry, London E1 2AT, UK; 2PHG Foundation,Cambridge, UK

AbstractObjective To evaluate current risk models and scores for type 2 diabetesand inform selection and implementation of these in practice.

Design Systematic review using standard (quantitative) and realist(mainly qualitative) methodology.

Inclusion criteria Papers in any language describing the developmentor external validation, or both, of models and scores to predict the riskof an adult developing type 2 diabetes.

Data sourcesMedline, PreMedline, Embase, and Cochrane databaseswere searched. Included studies were citation tracked in Google Scholarto identify follow-on studies of usability or impact.

Data extraction Data were extracted on statistical properties of models,details of internal or external validation, and use of risk scores beyondthe studies that developed them. Quantitative data were tabulated tocompare model components and statistical properties. Qualitative datawere analysed thematically to identify mechanisms by which use of therisk model or score might improve patient outcomes.

Results 8864 titles were scanned, 115 full text papers considered, and43 papers included in the final sample. These described the prospectivedevelopment or validation, or both, of 145 risk prediction models andscores, 94 of which were studied in detail here. They had been testedon 6.88 million participants followed for up to 28 years. Heterogeneityof primary studies precluded meta-analysis. Some but not all risk modelsor scores had robust statistical properties (for example, gooddiscrimination and calibration) and had been externally validated on adifferent population. Genetic markers added nothing to models overclinical and sociodemographic factors. Most authors described theirscore as “simple” or “easily implemented,” although few were specificabout the intended users and under what circumstances. Tenmechanisms were identified by which measuring diabetes risk mightimprove outcomes. Follow-on studies that applied a risk score as partof an intervention aimed at reducing actual risk in people were sparse.

ConclusionMuch work has been done to develop diabetes risk modelsand scores, but most are rarely used because they require tests notroutinely available or they were developed without a specific user orclear use in mind. Encouragingly, recent research has begun to tackleusability and the impact of diabetes risk scores. Two promising areasfor further research are interventions that prompt lay people to checktheir own diabetes risk and use of risk scores on population datasets toidentify high risk “hotspots” for targeted public health interventions.

IntroductionThe prevalence of diabetes is rising rapidly throughout theworld.1 By 2010 its prevalence in the adult populations of theUnited Kingdom, the United States, mainland China, and theUnited Arab Emirates had exceeded 7%,2 11%,3 15%,4 and 17%,5respectively. Americans born in 2000 or later have a lifetimerisk of more than one in three of developing diabetes.6 Type 2diabetes (which accounts for over 95% of diabetes worldwide)results from a complex gene-environment interaction for whichseveral risk factors, such as age, sex, ethnicity, family history,obesity, and hypertension, are well documented. The preciseinteraction of these and other risk factors with one another is,however, a complex process that varies both within and acrosspopulations.7-11 Epidemiologists and statisticians are striving toproduce weighted models that can be presented as scores toreflect this complexity but which at the same time are perceivedas sufficiently simple, plausible, affordable, and widelyimplementable in clinical practice.12 13

Cohort studies have shown that early detection of establisheddiabetes improves outcome, although the evidence base forscreening the entire population is weak.14 15 The proportion ofcases of incident type 2 diabetes in people with impaired glucosetolerance or impaired fasting glucose levels was reduced inlandmark trials from China,16 Finland,17 and the United States18by up to 33%, 50%, and 58%, respectively, through lifestyle

Correspondence to: D Noble [email protected]

Extra material supplied by the author (see http://www.bmj.com/content/343/bmj.d7163?tab=related#webextra)

Details of search strategy

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2011;343:d7163 doi: 10.1136/bmj.d7163 (Published 28 November 2011) Page 1 of 31

Research

RESEARCH

http://www.bmj.com/content/343/bmj.d7163?tab=related#webextra

http://www.bmj.com/permissions

http://www.bmj.com/subscribe

changes (increased exercise, weight loss) or pharmacotherapy,or both, although changes may be more modest in a non-trialpopulation. Some have argued that because combining knownrisk factors predicts incident diabetes at least as effectively asimpaired glucose metabolism, a diabetes risk score may be abetter and more practical means of identifying people forpreventive interventions than either a glucose tolerance test ora fasting blood glucose level.19 Others favour targeting theassessment of diabetes risk in those with established impairedglucose metabolism on the basis that interventions in this groupare particularly effective.20

Riskmodels and scores first emerged for cardiovascular disease,and these are widely used in clinical and public health practice.In the United Kingdom, for example, all electronic patient recordsystems in general practice offer the facility to calculate theFramingham score, a patient’s risk of a cardiovascular eventwithin 10 years. This risk score features in many guidelines anddecision pathways (such as the cut-off for statin therapy21), andgeneral practitioners receive financial rewards for calculatingit.22 In contrast, although numerous models and scores havebeen developed for diabetes risk, we found limited evidence foruse of these as part of a formal health policy, guideline, orincentive scheme for practitioners in any country (one Australianscheme incentivises general practitioners’ measurement of therisk of diabetes in adults aged 40-4923). This is perhapssurprising, given that morbidity and mortality fromcardiovascular disease has been decreasing in many countriessince the 1970s,24 whereas those from diabetes continue toincrease.3

A diabetes risk score is an example of a prognostic model.25Such scores should ideally be developed by taking a large, agedefined population cohort of people without diabetes, measuringbaseline risk factors, and following the cohort for a sufficientlylong time to see who develops diabetes.26Although prospectivelongitudinal designs in specially assembled cohorts areexpensive, difficult, and time consuming to execute, crosssectional designs in which risk factors are measured in apopulation including people both with and without diabetes aremethodologically inferior. They use prevalence as a proxy forincidence and conflate characteristics of people with diabeteswith risk factors in those without diabetes, and thus are incapableof showing that a putative risk factor predated the developmentof diabetes. In practice, researchers tend to take one of twoapproaches: they either study a cohort of people withoutdiabetes, which was assembled some years previously withrelevant baseline metrics for some other purpose (for example,the British Regional Heart Study27), or they analyse routinelyavailable data, such as electronic patient records.8 Bothapproaches are potentially susceptible to bias.Some diabetes risk scores are intended to be self administeredusing questions such as “have you ever been told you have highblood pressure?” Scores that rely entirely on such questionsmay be hosted on the internet (see for example www.diabetes.org.uk/riskscore). Some researchers have used self completionpostal questionnaires as the first part of a stepwise detectionprogramme.28 To the extent that these instruments are valid,they can identify two types of people: those who already havediabetes whether or not they know it (hence the questionnairemay serve as a self administered screening tool for undiagnoseddiabetes) and those at high risk of developing diabetes—that is,it may also serve as a prediction tool for future diabetes.Prevalence rates for diabetes derived from self assessmentstudies thus cannot be compared directly with the rate of incidentdiabetes in a prospective longitudinal sample from which thosetesting positive for diabetes at baseline have been excluded.

A good risk score is usually defined as one that accuratelyestimates individuals’ risk—that is, predictions based on thescore closely match what is observed (calibration); the scoredistinguishes reliably between high risk people, who are likelyto go on to develop the condition, and low risk people, who areless likely to develop the condition (discrimination); and itperforms well in new populations (generalisability). Validatinga risk model or score means testing its calibration anddiscrimination either internally (by splitting the original sample,developing the score on one part and testing it on another),temporally (re-running the score on the same or a similar sampleafter a time period), or, preferably, externally (running the scoreon a new population with similar but not identical characteristicsfrom the one on which it was developed).26 29 Caution is neededwhen extrapolating a risk model or score developed in onepopulation or setting to a different one—for example, secondaryto primary care, adults to children, or one ethnic group toanother.30

Risk scores and other prognostic models should be subject to“impact studies”—that is, studies of the extent to which thescore is actually used and leads to improved outcomes. Althoughmost authors emphasise quantitative evaluation of impact suchas through cluster randomised controlled trials,30 much mightalso be learnt from qualitative studies of the process of usingthe score, either alone or as an adjunct to experimental trials.One such methodology is realist evaluation, which considersthe interplay between context, mechanism (how the interventionis perceived and taken up by practitioners), and outcome.31 Inpractice, however, neither quantitative nor qualitative studiesof impact are common in the assessment of risk scores.30

We sought to identify, classify, and evaluate risk models andscores for diabetes and inform their selection andimplementation in practice. We wanted to determine the keystatistical properties of published scores for predicting type 2diabetes in adults and how they perform in practice. Hence wewere particularly interested in highlighting those characteristicsof a risk score that would make it fit for purpose in differentsituations and settings. To that end we reviewed the literatureon development, validation, and use of such scores, using bothquantitative data on demographics of populations and statisticalproperties of models and qualitative data on how risk scoreswere perceived and used by practitioners, policy makers, andothers in a range of contexts and systems.

MethodsTheoretical and methodological approachWe followed standard methodology for systematic reviews,summarised in guidance from a previous study and the YorkCentre for Reviews and Dissemination.32 33 The process waslater extended by drawing on the principles of realist review,an established form of systematic literature review that usesmainly qualitative methods to produce insights into theinteraction between context, mechanism, and outcome, henceexplaining instances of both success and failure.34 Our team isleading an international collaborative study, the Realist andMeta-narrative Evidence Synthesis: Evolving Standards(RAMESES) to develop methodological guidance andpublication standards for realist review.35

Search strategyWe identified all peer reviewed cohort studies in adults overage 18 that had derived or validated, or both, a statisticallyweighted risk model for type 2 diabetes in a population notpreselected for known risk factors or disease, and which could



RESEARCH

http://www.diabetes.org.uk/riskscore

http://www.diabetes.org.uk/riskscore



be applied to another population. Studies were included thathad developed a new risk model based on risk factors and thatused regression techniques to weight risk factors appropriately,or validated an existing model on a new population, or did both.Exclusion criteria were cross sectional designs, studies that hadnot finished recruiting, studies on populations preselected forrisk factors or disease, studies that did not link multiple riskfactors to form a scoring system or weighted model, screeningor early detection studies, genetic studies, case series, studieson under 18s, animal studies, and studies that applied a knownrisk model or score to a population but did not evaluate itsstatistical potential.In January 2011we undertook a scoping search, beginning withsources known to the research team and those recommendedby colleagues.We used the 29 papers from this search to developthe definitive protocol, including search terms and inclusionand exclusion criteria. In February 2011 a specialist librariandesigned a search strategy (see web extra) with assistance fromthe research team. Key words were predict, screen, risk, score,[type two] diabetes, model, regression, risk assessment, riskfactor, calculator, analysis, sensitivity and specificity, ROC andodds ratio. Both MESH terms and text words were used. Titlesand abstracts were searched in Medline, PreMedline, Embase,and relevant databases in the Cochrane Library from inceptionto February 2011, with no language restrictions.Search results from the different databases were combined inan endnote file and duplicates removed electronically andmanually. In February and March 2011 two researchersindependently scanned titles and abstracts and flaggedpotentially relevant papers for full text analysis.Two researchers independently read the interim dataset of fulltext papers and reduced this to a final dataset of studies,resolving disagreements by discussion. Bilingual academiccolleagues translated non-English papers and extracted data incollaboration with one of the research team. To identify recentlypublished papers two researchers independently citation trackedthe final dataset of studies in Google Scholar. Reference listsof the final dataset and other key references were also scanned.

Quantitative data extraction and analysisProperties of included studies were tabulated on an Excelspreadsheet. A second researcher independently double checkedthe extraction of primary data from every study. Discrepancieswere resolved by discussion. Where studies trialled multiplemodels with minimal difference in the number of risk factors,a judgment was made to extract data from the authors’ preferredmodels or (if no preferences were stated in the paper) the onesjudged by two researchers to be the most complete inpresentation of data or statistical robustness. Data extractioncovered characteristics of the population (age, sex, ethnicity,etc), size and duration of study, completeness of follow-up,method of diagnosing diabetes, details of internal or externalvalidation, or both, and the components and metrics used byauthors of these studies to express the properties of the score,especially their calibration and discrimination—for example,observed to predicted ratios, sensitivity and specificity, areaunder the receiver operating characteristic curve. We aimed touse statistical meta-analysis where appropriate and presentedheterogeneous data in disaggregated form.

Qualitative data extraction and analysisFor the realist component of the review we extracted data andentered these on a spreadsheet under seven headings (box 1).

One researcher extracted these data from our final sample ofpapers and another checked a one third sample of these. Ourresearch team discussed context-mechanism-outcomeinteractions hypothesised or implied by authors and reread thefull sample of papers with all emerging mechanisms in mind toexplore these further.

Impact analysisWe assessed the impact of each risk score in our final sampleusing three criteria: any description in the paper of use of thescore beyond the population for whom it was developed andvalidated; number of citations of the paper in Google Scholarand number of these that described use of the score in an impactstudy; and critical appraisal of any impact studies identified onthis citation track. In this phase we were guided by the question:what is the evidence that this risk score has been used in anintervention which improved (or sought to improve) outcomesfor individuals at high risk of diabetes?

Prioritising papers for reportingGiven the large number of papers, statistical models, and riskscores in our final sample, we decided for clarity to highlight asmall number of scores that might be useful to practisingclinicians, public health specialists, or lay people. Adaptingprevious quality criteria for risk scores,26we favoured those thathad external validation by a separate research team on a differentpopulation (generalisability), statistically significant calibration,a discrimination greater than 0.70, and 10 or fewer components(usability).

ResultsFigure 1⇓ shows the flow of studies through the review. Onehundred and fifteen papers were analysed in detail to producea final sample of 43. Of these 43 papers, 18 described thedevelopment of one or more risk models or scores,8 27 36-51 17described external validation of one or more models or scoreson new populations,9 10 19 52-65 and eight did both.7 66-72 In all, the43 papers described 145 risk models and scores, of which 94were selected for extraction of full data (the other 51 wereminimally different, were not the authors’ preferred model, orlacked detail or statistical robustness). Of the final sample of94 risk models, 55 were derivations of risk models on a basepopulation and 39 were external validations (of 14 differentmodels) on new populations. Studies were published between1993 and 2011, but most appeared in 2008-11 (fig 2⇓). Indeed,even given that weaker cross sectional designs had beenexcluded, the findings suggest that new risk models and scoresfor diabetes are currently being published at a rate of about oneevery three weeks.Table 1⇓ gives full details of the studies in the sample, includingthe origin of the study, setting, population, methodologicalapproach, duration, and how diabetes was diagnosed. The studieswere highly heterogeneous. Models were developed andvalidated in 17 countries representing six continents (30 inEurope, 25 in North America, 21 in Asia, 8 in Australasia, 8 inthe Middle East, 1 in South America, and 1 in Africa).Comparisons across studies were problematic owing toheterogeneity of data and highly variable methodology,presentation techniques, and missing data. Cohorts ranged insize from 399 to 2.54 million. The same data and participantswere often included in several different models in the samepaper. Ten research populations were used more than once indifferent papers.9 10 27 37 41 42 44 46-49 51-56 63-66 70 71 In total, risk models



RESEARCH



Box 1: Categories for data entry

Intended usersAuthors’ assumptions (if any) about who would use the risk score, on which subgroups or populations

Proposed action based on the score resultAuthors’ assumptions (if any) on what would be offered to people who score above the designated cut-off for high risk

MechanismAuthors’ hypothesised (or implied) mechanism by which use of the score might improve outcomes for patients

DescriptorAuthors’ adjectives to describe their risk model or score

Relative advantageAuthors’ claims for how and in what circumstances their model or score outperforms previous ones

ConcernsAuthors’ stated concerns about their model or score

Real world use, including citation trackingActual data in this paper or papers citing it on use of the score in the real world

were tested on 6.88 million participants, although this figureincludes duplicate tests on the same dataset. Participants aged18 to 98 were studied for periods ranging from 3.15 to 28 years.Completeness of follow-up ranged from 54% to 99% andincidence of diabetes across the time periods studied rangedfrom 1.3% to 20.9%.None of the models in the sample was developed on a cohortrecruited prospectively for the express purpose of devising it.Rather, all authors used the more pragmatic approach ofretrospectively studying a research dataset that had beenassembled some years previously for a different purpose. Fortytwo studies excluded known diabetes in the inception cohort.Diagnosis of diabetes in a cohort at inception and completionof the study was done in different ways, including self report,patient questionnaires, clinician diagnosis, electronic code,codes from the International Classification of Diseases, diseaseor drug registers, diabetes drugs, dietary treatment, fastingplasma glucose levels, oral glucose tolerance test, andmeasurement of haemoglobin A1c. In some studies the methodwas not stated. Half the studies used different diagnostic testsat inception and completion of the study.One third of the papers focused almost exclusively on thestatistical properties of the models. Many of the remainder hada clinician (diabetologist or general practitioner) as coauthorand included an (often short and speculative) discussion on howthe findings might be applied in clinical practice. Threedescribed their score as a “clinical prediction rule.”45 51 59

Quantitative findingsTable 2⇓ gives details of the components of the 94 risk modelsincluded in the final sample and their statisticalproperties—including (where reported) their discrimination,calibration, sensitivity, specificity, positive and negativepredictive value, and area under the receiver operatingcharacteristic curve. Many papers offered additionalsophisticated statistical analysis, although there was noconsistency in the approach used or statistical tests.Heterogeneity of data (especially demographic and ethnicdiversity of validation cohorts and different score components)in the primary studies precluded formal meta-analysis.All 94 models presented a combination of risk factors assignificant in the final model, and different models weighteddifferent components differently. The number of components

in a single risk score varied from 3 to 14 (n=84, mean 7.8, SD2.6). The seven risk scores that were classified as having highpotential for use in practice offered broadly similar componentsand had similar discriminatory properties (area under receiveroperating characteristic curve 0.74-0.85, table 4). Overall, theareas under the receiver operating characteristic curve rangedfrom 0.60 to 0.91. Certain components used in some models(for example, biomarkers) are rarely available in some pathologylaboratories and potentially too expensive for routine use. Somemodels that exhibited good calibration and discrimination onthe internal validation cohort performed much less well whentested on an external cohort,62 67 suggesting that the initial modelmay have been over-fitted by inclusion of too many variablesthat had only minor contributions to the total risk.73 Althoughthis study did not seek out genetic components, those studiesthat had included genetic markers alongside sociodemographicand clinical data all found that the genetic markers added littleor nothing to the overall model.9 10 36 50

Reporting of statistical data in some studies was incomplete—forexample, only 40 of the 94 models quantified any form ofcalibration statistic. Forty three presented sensitivity andspecificity, 27 justified the rationale for cut-off points, 22presented a positive predictive value, 19 presented a negativepredictive value, and 26 made some attempt to indicate thepercentage of the population that would need clinical follow-upor testing if they scored as “high risk.” Somemodels performedpoorly—for example, there was a substantial gap betweenexpected and observed numbers of participants who developeddiabetes over the follow-up period. The false positive and falsenegative rates in many risk scores raised questions about theirutility in clinical practice (for example, positive predictive valuesranged from 5% to 42%, negative predictive values from 88%to 99%). However, some scores were designed as non-invasivepreliminary instruments, with a recommended second phaseinvolving a blood test.7 43 52 53 55 58 65

Risk models and scores tended to “morph” when they wereexternally validated because research teams dropped componentsfrom the original (for example, if data on these were notavailable), added additional components (for example, tocompensate for missing categories), or modified what countedin a particular category (for example, changing how ethnicitywas classified); in some cases these modifications were notclarified. A key dimension of implementation is appropriate



RESEARCH



adaptation to a new context. It was considered that this did notnegate the external validation.

Qualitative findingsTable 3⇓ provides the qualitative findings from the risk scores.Of the 43 papers in the full sample, three did not recommenduse of the model tested because the authors believed it had noadvantage over existing ones.50 56 60 Authors of the other 40papers considered that at least one of their scores should beadopted and used, and to justify this made various claims. Thecommonest adjective used by authors to describe their scorewas “simple” (26 of 43); others mentioned “low cost,” “easilyimplemented,” “feasible,” and “convenient.”Sixteen of the 43 studies that recommended use of a particularrisk model or score did not designate an intended user for it.Some authors assigned agency to a risk score—that is, theystated, perhaps inadvertently, that the score itself had thepotential to prevent diabetes, change behaviour, or reduce healthinequalities. Although most authors did state an intended targetgroup, this was usually given in vague terms, such as “thegeneral population” or “individuals who are likely to developdiabetes.” Eleven of the 43 papers gave a clear statement ofwhat intervention might be offered, by whom, to people whoscored above the cut-off for high risk; the other papers madeno comment on this or used vague terms such as “preventivemeasures,” without specifying by whom these would bedelivered.In all, authors of the papers in the full sample either explicitlyidentified or appeared to presume 10 mechanisms (box 2) bywhich, singly or in combination, use of the diabetes risk scoremight lead to improved patient outcomes (see table 3).Risk models and scores had been developed in a range of healthsystems. Differences in components could be explained partlyin terms of their intended context of use. For example, theQDScore, intended for use by general practitioners, wasdeveloped using a database of electronic records of a nationallyrepresentative sample of the UK general practice populationcomprising 2.5 million people. The QDScore is composedentirely of data items that are routinely recorded on generalpractice electronic records (including self assigned ethnicity, adeprivation score derived from the patient’s postcode, andclinical and laboratory values).8 Another score, also intendedto be derived from electronic records but in a US healthmaintenance organisation (covering people of working age whoare in work), has similar components to the QDScore exceptthat ethnicity and socioeconomic deprivation are not included.In contrast, the FINDRISC score was developed as a populationscreening tool intended for use directly by lay people; it consistsof questions on sociodemographic factors and personal historyalong with waist circumference but does not include clinical orlaboratory data; high scorers are prompted to seek further advicefrom a clinician.52 Such a score makes sense in many parts ofFinland and also in the Netherlands where health andinformation literacy rates are high, but would be less fit forpurpose in a setting where these were low.

Prioritising scores for practising cliniciansTable 4⇓ summarises the properties of seven validated diabetesrisk scores which we judged to be the most promising for usein clinical or public health practice. The judgments on whichthis selection was based were pragmatic; other scores not listedin table 4 (also see tables 1 and 2) will prove more fit for purposein certain situations and settings. One score that has not yet beenexternally validated was included in table 4 because it is the

only score already being incentivised in a national diabetesprevention policy.23

Studies of impact of risk scores on patientoutcomesNone of the 43 papers that validated one or more risk scoresdescribed the actual use of that score in an intervention phase.Furthermore, although these papers had been cited by a total of1883 (range 0-343, median 12) subsequent papers, only nine ofthose 1883 papers (table 5⇓) described application and use ofthe risk score as part of an impact study aimed at changingpatient outcomes. These covered seven studies, of which (todate) three have reported definitive results. All three reportedpositive changes in individual risk factors, but surprisingly nonerecalculated participants’ risk scores after the intervention periodto see if they had changed. While one report on the ongoingFIN-D2D study suggests that incident diabetes has been reducedin “real world” (non-trial) participants whowere picked up usinga diabetes risk score and offered a package of preventive care,74this is a preliminary and indirect finding based on drugreimbursement claims, and no actual data are given in the paper.With that exception, no published impact study on a diabetesrisk score has yet shown a reduction in incident diabetes.

DiscussionNumerous diabetes risk scores now exist based on readilyavailable data and provide a good but not perfect estimate ofthe chance of an adult developing diabetes in the medium termfuture. A few research teams have undertaken exemplarydevelopment and validation of a robust model, reported itsstatistical properties thoroughly, and followed through withstudies of impact in the real world.

Limitations of included studiesWe excluded less robust designs (especially cross sectionalstudies). Nevertheless, included studies were not entirely freefrom bias and confounding. This is because the “pragmatic” useof a previously assembled database or cohort brings an inherentselection bias (for example, the British Regional Heart Studycohort was selected to meet the inclusion criteria for age andcomorbidity defined by its original research team and orientedto research questions around cardiovascular disease; thepopulation for the QDScore is drawn from general practicerecords and hence excludes those not registered with a generalpractitioner).Most papers in our sample had one or more additionallimitations. They reported models or scores that requiredcollection of data not routinely available in the relevant healthsystem; omitted key statistical properties such as calibrationand positive and negative predictive values that would allow aclinician or public health commissioner to judge the practicalvalue of the score; or omitted to consider who would use thescore, on whom, and in what circumstances. We identified amismatch between the common assumption of authors whodevelop a risk model (that their “simple” model can now betaken up and used) and the actual uptake and use of such models(which seems to happen very rarely). However, there hasrecently been an encouraging—if limited—shift in emphasisfrom the exclusive pursuit of statistical elegance (for example,maximising area under the receiver operating curve) toundertaking applied research on the practicalities and outcomesof using diabetes risk scores in real world preventionprogrammes.



RESEARCH



Box 2: 10 suggested mechanisms by which diabetes risk scores could help improve patient outcomes

ClinicalDirect impact—clinicians will pick up high risk patients during consultations and offer advice that leads to change in patients’ behaviourand lifestyleIndirect impact—routine use of the score increases clinicians’ awareness of risk for diabetes and motivation to manage it

Self assessmentDirect impact—people are alerted by assessing their own risk (for example, using an online tool), directly leading to change in lifestyleIndirect impact—people, having assessed their own risk, are prompted to consult a clinician to seek further tests or advice on prevention

TechnologicalIndividual impact—a risk model programmed into the electronic patient record generates a point of care prompt in the clinical encounterPopulation impact—a risk model programmed into the electronic patient record generates aggregated data on risk groups, which willinform a public health intervention

Public healthPlanners and commissioners use patterns of risk to direct resources into preventive healthcare for certain subgroups

AdministrativeAn administrator or healthcare assistant collects data on risk and enters these onto the patients’ records, which subsequently triggersthe technological, clinical, or public health mechanisms

Research into practiceUse of the risk score leads to improved understanding of risk for diabetes or its management by academics, leading indirectly to changesin clinical practice and hence to benefits for patients

Future researchUse of the risk score identifies focused subpopulations for further research (with the possibility of benefit to patients in later years)

Strengths and limitations of the reviewThe strengths of this review are our use of mixed methodology,orientation to patient relevant outcomes, extraction and doublechecking of data by five researchers, and inclusion of a citationtrack to identify recently published studies and studies of impact.We applied both standard systematic review methods (toundertake a systematic and comprehensive search, translate allnon-English texts, and extract and analyse quantitative data)and realist methods (to consider the relation between thecomponents of the risk score, the context in which it wasintended to be used, and the mechanism by which it mightimprove outcomes for patients).The main limitation of this review is that data techniques andpresentation in the primary studies varied so much that it wasproblematic to determine reasonable numerators anddenominators for many of the calculations. This required us tomake pragmatic decisions to collate and present data as fairlyand robustly as possible while also seeking to make sense ofthe vast array of available risk scores to the general medicalreader. We recognise that the final judgment on which riskscores are, in reality, easy to use will lie with the end user inany particular setting. Secondly, authors of some of the primarystudies included in this review were developing a local tool forlocal use and made few or no claims that their score should begeneralised elsewhere. Yet, the pioneers of early well knownrisk scores49 68 have occasionally found their score being appliedto other populations (perhaps ethnically and demographicallydifferent from the original validation cohort), their selection ofrisk factors being altered to fit the available categories in otherdatasets, and their models being recalibrated to provide bettergoodness of fit. All this revision and recalibration to produce“new” scores makes the systematic review of such scores atbest an inexact science.

Why did we not recommend a “best” riskscore?We have deliberately not selected a single, preferred diabetesrisk score. There is no universal ideal risk score, as the utilityof any score depends not merely on its statistical properties butalso on its context of use, which will also determine which typesof data are available to be included.75 76 Even when a risk modelhas excellent discrimination (and especially when it does not)the trade-off between sensitivity and specificity plays outdifferently depending on context. Box 3 provides some questionsto ask when selecting a diabetes risk score.

Risk scores as complex interventionsOur finding that diabetes risk scores seem to be used rarely canbe considered in the light of the theoretical literature on diffusionof innovation. As well as being a statistical model, a risk scorecan be thought of as a complex, technology based innovation,the incorporation of which into business as usual (or not) isinfluenced bymultiple contextual factors including the attributesof the risk score in the eyes of potential adopters (relativeadvantage, simplicity, and ease of use); adopters’ concerns(including implications for personal workload and how tomanage a positive score); their skills (ability to use and interpretthe technology); communication and influence (for example,whether key opinion leaders endorse it); system antecedents(including a healthcare organisation’s capacity to embrace newtechnologies, workflows, and ways of working); and externalinfluences (including policy drivers, incentive structures, andcompeting priorities).77 78

Challenges associated with risk scores in useWhile the developers of most diabetes risk scores are in littledoubt about their score’s positive attributes, this confidenceseems not to be shared by practitioners, who may doubt theaccuracy of the score or the efficacy of risk modificationstrategies, or both. Measuring diabetes risk competes forpractitioners’ attention with a host of other tasks, some of which



RESEARCH



Box 3: Questions to ask when selecting a diabetes risk score, and examples of intended use

What is the intended use case for the score?If intended for use:In clinical consultations, score should be based on data on the medical recordFor self assessment by lay people, score should be based on things a layperson would know or be able to measureIn prevention planning, score should be based on public health data

What is the target population?If intended for use in high ethnic and social diversity, a score that includes these variables may be more discriminatory

What is expected of the user of the score?If for opportunistic use in clinical encounters, the score must align with the structure and timeframe of such encounters and competenciesof the clinician, and (ideally) be linked to an appropriate point of care prompt. Work expected from the intended user of the score mayneed to be incentivised or remunerated, or both

What is expected of the participants?If to be completed by laypeople, the score must reflect the functional health literacy of the target population

What are the consequences of false positive and false negative classifications?In self completion scores, low sensitivity may falsely reassure large numbers of people at risk and deter them from seeking further advice

What is the completeness and accuracy of the data from which the score will be derived?A score based on automated analysis of electronic patient records may include multiple components but must be composed entirely ofdata that are routinely and reliably entered on the record in coded form, and readily searchable (thus, such scores are only likely to beuseful in areas where data quality in general practice records is high)

What resource implications are there?If the budget for implementing the score and analysing data is fixed, the cost of use must fall within this budget

Given the above, what would be the ideal statistical and other properties of the score in this context of use?What trade-offs should be made (sensitivity v specificity, brevity v comprehensiveness, one stage v two stage process)?

bring financial and other rewards. At the time of writing, fewopinion leaders in diabetes seem to be promoting particularscores or the estimation of diabetes risk generally—perhapsbecause, cognisant of the limited impacts shown to date(summarised in table 5), they are waiting for further evidenceof whether and how use of the risk score improves outcomes.Indeed, the utility of measuring diabetes risk in addition tocardiovascular risk is contested within the diabetes researchcommunity.79 In the United Kingdom, the imminent inclusionof an application for calculating QDScore on EMIS, thecountry’s most widely used general practice computer system,may encourage its use in the clinical encounter. But unless theassessment of diabetes risk becomes part of the UKQuality andOutcomes Framework, this task may continue to be perceivedas low priority by most general practitioners. Given currentevidence, perhaps this judgment is correct. Furthermore, thelow positive predictive values may spell trouble forcommissioners. Identifying someone as “[possibly] high risk”will inevitably entail a significant cost in clinical review, bloodtests, and (possibly) intervention and follow-up. Pending theresults of ongoing impact studies, this may not be the best useof scarce resources.Delivering diabetes prevention in people without any diseaserequires skills that traditionally trained clinicians may notpossess.80 We know almost nothing about the reach, uptake,practical challenges, acceptability, and cost of preventiveinterventions in high risk groups in different settings.12 Therelative benefit of detecting and targeting high risk people ratherthan implementing population-wide diabetes preventionstrategies is unknown.13 Effective prevention and early detectionof diabetes are likely to require strengthening of health systemsand development of new partnerships among the clinicians,community based lifestyle programmes, and healthcare funders.81

Mechanisms bywhich risk scoresmight haveimpactAlthough most authors of papers describing diabetes risk scoreshave hypothesised (or seem to have assumed) a clinicalmechanism of action (that the score would be used by theindividual’s clinician to target individual assessment and advice),the limited data available on impact studies (see table 5) suggestthat a particularly promising area for further research isinterventions that prompt self assessment—that is, laypeoplemeasuring their own risk of diabetes. The preliminary findingsfrom the impact studies covered in this review also suggest thatnot everyone at high risk is interested in coming forward forindividual preventive input, nor will they necessarily stay thecourse of such input. It follows that in areas where aggregateddata from electronic patient records are available, the diabetesrisk scores may be used as a population prediction tool—forexample, to produce small area statistics (perhaps as pictorialmaps) of diabetes risk across a population, thereby allowingtargeted design and implementation of community level publichealth interventions.82 Small area mapping of diabetes risk maybe a way of operationalising the recently published guidanceon diabetes prevention from the National Institute for Healthand Clinical Excellence, which recommends the use of “localand national tools . . . to identify local communities at high riskof developing diabetes to assess their specific needs.”83

Towards an impact oriented research agendafor risk scoresWe recommend that funding bodies and journal editors helptake this agenda forward by viewing the risk score in use as acomplex intervention and encouraging more applied researchstudies in which real people identified as at “high risk” using aparticular risk score are offered real interventions; success inrisk score development is measured in terms of patient relevantintermediate outcomes (for example, change in risk score) and



RESEARCH



final outcomes (incident diabetes and related morbidity) ratherthan in terms of the statistical properties of the tool; a qualitativecomponent (for example, process evaluation, organisationalcase study, patient’s experience of lifestyle modification)explores both facilitators and barriers of using the score in areal world setting; and an economic component evaluates costand cost effectiveness.

ConclusionMillions of participants across the world have alreadyparticipated in epidemiological studies aimed at developing adiabetes risk score. An extensive menu of possible scores arenow available to those who seek to use them clinically or tovalidate them in new populations, none of which is perfect butall of which have strengths. Nevertheless, despite the growingpublic health importance of type 2 diabetes and the enticingpossibility of prevention for those at high risk of developing it,questions remain about how best to undertake risk predictionand what to do with the results. Appropriately, the balance ofresearch effort is now shifting from devising new risk scores toexploring how best to use those we already have.

We thank Helen Elwell, librarian at the British Medical AssociationLibrary, for help with the literature search; Samuel Rigby for manuallyremoving duplicates; and Sietse Wieringa, Kaveh Memarzadeh, andNicholas Swetenham for help with translation of non-English papers.BMJ reviewers Wendy Hu and John Furler provided helpful commentson an earlier draft.Contributors: DN conceptualised the study, managed the project, briefedand supported all researchers, assisted with developing the searchstrategy and ran the search, scanned all titles and abstracts, extractedquantitative data on half the papers, citation tracked all papers, checkeda one third sample of the qualitative data extraction, and cowrote thepaper. TG conceptualised the qualitative component of the study,extracted qualitative data on all papers, independently citation trackedall papers, and led on writing the paper. RM independently scanned alltitles and abstracts of the electronic search, extracted quantitative datafrom some papers, assisted with other double checking, and helpedrevise drafts of the paper. TD helped revise and refine the study aims,independently double checked quantitative data extraction from allpapers, and helped revise drafts of the paper. CM advised on systematicreview methodology, helped develop the search strategy, extractedquantitative data from some papers, and helped revise drafts of thepaper. TG acts as guarantor.Funding: This study was funded by grants from Tower Hamlets,Newham, and City and Hackney primary care trusts, by a NationalInstitute of Health Research senior investigator award for TG, and byinternal funding for staff time from Barts and the London School ofMedicine and Dentistry. The funders had no input into the selection oranalysis of data or the content of the final manuscript.Competing interests: All authors have completed the ICMJE uniformdisclosure form at www.icmje.org/coi_disclosure.pdf (available onrequest from the corresponding author) and declare: no support fromany organisation for the submitted work; no financial relationships withany organisations that might have an interest in the submitted work inthe previous three years; and no other relationships or activities thatcould appear to have influenced the submitted work.Ethical approval: Not required.Data sharing: No additional data available.

1 Shaw JE, Sicree RA, Zimmet PZ. Global estimates of the prevalence of diabetes for 2010and 2030. Diabetes Res Clin Pract 2010;87:4-14.

2 Holman N, Forouhi NG, Goyder E,Wild SH. The Association of Public Health Observatories(APHO) Diabetes Prevalence Model: estimates of total diabetes prevalence for England,2010-2030. Diabetic Med 2011;28:575-82.

3 National Diabetes Clearinghouse. National Diabetes Statistics 2011. National Institute ofHealth, USA. http://diabetes.niddk.nih.gov/dm/pubs/statistics/#Diagnosed20.

4 Yang SH, Dou KF, Song WJ. Prevalence of diabetes among men and women in China.N Engl J Med 2010;362:2425-6.

5 Saadi H, Carruthers SG, Nagelkerke N, Al-Maskari F, Afandi B, Reed R, et al. Prevalenceof diabetes mellitus and its complications in a population-based sample in Al Ain, UnitedArab Emirates. Diabetes Res Clin Pract 2007;78:369-77.

6 Williamson DF, Vinicor F, Bowman BA. Primary prevention of type 2 diabetes mellitus bylifestyle intervention: implications for health policy. Ann Intern Med 2004;140:951-7.

7 Aekplakorn W, Bunnag P, Woodward M, Sritara P, Cheepudomwit S, Yamwong S, et al.A risk score for predicting incident diabetes in the Thai population. Diabetes Care2006;29:1872-7.

8 Hippisley-Cox J, Coupland C, Robson J, Sheikh A, Brindle P. Predicting risk of type 2diabetes in England and Wales: prospective derivation and validation of QDScore. BMJ2009;338:b880.

9 Meigs JB, Shrader P, Sullivan LM, McAteer JB, Fox CS, Dupuis J, et al. Genotype scorein addition to common risk factors for prediction of type 2 diabetes. N Engl J Med2008;359:2208-9.

10 Talmud PJ, Hingorani AD, Cooper JA, Marmot MG, Brunner EJ, Kumari M, et al. Utilityof genetic and non-genetic risk factors in prediction of type 2 diabetes: Whitehall IIprospective cohort study. BMJ 2010;340:b4838.

11 Park KS. The search for genetic risk factors of type 2 diabetes mellitus. Diabetes MetabJ 2011;35:12-22.

12 Simmons RK, Unwin N, Griffin SJ. International Diabetes Federation: an update of theevidence concerning the prevention of type 2 diabetes. Diabetes Res Clin Pract2010;87:143-9.

13 Yates T, Davies M, Khunti K. Preventing type 2 diabetes: can we make the evidencework? Postgrad Med J 2009;85:475-80.

14 Simmons RK, Echouffo-Tcheugui JB, Griffin SJ. Screening for type 2 diabetes: an updateof the evidence. Diabetes Obes Metab 2010;12:838-44.

15 Gillies CL, Lambert PC, Abrams KR, Sutton AJ, Cooper NJ, Hsu RT, et al. Differentstrategies for screening and prevention of type 2 diabetes in adults: cost effectivenessanalysis. BMJ 2008;336:1180-5.

16 Pan XR, Li GW, Hu YH, Wang JX, Yang WY, An ZX, et al. Effects of diet and exercise inpreventing NIDDM in people with impaired glucose tolerance. The Da Qing IGT andDiabetes Study. Diabetes Care 1997;20:537-44.

17 Tuomilehto J, Lindstrom J. The major diabetes prevention trials. Curr Diab Rep2003;3:115-22.

18 Knowler WC, Barrett-Connor E, Fowler SE, Hamman RF, Lachin JM, Walker EA, et al.Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. NEngl J Med 2002;346:393-403.

19 Mann DM, Bertoni AG, Shimbo D, Carnethon MR, Chen H, Jenny NS, et al. Comparativevalidity of 3 diabetes mellitus risk prediction scoring models in a multiethnic US cohort:the Multi-Ethnic Study of Atherosclerosis. Am J Epidemiol 2010;171:980-8.

20 Tuomilehto J, Lindstrom J, Hellmich M, Lehmacher W, Westermeier T, Evers T, et al.Development and validation of a risk-score model for subjects with impaired glucosetolerance for the assessment of the risk of type 2 diabetes mellitus-The STOP-NIDDMrisk-score. Diabetes Res Clin Pract 2010;87:267-74.

21 National Institute for Health and Clinical Excellence. Lipid modification. 2011. www.nice.org.uk/nicemedia/live/11982/40675/40675.pdf.

22 McElduff P, Lyratzopoulos G, Edwards R, Heller RF, Shekelle P, Roland M. Will changesin primary care improve health outcomes? Modelling the impact of financial incentivesintroduced to improve quality of care in the UK. Qual Saf Health Care 2004;13:191-7.

23 Anon. Type 2 diabetes risk evaluation for men and women aged 40-49 years at high riskof type 2 diabetes. MBS item 713. 2011. www.agpn.com.au/__data/assets/pdf_file/0015/15009/20081126_brf_practice-detailing-card-item-713.pdf.

24 Sytkowski PA, Kannel WB, D’Agostino RB. Changes in risk factors and the decline inmortality from cardiovascular disease. The Framingham Heart Study. N Engl J Med1990;322:1635-41.

25 Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognosticresearch: what, why, and how? BMJ 2009;338:b375.

26 Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research:validating a prognostic model. BMJ 2009;338:b605.

27 Wannamethee SG, Papacosta O, Whincup PH, Thomas MC, Carson C, Lawlor DA, etal. The potential for a two-stage diabetes risk algorithm combining non-laboratory-basedscores with subsequent routine non-fasting blood tests: results from prospective studiesin older men and women. Diabet Med 2011;28:23-30.

28 Wareham NJ, Griffin SJ. Risk scores for predicting type 2 diabetes: comparing axes andspades. Diabetologia 2011;54:994-5.

29 Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research:developing a prognostic model. BMJ 2009;338:b604.

30 Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research:application and impact of prognostic models in clinical practice. BMJ 2009;338:b606.

31 Pawson R, Tilley N. Realistic evaluation . Sage, 1997.32 Kahn KS, Kunz R, Kleijnen J, Antes G. Systematic reviews to support evidence-based

medicine , 2nd ed. Royal Society of Medicine, 2011.33 Centre for Reviews and Dissemination. Systematic Reviews: centre for reviews and

dissemination guidance for undertaking reviews in health care. York Publishing Services.2011. www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf.

34 Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—a new method ofsystematic review designed for complex policy interventions. J Health Serv Res Policy2005;10(suppl 1):21-34.

35 Greenhalgh T, Wong G,Westhorp G, Pawson R, on behalf of the RAMESES study group.Protocol-realist and meta-narrative evidence synthesis: evolving standards (RAMESES).BMC Med Res Methodol 2011;11:115.

36 Balkau B, Lange C, Fezeu L, Tichet J, de Lauzon-Guillain B, Czernichow S, et al. Predictingdiabetes: clinical, biological, and genetic approaches: data from the Epidemiological Studyon the Insulin Resistance Syndrome (DESIR). Diabetes Care 2008;31:2056-61.

37 Chen L, Magliano DJ, Balkau B, Colagiuri S, Zimmet PZ, Tonkin AM, et al. AUSDRISK:an Australian type 2 diabetes risk assessment tool based on demographic, lifestyle andsimple anthropometric measures. Med J Aust 2010;192:197-202.

38 Chuang SY, Yeh WT, Wu YL, Chang HY, Pan WH, Tsao CK. Prediction equations andpoint system derived from large-scale health check-up data for estimating diabetic risk inthe Chinese population of Taiwan. Diabetes Res Clin Pract 2011;92:128-36.



RESEARCH

http://www.icmje.org/coi_disclosure.pdf

http://diabetes.niddk.nih.gov/dm/pubs/statistics/#Diagnosed20

http://www.nice.org.uk/nicemedia/live/11982/40675/40675.pdf

http://www.nice.org.uk/nicemedia/live/11982/40675/40675.pdf

http://www.agpn.com.au/__data/assets/pdf_file/0015/15009/20081126_brf_practice-detailing-card-item-713.pdf

http://www.agpn.com.au/__data/assets/pdf_file/0015/15009/20081126_brf_practice-detailing-card-item-713.pdf

http://www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf



What is already known on this topic

The many known risk factors for type 2 diabetes can be combined in statistical models to produce risk scores

What this study adds

Dozens of risk models and scores for diabetes have been developed and validated in different settingsSociodemographic and clinical data were much better predictors of diabetes risk than genetic markersResearch on this topic is beginning to shift from developing new statistical risk models to considering the use and impact of risk scoresin the real world

39 GaoWG, Qiao Q, Pitkaniemi J, Wild S, Magliano D, Shaw J, et al. Risk prediction modelsfor the development of diabetes in Mauritian Indians. Diabet Med 2009;26:996-1002.

40 Joseph J, Svartberg J, Njolstad I, Schirmer H. Incidence of and risk factors for type-2diabetes in a general population: the Tromso Study.Scand J Public Health 2010;38:768-75.

41 Kahn HS, Cheng YJ, Thompson TJ, Imperatore G, Gregg EW. Two risk-scoring systemsfor predicting incident diabetes mellitus in US adults age 45 to 64 years. Ann Intern Med2009;150:741-51.

42 Kolberg JA, Jorgensen T, Gerwien RW, Hamren S, McKenna MP, Moler E, et al.Development of a type 2 diabetes risk model from a panel of serum biomarkers from theInter99 cohort. Diabetes Care 2009;32:1207-12.

43 Liu M, Pan C, Jin M. A Chinese diabetes risk score for screening of undiagnosed diabetesand abnormal glucose tolerance. Diabetes Technol Ther 2011;13:501-7.

44 Mehrabi Y, Sarbakhsh P, Hadaegh F, Khadem-Maboudi A. Prediction of diabetes usinglogic regression. Iran J Endocrinol Metab 2010;12:16-24.

45 Rathmann W, Kowall B, Heier M, Herder C, Holle R, Thorand B, et al. Prediction modelsfor incident type 2 diabetes mellitus in the older population: KORA S4/F4 cohort study.Diabet Med 2010;27:1116-23.

46 Schmidt MI, Duncan BB, Bang H, Pankow JS, Ballantyne CM, Golden SH, et al. Identifyingindividuals at high risk for diabetes: the Atherosclerosis Risk in Communities study.Diabetes Care 2005;28:2013-8.

47 Schulze MB, Weikert C, Pischon T, Bergmann MM, Al-Hasani H, Schleicher E, et al. Useof multiple metabolic and genetic markers to improve the prediction of type 2 diabetes:the EPIC-Potsdam Study. Diabetes Care 2009;32:2116-9.

48 Stern MP, Morales PA, Valdez RA, Monterrosa A, Haffner SM, Mitchell BD, et al. Predictingdiabetes. Moving beyond impaired glucose tolerance. Diabetes 1993;42:706-14.

49 Stern MP,Williams K, Haffner SM. Identification of persons at high risk for type 2 diabetesmellitus: do we need the oral glucose tolerance test? Ann Intern Med 2002;136:575-81.

50 von Eckardstein A, Schulte H, Assmann G. Risk for diabetes mellitus in middle-agedCaucasian male participants of the PROCAM study: implications for the definition ofimpaired fasting glucose by the American Diabetes Association. Prospective CardiovascularMunster. J Clin Endocrinol Metab 2000;85:3101-8.

51 Wilson PW, Meigs JB, Sullivan L, Fox CS, Nathan DM, D’Agostino RB Sr. Prediction ofincident diabetes mellitus in middle-aged adults: the Framingham Offspring Study. ArchIntern Med 2007;167:1068-74.

52 Alssema M, Feskens EJ, Bakker SJ, Gansevoort RT, Boer JM, Heine RJ, et al. [Finnishquestionnaire reasonably good predictor of the incidence of diabetes in The Netherlands].Ned Tijdschr Geneeskd 2008;152:2418-24.

53 Alssema M, Vistisen D, Heymans MW, Nijpels G, Glumer C, Zimmet PZ, et al. TheEvaluation of Screening and Early Detection Strategies for Type 2 Diabetes and ImpairedGlucose Tolerance (DETECT-2) update of the Finnish diabetes risk score for predictionof incident type 2 diabetes. Diabetologia 2011;54:1004-12.

54 Bozorgmanesh M, Hadaegh F, Azizi F. Transportability of the updated diabetes predictionmodel from Atherosclerosis Risk in Communities Study to a Middle Eastern adultpopulation: community-based cohort study. Acta Diabetol 2010; doi: 10.1007/s00592-010-0241-1.

55 Bozorgmanesh M, Hadaegh F, Zabetian A, Azizi F. San Antonio heart study diabetesprediction model applicable to a Middle Eastern population? Tehran glucose and lipidstudy. Int J Public Health 2010;55:315-23.

56 Cameron AJ, Magliano DJ, Zimmet PZ, Welborn TA, Colagiuri S, Tonkin AM, et al. Themetabolic syndrome as a tool for predicting future diabetes: the AusDiab study. J InternMed 2008;264:177-86.

57 Collins GS, Altman DG. External validation of QDSCORE((R)) for predicting the 10-yearrisk of developing type 2 diabetes. Diabet Med 2011;28:599-607.

58 Guerrero-Romero F, Rodriguez-Moran M. [Validation of an instrument for screening casesof type 2 diabetes and monitoring at-risk individuals in Mexico]. Rev Panam Salud Publica2010;27:181-6.

59 Kanaya AM, Wassel Fyr CL, de Rekeneire N, Shorr RI, Schwartz AV, Goodpaster BH,et al. Predicting the development of diabetes in older adults: the derivation and validationof a prediction rule. Diabetes Care 2005;28:404-8.

60 Mainous AG, III, Diaz VA, Everett CJ. Assessing risk for development of diabetes in youngadults. Ann Fam Med 2007;5:425-9.

61 McNeely MJ, Boyko EJ, Leonetti DL, Kahn SE, Fujimoto WY. Comparison of a clinicalmodel, the oral glucose tolerance test, and fasting glucose for prediction of type 2 diabetesrisk in Japanese Americans. Diabetes Care 2003;26:758-63.

62 Nichols GA, Brown JB. Validating the FraminghamOffspring Study equations for predictingincident diabetes mellitus. Am J Manag Care 2008;14:574-80.

63 Rahman M, Simmons RK, Harding AH, Wareham NJ, Griffin SJ. A simple risk scoreidentifies individuals at high risk of developing type 2 diabetes: a prospective cohort study.Fam Pract 2008;25:191-6.

64 Urdea M, Kolberg J, Wilber J, Gerwien R, Moler E, Rowe M, et al. Validation of amultimarker model for assessing risk of type 2 diabetes from a five-year prospective studyof 6784 Danish people (Inter99). J Diabetes Sci Technol 2009;3:748-55.

65 Wannamethee SG, Shaper AG, Lennon L, Morris RW.Metabolic syndrome vs FraminghamRisk Score for prediction of coronary heart disease, stroke, and type 2 diabetes mellitus.Arch Intern Med 2005;165:2644-50.

66 Bozorgmanesh M, Hadaegh F, Ghaffari S, Harati H, Azizi F. A simple risk score effectivelypredicted type 2 diabetes in Iranian adult population: population-based cohort study. EurJ Public Health 2011;21:554-9.

67 Chien K, Cai T, Hsu H, Su T, Chang W, Chen M, et al. A prediction model for type 2diabetes risk among Chinese people. Diabetologia 2009;52:443-50.

68 Lindstrom J, Tuomilehto J. The diabetes risk score: a practical tool to predict type 2diabetes risk. Diabetes Care 2003;26:725-31.

69 Rosella LC, Manuel DG, Burchill C, Stukel TA. A population-based risk algorithm for thedevelopment of diabetes: development and validation of the Diabetes Population RiskTool (DPoRT). J Epidemiol Community Health 2011;65:613-20.

70 SchulzeMB, Hoffmann K, Boeing H, Linseisen J, Rohrmann S, Mohlig M, et al. An accuraterisk score based on anthropometric, dietary, and lifestyle factors to predict the developmentof type 2 diabetes. Diabetes Care 2007;30:510-5.

71 Simmons RK, Harding AH, Wareham NJ, Griffin SJ. Do simple questions about diet andphysical activity help to identify those at risk of type 2 diabetes?Diabet Med 2007;24:830-5.

72 Sun F, Tao Q, Zhan S. An accurate risk score for estimation 5-year risk of type 2 diabetesbased on a health screening population in Taiwan. Diabetes Res Clin Pract2009;85:228-34.

73 Harrell F. Regression modelling strategies with applications to linear models, logisticregression and survival analysis . Springer-Verlag, 2001.

74 Lindstrom J, Absetz P, Hemio K, Peltomaki P, Peltonen M. Reducing the risk of type 2diabetes with nutrition and physical activity—efficacy and implementation of lifestyleinterventions in Finland. Public Health Nutr 2010;13(6A):993-9.

75 Herman WH. Predicting risk for diabetes: choosing (or building) the right model. AnnIntern Med 2009;150:812-4.

76 Wright C, Dent T. Quality standards in risk prediction: summary of an expert meeting.PHG Foundation, 2011.www.phgfoundation.org.

77 Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Diffusion ofinnovations in service organisations: systematic literature review and recommendationsfor future research. Milbank Q 2004;82:581-629.

78 Davis FD. Perceived usefulness, perceived ease of use, and user acceptance ofinformation technology. MIS Quarterly 1989;13:319-40.

79 Chamnan P, Simmons RK, Jackson R, Khaw KT, Wareham NJ, Griffin SJ. Non-diabetichyperglycaemia and cardiovascular risk: moving beyond categorisation to individualinterpretation of absolute risk. Diabetologia 2011;54:291-9.

80 Saaristo T, Peltonen M, Keinanen-Kiukaanniemi S, Vanhala M, Saltevo J, Niskanen L,et al. National type 2 diabetes prevention programme in Finland: FIN-D2D. Int JCircumpolar Health 2007;66:101-12.

81 Narayan KM, Williamson DF. Prevention of type 2 diabetes: risk status, clinic, andcommunity. J Gen Intern Med 2010;25:154-7.

82 Stewart JE, Battersby SE, Lopez-De FA, Remington KC, Hardin JW, Mayfield-Smith K.Diabetes and the socioeconomic and built environment: geovisualization of diseaseprevalence and potential contextual associations using ring maps. Int J Health Geogr2011;10:18.

83 National Institute for Health and Clinical Excellence. Preventing type 2 diabetes—populationand community interventions. 2011. www.nice.org.uk/guidance/PH35.

84 Stone MA, Camosso-Stefinovic J, Wilkinson J, de Lusignan S, Hattersley AT, Khunti K.Incorrect and incomplete coding and classification of diabetes: a systematic review.DiabetMed 2010;27:491-7.

85 Rathmann W, Kowall B, Schulze MB. Development of a type 2 diabetes risk model froma panel of serum biomarkers from the Inter99 cohort: response to Kolberg et al. DiabetesCare 2010;33:e28.

86 Stern M, Williams K, Eddy D, Kahn R. Validation of prediction of diabetes by theArchimedes model and comparison with other predicting models. Diabetes Care2008;31:1670-1.

87 Colagiuri S, Vita P, Cardona-Morrell M, Singh MF, Farrell L, Milat A, et al. The SydneyDiabetes Prevention Program: a community-based translational study. BMCPublic Health2010;10:328.

88 Absetz P, Oldenburg B, Hankonen N, Valve R, Heinonen H, Nissinen A, et al. Type 2diabetes prevention in the real world: three-year results of the GOAL lifestyleimplementation trial. Diabetes Care 2009;32:1418-20.

89 Jallinoja P, Pajari P, Absetz P. Repertoires of lifestyle change and self-responsibilityamong participants in an intervention to prevent type 2 diabetes. Scand J Caring Sci2008;22:455-62.

90 Kulzer B, Hermanns N, Gorges D, Schwarz P, Haak T. Prevention of diabetesself-management program (PREDIAS): effects on weight, metabolic risk factors, andbehavioral outcomes. Diabetes Care 2009;32:1143-6.

91 Laatikainen T, Dunbar JA, Chapman A, Kilkkinen A, Vartiainen E, Heistaro S, et al.Prevention of type 2 diabetes by lifestyle intervention in an Australian primary health caresetting: Greater Green Triangle (GGT) Diabetes Prevention Project. BMC Public Health2007;7:249.

92 Vermunt PW, Milder IE, Wielaard F, van Oers JA, Westert GP. An active strategy toidentify individuals eligible for type 2 diabetes prevention by lifestyle intervention in Dutchprimary care: the APHRODITE study. Fam Pract 2010;27:312-9.

Accepted: 5 October 2011

Cite this as: BMJ 2011;343:d7163This is an open-access article distributed under the terms of the Creative CommonsAttribution Non-commercial License, which permits use, distribution, and reproduction inany medium, provided the original work is properly cited, the use is non commercial andis otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.



RESEARCH

http://dx.doi.org/10.1007/s00592-010-0241-1

http://dx.doi.org/10.1007/s00592-010-0241-1

http://www.phgfoundation.org

http://www.nice.org.uk/guidance/PH35

http://creativecommons.org/licenses/by-nc/2.0/

http://creativecommons.org/licenses/by-nc/2.0/

http://creativecommons.org/licenses/by-nc/2.0/legalcode





RESEARCH



Tables

Table 1| Summary of 43 papers from which 94 diabetes risk models or scores were identified for systematic review

Howincident

Howdiabetes

Age: mean(SD) orrange

Duration:mean (SD),

range (years),or as reported

Samplesize

Why inceptioncohort wasassembled

Study designand sampling

frameName of risk

scoreName of studyCountryStudy*

diabeteswas

diagnosed

wasexcluded atinception

Diagnosis ofdiabetes,fastingplasmaglucose, oralglucose

History ofdiabetes,fastingplasmaglucose, oralglucose

35-5412, 1985-97; 5,1998-2003

3254;2420

Study of vascular risk;implicitly, study ofdiabetes risk

Power plantworkers:cohortderivationstudy; andcohort external

NSElectricGeneratingAuthority ofThailand Study

ThailandAekplakorn 20067

(two of six modelsreported)

tolerancetolerancevalidationstudy test, diabetestest; and not

stated drugs; andfastingplasmaglucose

NSOral glucosetolerancetest; fastingplasmaglucose

≥45; 28-756.4 (0.5),1989-98; 4.2(0.4),1997-2003

2439;3345

Studies of glucosetolerance;cardiovasculardisease and renaldisease

Cohortexternalvalidationstudy, sampleNS

ModifiedFINDRISC forDutch population

Hoorn study,PREVEND study

NetherlandsAlssema 200852

(two of threemodels reported)

Oral glucosetolerance test


Ranged from46.3 (7.8) to60.3 (6.9) infive studies

4.8-5,1986-2001

18 301NSCohortexternalvalidationstudy ofFINDRISC incombined

Based onFINDRISC

DETECT-2(includesAusdiab, Hoorn,Inter99,MONICA,Whitehall-II)

Netherlands,Denmark,Sweden, UK,Australia,Mauritius

Alssema 201153


samples fromfive studies

Fastingplasmaglucose,diabetesdrugs

NS47 (10)9 (<1996)1863and1954

Study of insulinresistance syndrome

Cohortderivationstudy involunteers forfree healthexaminations

NSDESIRFranceBalkau 200836

(both modelsreported)

Oral glucosetolerancetest, fastingplasmaglucose,diabetesdrugs


Men 42.8(14.8);women 40.7(12.5)

6, 1999-20085018Study of lipid andglucose risk factors

Cohortexternalvalidationstudy ingeneralpopulation

Modified ARIC(AtherosclerosisRisk InCommunities)

Tehran Lipid andGlucose Study

IranBozorgmanesh201054



41.6 (13.2)6, 1999-20085018Study of lipid andglucose risk factors

Cohortderivationstudy, andcohort externalvalidationstudy, ingeneralpopulation

NSTehran Lipid andGlucose Study

IranBozorgmanesh201166 (all fivemodels reported)



Men 42.8(14.8);women 40.7(12.5)

6.3, 1999-20085018Study of lipid andglucose risk factors


San Antoniodiabetesprediction model

Tehran Lipid andGlucose Study

IranBozorgmanesh201055 (one of sixmodels reported)

WHO criteriaWHO criteria50.9(50.6-51.2)

5, 200011 247Diabetesincidence/prevalencestudy


Diabetespredictionmodel; andFinnish diabetesrisk score

AusDiabAustraliaCameron 200856




RESEARCH



Table 1 (continued)

Howincidentdiabeteswas

diagnosed

Howdiabeteswas

excluded atinception


Duration:mean (SD),


Samplesize



frameName of risk


Fastingplasma

NS≥255, 1999-200511 247Diabetesincidence/prevalencestudy

Cohortderivationstudy in

AusdriskAusdiabAustraliaChen 201037 (allsix modelsreported) glucose, oral

generalpopulation

glucosetolerancetest, diabetesdrugs



5410, 19902960NSCohortderivationstudy ingeneralpopulation

Cambridge riskscore as well asseveralunnamed

Chin-ShanCommunityCardiovascularCohort

TaiwanChien 200967

(seven of eightmodels reported)



49.2 (10.4)5.61 (3.33),1994-2006

19 919(3scores),6111 (3scores)

Data from routinehealth checks

Cohortderivationstudy in privatehealth clinicpatients

NSMJ HealthScreen

TaiwanChuang 201138(allsix modelsreported)

Read codeC10(diagnosis ofdiabetes)

Read codeC10(diagnosis ofdiabetes)

Median(interquartilerange) men44 (34-57),women 43(34-56)

15, 1993-20082 396392

Data from primarycare database

Cohortexternalvalidationstudy in UKgeneralpracticepopulation

QDScoreTHIN databaseUKCollins 201157


History ofdiabetes,fastingplasmaglucose, oralglucosetolerance test

<6511, 1987-981544Study ofnon-communicablediseases

Cohortderivationstudy inrandomsample ofentire islandpopulation

NSNSMauritiusGao 200939(one ofthree modelsreported)

tolerancetest, diabetesdrugs

NSNS20-657 (range4.5-10),1996-2006

525NSCohortexternalvalidationstudy, sampleNS

ITD (InstrumentoPara ElTamizaje de ladiabetes tipo 2)

NSMexicoGuerrero-Romero201058 (one of twomodels reported)

Read codeC10(diagnosis ofdiabetes)less thosereceiving

Read codeC10(diagnosis ofdiabetes)less thosereceiving

25-79(median 41)

15, 1993-20082samples2 540753 and1 232832

Data from primarycare database

Cohortderivationstudy ingeneralpracticeelectronic

QDScoreQResearchdatabase

UKHippisley-Cox20098 (two of fourmodels reported)

insulin <age35

insulin <age35

recorddatabase

“T2DMevent”

Self report,haemoglobinA1c, ICD-10,plasmaglucose,diabetesdrugs

25-9810.8 (median),1994-2005

26 168NSCohortderivationstudy in singleacademichealth centre(Tromsø)

NSTromsø StudyNorwayJoseph 201040

Varied overstudy period.Fastingplasmaglucose, oralglucose

NS45-6414.9,1987-2003

9587;3142;3142

Study ofatherosclerosis risk

Cohortderivationstudy in fourUScommunities

NSARIC(AtherosclerosisRisk inCommunities)

USAKahn 200941 (allthree modelsreported)

tolerancetest, selfreport,



RESEARCH



Table 1 (continued)


diagnosed

Howdiabeteswas



Duration:mean (SD),


Samplesize



frameName of risk


record,survey

Fastingplasmaglucose

Self report,diabetesdrugs, fastingplasmaglucose

70-796, 1997-20032503NSCohortexternalvalidationstudy in twoclinics(Memphis andPittsburgh)

NSHealth, Aging,and BodyCompositionStudy(Validation)

USAKanaya 200559

Fastingplasmaglucose, oralglucosetolerance test


30-605, NS632Lifestyle interventiontrial for cardiovasculardisease

Cohortderivationstudy, samplefrom Danishcivil register

NSInter99USAKolberg 200942

Fastingplasmaglucose, oralglucosetolerancetest, diabetesdrugs


45-6410, 1987-97; 5,1992-7

4746;4615

NSCohortderivationstudy, nationalpopulationregister; andcohort externalvalidation

Diabetes riskscore

FINRISK StudiesFinlandLindstrom 200368


study,FINRISK

Self report,fastingplasmaglucose, oralglucosetolerance


48-8710, 1996-20061457Analysis of routinedata from healthchecks

Cohortderivationstudy inhospitalscreeningcentre formilitary officers

Chinesediabetes riskscore

NSChinaLiu 201143(allthree modelsreported)

test, diabetesdrugs

Self report,fastingplasmaglucose

Self report,fastingplasmaglucose

18-3010, 1985-952543Study of coronaryheart disease risk

Cohortexternalvalidationstudy in youngadultsrecruited toCARDIA study

NSCoronary ArteryRiskDevelopment inYoung Adults(CARDIA)

USAMainous 200760



61.6 (45-84)4.75, 2000-65329Study ofatherosclerosis risk

Cohortexternalvalidationstudy in adultswithoutcardiovascular

NSMulti-ethnicStudy ofAtherosclerosis(MESA)

USAMann 201019 (allthree modelsreported)

disease in sixdiverse UScommunities



52.1 (34-75)5-10, NS518Community diabetesstudy

Cohortexternalvalidationstudy, sampleNS

NSJapaneseAmericanCommunityDiabetes Study

USAMcNeely 200361

(one of twomodels reported)

NSFastingplasmaglucose, oralglucosetolerancetest, diabetesdrugs

Men 43.4(14.1),women 40.4(12.6)

9, 1998-20075114Study of lipid andglucose risk factors

Cohortderivationstudy, sampleNS

NSTehran Lipid andGlucose Study

IranMehrabi 201044

(one of fourmodels reported)



RESEARCH



Table 1 (continued)


diagnosed

Howdiabeteswas



Duration:mean (SD),


Samplesize



frameName of risk


Fastingplasma

Fastingplasma

28-6228, 1971-20012377Study of children ofFramingham HeartStudy participants

Cohortexternalvalidation

Genotype scoreFraminghamOffspring Study

USAMeigs 20089

glucose,glucose,study, sampleNS

diabetesdrugs

diabetesdrugs

Diagnosis ofdiabetes(ICD-9codes),fastingplasma

NS57.47, 1999-200720, 644Analysis of healthmaintenanceorganisationelectronic records

Cohortexternalvalidationstudy in healthmaintenanceorganisation

FraminghamOffspring Studyscore

KaiserPermanenteNorthwestelectronicrecords

USANichols 200862 (allthree modelsreported)

glucose,registeredpopulation diabetes

drugs

As inceptionSelf report,diabetesdrugs, clinicregisters,deathcertificates

58.9 (40-79)4.8 (1.3),1993-2000

24, 495Study of causes ofcancer

Cohortexternalvalidationstudy in UKgeneralpractice

Cambridge riskscore

EuropeanProspectiveInvestigation ofCancer(EPIC)-Norfolk

UKRahman 200863

Diagnosis ofdiabetes, oralglucosetolerance test


55-74Implicitly, 7,1999-2008

1202NSCohortderivationstudy, sampleNS

NSKORA S4/F4study

GermanyRathmann 201085

(all three modelsreported)

Hospitaldiagnosis ofdiabetes(ICD code),physicianclaims

NSMen 44,women 46;men 44,women 47;men 44,women 46

9, 1996-7; 9,1996-2005; 5,2000-5

19 795;9899; 26465

Health surveyCohortderivationstudy, sampleNS

Dport (Diabetespopulation atrisk tool)

NationalPopulationHealthSurvey—Ontario

CanadaRosella 201069 (allthree modelsreported)


Diagnosis ofdiabetes(includingself report),fastingplasma

Median 54(45-64)

9, 1987-987915Study ofatherosclerosis risk

Cohortderivationstudy in fourUScommunities

NSARIC(AtherosclerosisRisk inCommunities)

USASchmidt 200546

(all three modelsreported)

toleranceglucose,test, diabetesdrugs

diabetesdrugs

Self report,verified byICD-10; selfreport,record, deathcertificate

NSMen 40-65,women35-65; NS

7, NS; 5, NS27 548;25 540

Study of causes ofcancer

Cohortderivationstudy(Potsdam);cohort externalvalidation

Germandiabetes riskscore

EPIC-Potsdam;andEPIC-Heidelberg

GermanySchulze 200770


study(Heidelberg)

Self reportverified byphysician

Self reportverified byphysician

35-657.1, 19941962Study of causes ofcancer

Cohortderivationstudy ingeneralpopulation(Potsdam)

Adaptation ofGermandiabetes riskscore

EPIC-PotsdamGermanySchulze 200947

Healthcheck, clinicregisters,diabetesdrugs,haemoglobinA1C

Self report40-794.6, 1993-200012 591Study of causes ofcancer

Cohortderivationstudy; cohortexternalvalidationstudy, sampleNS

NS; Cambridgerisk score

EPIC-NorfolkUKSimmons 200771




RESEARCH



Table 1 (continued)


diagnosed

Howdiabeteswas



Duration:mean (SD),


Samplesize



frameName of risk


Fastingplasma

Fastingplasma

25-648, 1979-872217Population basedstudy of diabetes and

Cohortderivation

NSSan AntonioHeart Study

USAStern 199348 (twoof six modelsreported) glucose, oralglucose, oralcardiovascular

diseasestudy, sampleNS glucoseglucose

tolerancetolerancetest, diabetesdrugs

test, diabetesdrugs



25-647-8, 1979-885158Population basedstudy of diabetes andcardiovasculardisease

Cohortderivationstudy, sampleNS

NSSan AntonioHeart Study

USAStern 200286 (bothmodels reported)

NSFastingplasmaglucose,diabetesdrugs

47.5 (35-74)Median 3.15,1997-2006

10 294NSCohortderivationstudy in privatepatient sample

AtherosclerosisRisk inCommunities(ARIC) score

Taiwanhealth-check-updatabase(MJLPD)

TaiwanSun 200972 (threeof six modelsreported)

Oral glucosetolerancetest, diabetesdrugs, selfreport ofdoctordiagnosis


49 (35-55)11.7 (median),NS

8713Study of health in civilservants

Cohortexternalvalidationstudy in civilservantsample

Cambridge RiskScore; andFraminghamOffspring Studyscore

Whitehall IIUKTalmud 201010


NSNS40-555, NS399; 400Primary preventionstudy ofcardiovasculardisease

Cohortexternalvalidationstudy, samplenot stated

PreDx diabetesrisk scoretraining set;PreDx diabetesrisk scorevalidation set

Inter99DenmarkUrdea 200964 (onescore, two studies,both reported)

Self report,diabetesdrugs, fastingplasmaglucose

Self report,fastingplasmaglucose,diabetesdrugs

30-604-10, 1979-953737To examinecardiovascular riskfactors, events, andmortality

Cohortderivationstudy inemployees of52 companiesand authoritiesin Münster

Multiple logisticfunction model

PROCAM(ProspectiveCardiovascularMünster Study)

GermanyVon Eckardstein200050

Recordreview, selfreport

Doctordiagnosis ofdiabetes,fastingplasmaglucose

60-797, 1998-20076927Study ofcardiovascular risk

Cohortderivationstudy, samplenot stated

NSBritish RegionalHeart Study andBritish Women’sHeart and HealthStudy

UKWannamethee201127 (all threemodels reported)

NSRecall ofdoctordiagnosis,high bloodglucose

50.3 (5.7),40-59

21.3,1978-2000

5128Heart studyCohortexternalvalidationstudy insample ofmostly manualsocial class

Framingham riskscore

British RegionalHeart Study

UKWannamethee200565


History ofdiabetes, oralglucosetolerancetest, fastingplasma

547,mid-1990-2001

3140Population basedstudy of healthoutcomes

Cohortderivationstudy, samplenot stated

NSFraminghamOffspring Study

USAWilson 200751

(one of sevenmodels reported)

glucose,diabetesdrugs

NS=not stated; WHO=World Health Organization; ICD-10=International Classification of Disease, 10th revision; ICD-9=International Classification of Diseases, ninth revision.



RESEARCH



Table 1 (continued)


diagnosed

Howdiabeteswas



Duration:mean (SD),


Samplesize



frameName of risk


Some studies tested multiple models, with minimal difference in number of risk factors; in such cases authors’ preferred models were selected or, if no preference stated, wemade our own judgment.*Bracketed information shows how many scores tested by the original authors were included in this systematic review.



RESEARCH



Table 2| Key characteristics of 94 diabetes risk models or scores included in systematic review

%needing

Calibration

Positive/negativepredictive value

(%)AUROC(95% CI)

Sensitivity/specificity†%Components of score

Diabetesincidence

(%)*Studyfurthertests

NSHosmer-LemeshowP=0.8

NS/NS0.74 (0.71 to0.78)

77/60Age, BMI, waist circumference,hypertension, family history ofdiabetes in first degree relative

11.1Aekplakorn 20067

NSNSNS/NS0.75 (0.71 to0.80)

84.4/52.5Age, BMI, waist circumference,hypertension, family history ofdiabetes in first degree relative

5.2Aekplakorn 20067

28NS19/94 (cut-off ≥7);26/91 (cut-off ≥10)

0.71 (0.68 to0.75)

84/42 (cut-off ≥7); 52/76(cut-off ≥10)

Age, BMI, waist circumference, useof antihypertensive drugs, parentalhistory of diabetes, family history ofdiabetes in first degree relative

22.3 per1000personyears

Alssema 200852

16NS9/98 (cut-off ≥7);12/97 (cut-off ≥10)

0.77 (0.73 to0.80)

78/64 (cut-off ≥7); 43/85(cut-off ≥10)

Age, BMI, waist circumference, useof antihypertensive drugs, parentalhistory of diabetes, family history ofdiabetes in first degree relative

10.7 per1000personyears

Alssema 200852

NSNSNS/NS0.77 (0.75 to0.78)

NS/NSAge, BMI, waist circumference, useof antihypertensive drugs, history ofgestational diabetes

Range2.3-9.9across fivesubstudies

Alssema 201153

40Hosmer-LemeshowP=0.27

11/NS0.76 (0.75 to0.78)

76/63Age, BMI, waist circumference, useof antihypertensive drugs, history ofgestational diabetes, sex, smoking,family history of diabetes

Range2.3-9.9across fivesubstudies

Alssema 201153


NS/NS0.71 (NS)NS/NSWaist circumference, smoking,hypertension

7.5Balkau 200836


NS/NS0.83NS/NSWaist circumference, family historyof diabetes, hypertension

3.2Balkau 200836


NS/NSMen 0.79,women0.829

Men 71.6/75.3, women67.1/85.0

Age, family history of diabetes,hypertension, waist circumference,fasting plasma glucose level, height,pulse, triglyceride-high densitylipoprotein ratio

4.6Bozorgmanesh201154

NSNSNS/NS0.75 (0.72 to0.78)

NS/NSAge, family history of diabetes,systolic blood pressure, waist-hipratio, waist-height ratio


NSNSNS/NS0.85 (0.82 to0.87)

NS/NSFamily history of diabetes, systolicblood pressure, waist-height ratio,triglyceride-high density lipoproteinratio, fasting plasma glucose level


NSNSNS/NS0.86 (0.83 to0.89)

NS/NSFamily history of diabetes, systolicblood pressure, waist-height ratio,triglyceride-high density lipoproteinratio, fasting plasma glucose level,two hour postprandial plasma glucoselevel



NS/NS0.83 (0.80 to0.86)

75/77Systolic blood pressure, waist-heightratio, fasting plasma glucose level,triglyceride-high density lipoproteinratio, family history of diabetes



NS/NS0.78 (0.75 to0.81)

NS/NSNS4.6Bozorgmanesh201166

NSHosmer-LemeshowP<0.001, whenrecalibratedP=0.131

NS/NS0.83 (0.80 to0.86)

NS/NS“San Antonio diabetes predictionmodel”


19.3NS11.9/98.3NS62.4/82.3Age, sex, ethnicity, fasting plasmaglucose level, systolic blood pressure,high density lipoprotein cholesterollevel, BMI, family history of diabetes

2.0Cameron 200856

30.6NS6.8/98.2NS62.3/70.5NS2.0Cameron 200856



RESEARCH



Table 2 (continued)

%needingfurthertestsCalibration


(%)AUROC(95% CI)


Diabetesincidence

(%)*Study


NS/NS0.79 (0.76 to0.81)

NS/NSAge, sex, ethnicity, parental historyof diabetes, history of high blood

3.2Chen 201037

glucose levels, use ofantihypertensive drugs, lipid loweringdrugs, smoking, physical inactivity,waist circumference, BMI, education,occupation


NS/NS0.79 (0.76 to0.81)

NS/NSAge, sex, ethnicity, parental historyof diabetes, history of high bloodglucose levels, use ofantihypertensive drugs, lipid loweringdrugs, smoking, physical inactivity,waist circumference, BMI, education

3.2Chen 201037


NS/NS0.79 (0.76 to0.81)

NS/NSAge, sex, ethnicity, parental historyof diabetes, history of high bloodglucose levels, use ofantihypertensive drugs, lipid loweringdrugs, smoking, physical inactivity,waist circumference, BMI

3.2Chen 201037


NS/NS0.79 (0.76 to0.81)

NS/NSAge, sex, ethnicity, parental historyof diabetes, history of high bloodglucose levels, antihypertensivedrugs, smoking, physical inactivity,waist circumference, BMI

3.2Chen 201037


NS/NS0.78 (0.76 to0.81)

NS/NSAge, sex, ethnicity, parental historyof diabetes, history of high bloodglucose levels, use ofantihypertensive drugs, smoking,physical inactivity, waistcircumference

3.2Chen 201037


NS/NS0.78 (0.75 to0.80)

NS/NSAge, sex, ethnicity, parental historyof diabetes, history of high bloodglucose levels, use ofantihypertensive drugs, smoking,physical inactivity, BMI

3.2Chen 201037


NS/NS0.70 (0.68 to0.73)

52/78Age, BMI, white blood cell count,triglyceride level, high densitylipoprotein cholesterol level, fastingplasma glucose level

18.5Chien 200967

NSNSNS/NS0.70 (0.68 to0.73)

69/62Age, BMI, white blood cell count,triglyceride level, high densitylipoprotein cholesterol level, fastingplasma glucose level, family historyof diabetes, systolic blood pressure

18.5Chien 200967

NSNSNS/NS0.65 (0.62 to0.67)

NS/NSAge, sex, BMI, family history ofdiabetes, use of antihypertensivedrugs

18.5Chien 200967


NS/NSNS66/56NS18.5Chien 200967







NSNSNS/NS0.71 (0.70 to0.73)

NS/NSAge, sex, education, alcohol, BMI,waist circumference

6.4Chuang 201138

NSNSNS/NS0.720 (0.71to 0.74)

NS/NSAge, sex, education, alcohol, BMI,waist circumference, blood pressure,hypertension

6.4Chuang 201138



RESEARCH



Table 2 (continued)



(%)AUROC(95% CI)


Diabetesincidence

(%)*Study

NSNSNS/NS0.82 (0.81 to0.83)

NS/NSAge, sex, education, alcohol, BMI,waist circumference, triglyceride level,

6.4Chuang 201138

blood pressure, hypertension, fastingplasma glucose level

NSNSNS/NS0.75 (0.73 -0.78)

NS/NSAge, sex, education, alcohol, BMI,waist circumference, family history ofdiabetes

6.4Chuang 201138

NSNSNS/NS0.76 (0.73 to0.79)

NS/NSAge, sex, education, family history ofdiabetes, alcohol, BMI, waistcircumference, blood pressure,hypertension

6.4Chuang 201138

NSNSNS/NS0.84 (0.81 to0.86)

NS/NSAge, sex, education, alcoholconsumption, BMI, waistcircumference, blood pressure,hypertension, fasting plasma glucoselevel, triglyceride level, family historyof diabetes

6.4Chuang 201138

NSBrier score: men0.053 (0.051-0.054),women 0.041(0.040-0.043)

NS/NSWomen 0.81,men 0.80

NS/NSAge, sex, ethnicity, BMI, smoking,family history of diabetes,cardiovascular disease, Townsendscore, treated high blood pressure,current use of corticosteroids

3.0Collins 201157

NSNSNS/NSMen 0.62(0.56 to0.68),women 0.64(0.59 to 0.69)

Men 72 (71-74)/0.47(0.45-0.49), women 77(75-78)/0.50 (0.48-0.52)

BMI, waist circumference, familyhistory of diabetes

16.5Gao 200939

NSNS35/97.50.9192/71Age, sex, family history of diabetes,family history of hypertension, familyhistory of obesity, history ofgestational diabetes or macrosomia,fasting plasma glucose level, physicalinactivity, triglyceride level, systolicor diastolic blood pressure, BMI

11.8Guerrero-Romero201058

NSNSNS/NSNSNS/NSAge, sex, ethnicity, BMI, smoking,family history of diabetes, Townsendscore, treated hypertension,cardiovascular disease, current useof corticosteroids

3.1Hippisley-Cox20098

NSBrier score: men0.078 (0.075-0.080),women 0.058(0.055-0.060)

NS/NSWomen 0.85(0.85 to0.86), men0.83 (0.83 to0.84)

NS/NSAge, sex, ethnicity, BMI, smoking,family history of diabetes, Townsendscore, treated hypertension,cardiovascular disease, current useof corticosteroids

3.0Hippisley-Cox20098

NSNSNS/NSMen 0.87,women 0.88

NS/NSAge, BMI, total cholesterol,triglyceride level, high densitylipoprotein cholesterol level,hypertension, family history ofdiabetes, education, physicalinactivity, smoking

Men 2.5,women 1.5

Joseph 201040

NSNSNS/NSNSNS/NSSee next two rows for description ofboth models

Men 19.4,women18.6

Kahn 200941

NSNSNS/NS0.71 (0.69 to0.73)

69/64Waist circumference, parental historyof diabetes, hypertension, shortstature, black race, age >55, weight,pulse, smoking

17.7 at 10years

Kahn 200941



RESEARCH



Table 2 (continued)



(%)AUROC(95% CI)


Diabetesincidence

(%)*Study

NSNSNS/NS0.79 (0.77 to0.81)

74/71Glucose, waist circumference,parental history of diabetes,

17.7 at 10years

Kahn 200941

hypertension, triglyceride level, blackrace, high density lipoproteincholesterol level, short stature, highuric acid level, age >55, pulse,alcohol consumption

NSNSNS/NS0.71 (NS)NS/NSAge, sex, triglyceride level, fastingplasma glucose level

5.7Kanaya 200559

10%classifiedas highrisk

NSNS/NS0.78 (NS)NS/NSSix biomarkers: adiponectin, Creactive protein, ferritin, glucose,interleukin 2 receptor A, insulin

2.7Kolberg 200942

25% intwohighestriskcategories

NS0.13(0.11-0.15)/0.99(0.98-0.99)

0.85 (NS)78 (71-84)/77 (76-79)Age, BMI, waist circumference, useof antihypertensive drugs, history ofhypertension, physical inactivity, diet(vegetables, fruits or berries)

4.1Lindstrom 200368

26% ofmen and24% ofwomen intwohighest

NS0.05(0.04-0.06)/0.996(0.993-0.998)

0.87 (NS)81 (69-89)/76 (74-77)Age, BMI, waist circumference, useof antihypertensive drugs, history ofhypertension, physical inactivity, diet(vegetables, fruit or berries)

1.5Lindstrom 200368

riskcategories

NSNSNS/NS0.68 (0.65 to0.72)

NS/NSAge, hypertension, history of highblood glucose level, BMI

20.9Liu 201143

NSNSNS/NS0.71 (0.68 to0.74)

NS/NSAge, hypertension, history of highblood glucose level, BMI, fastingplasma glucose level

20.9Liu 201143

NSNS37.70/88.600.72 (0.69 to0.76)

64.5/71.6Age, hypertension, history of highblood glucose level, BMI, fastingplasma glucose level, triglyceridelevel, high density lipoproteincholesterol level

20.9Liu 201143

NSNSNS/NS0.7015/98Waist circumference, hypertension oruse of antihypertensive drugs, lowdensity lipoprotein cholesterol level,triglyceride level, BMI,hyperglycaemia

3.9Mainous 200760

27.7 inhighestrisk fifth

Hosmer-LemeshowP<0.001 beforecalibration, P>0.10after recalibration

NS/NS0.78 (0.74 to0.82)

NS/NSOverweight or obese, impaired fastingglucose, high density lipoproteincholesterol level, triglyceride level,hypertension, parental history ofdiabetes

8.4Mann 201019



NS/NS0.84 (0.82 to0.86)

NS/NSHeight, waist circumference, blackethnicity, systolic blood pressure,fasting plasma glucose level, highdensity lipoprotein cholesterol level,triglyceride level, parental history ofdiabetes, age

8.4Mann 201019



NS/NS0.83 (0.81 to0.85)

NS/NSAge, sex, Mexican-Americanethnicity, fasting plasma glucoselevel, systolic blood pressure, highdensity lipoprotein cholesterol level,BMI, family history of diabetes

8.4Mann 201019

NSNSNS/NS0.76 (0.70 to0.81) at 5-6years, 0.79(0.74 to 0.85)at 10 years

60 and 73.3 at 5-6years/64.9 and 78.4 at10 years

Age, sex, ethnicity, BMI, systolicblood pressure, fasting plasmaglucose level, high density lipoproteincholesterol level, family history ofdiabetes in first degree relative

9.7 at 5years 14.3at 10 years

McNeely 200361



RESEARCH



Table 2 (continued)



(%)AUROC(95% CI)


Diabetesincidence

(%)*Study

NSNSNS/NS0.843 (0.813to 0.874)

NS/NSImpaired fasting glucose, familyhistory of diabetes, impaired glucose

4.2Mehrabi 201044

tolerance, waist circumference,triglyceride level

NSNSNS/NS0.90 (0.88 to0.92)

NS/NSAge, sex, family history of diabetes,BMI, triglyceride level, fasting plasmaglucose level, systolic blood pressure,high density lipoprotein cholesterollevel (Framingham simple clinicalmodel)

9.2Meigs9

NSNSNS/NS0.68 (NS)NS/NSAge, sex, parental history of diabetes,BMI

16.5Nichols 200862

NSHosmer-LemeshowP<0.001

NS/NS0.82 (NS)NS/NSAge, sex, parental history of diabetes,BMI, hypertension or antihypertensivedrugs, high density lipoproteincholesterol level, triglyceride level,fasting plasma glucose level

16.5Nichols 200862

NSNSNS/NS0.84 (NS)NS/NSAge, sex, parental history of diabetes,BMI, systolic blood pressure, highdensity lipoprotein cholesterol level,triglyceride level, fasting plasmaglucose level, waist circumference

16.5Nichols 200862

20NSNS/NS0.74 (NS)54.5/80Age, sex, current use ofcorticosteroids, use ofantihypertensive drugs, family historyof diabetes, BMI, smoking

1.3Rahman 200863

NSHosmer-LemeshowP=0.66, Brier score0.0848

23.7/95.40.76 (0.71 to0.81)

69.2/74Age, sex, BMI, parental history ofdiabetes, smoking, hypertension

7.6Rathmann 201085


26.1/97.30.84 (0.80 to0.89)

82.4/72.9Age, sex, BMI, parental history ofdiabetes, smoking, hypertension,fasting plasma glucose level,haemoglobin A1c concentration, uricacid level

7.6Rathmann 201085


37.4/97.50.89 (0.85 to0.92)

81.3/84.1Age, sex, BMI, parental history ofdiabetes, smoking, hypertension,fasting plasma glucose level,haemoglobin A1c concentration, uricacid level, oral glucose tolerance test

7.6Rathmann 201085

NSHosmer-LemeshowNS/NSMen 0.77(0.76 to0.79),women 0.78(0.76 to 0.79)

NS/NSAge, ethnicity, BMI, hypertension,immigrant status, smoking, education,cardiovascular disease

7.1Rosella 201069



5.3Rosella 201069



4.2Rosella 201069

50NSRange25-32/range 88-93(at differentcut-offs)

0.71Range 40-77/55-84 (atdifferent cut-offs)

Age, waist circumference, height,systolic blood pressure, family historyof diabetes, ethnicity

16.3Schmidt 200546

50NSRange27-41/90-94 (atdifferent cut-offs)


Age, waist circumference, height,systolic blood pressure, family historyof diabetes, ethnicity, fasting plasmaglucose level

16.3Schmidt 200546



RESEARCH



Table 2 (continued)



(%)AUROC(95% CI)


Diabetesincidence

(%)*Study

50NSRange27-42/range 90-95


Age, ethnicity, waist circumference,height, systolic blood pressure, family

16.3Schmidt 200546

(at differentcut-offs)

history of diabetes, fasting plasmaglucose level, triglyceride level, highdensity lipoprotein cholesterol level

23.20Observed topredicted incidence

5.9, 7.7, 10.7 atdifferentcut-offs/NS

0.8483.1, 67.5, 50.3/68.3,80.6, 89.9 (at differentcut-offs)

Age, waist circumference, height,history of hypertension, physicalinactivity, smoking, consumption ofred meat, whole grain bread, coffee,and alcohol

3.1Schulze 2007 70

NSObserved topredicted incidence

NS/NS0.8294.4 ≥500 points, 79.7≥550 points/66.7 ≥500points, 79.3 ≥550 points

Age, waist circumference, height,history of hypertension, physicalinactivity, smoking, consumption ofred meat, whole grain bread, coffee,and alcohol

2.6Schulze 200770

NSHosmer-Lemeshowtests showed bettercalibration withhaemoglobin A1c orglucose included

NS/NS0.90 (0.89 to0.91)

NS/NSDiabetes risk score plus haemoglobinA1c concentration, glucose level,triglyceride level, high densitylipoprotein cholesterol level,γ-glutamyltransferase level, alanineaminotransferase level

3Schulze 200947

NSNSNS/NS0.76 (0.73 to0.79)

NS/NSAge, sex, use of antihypertensivedrugs, BMI, family history of diabetes,physical inactivity, diet (green leafyvegetables, fresh fruit, wholemealbread)

1.7Simmons 200771

NSNSNS/NS0.76 (0.73 to0.79)

NS/NSAge, sex, current use ofcorticosteroids, use ofantihypertensive drugs, family historyof diabetes, BMI, smoking

1.7Simmons 200771

12.8NS26.80/98.40NS75/88.5Fasting plasma glucose level, twohour postprandial plasma glucoselevel, BMI, high density lipoproteincholesterol level, pulse pressure

3.7Stern 199348

14.7NS25.20/98.10NS69.6/88.1Sex, fasting plasma glucose level,BMI, high density lipoproteincholesterol level, pulse pressure

3.7Stern 199348

NSHosmer-LemeshowP>0.2

NS/NS0.86 (0.84 to0.88)

NS/NSAge, sex, ethnicity, triglyceride level,total cholesterol level, low and highdensity lipoprotein cholesterol levels,fasting plasma glucose level, familyhistory of diabetes in first degreerelative, two hour postprandial plasma

6.0Stern 200286

glucose level, systolic and diastolicblood pressure, BMI

NSHosmer-LemeshowP>0.2

NS/NS0.84 (0.82 to0.87)

NS/NSAge, sex, ethnicity, fasting plasmaglucose level, systolic blood pressure,high density lipoprotein cholesterollevel, BMI, family history of diabetesin first degree relative

6/0Stern 200286

31.2Observed topredicted incidenceP=0.410

17.18/98.380.85 (0.83 to0.87)

72.3/82.8Age, sex, education, family history ofdiabetes, smoker, sport time, highblood pressure, BMI, waistcircumference, fasting plasmaglucose level

4.7Sun 200972

23.5NS13.54/98.470.8475.2/79.0Age, ethnicity, waist circumference,height, systolic blood pressure, familyhistory of diabetes, fasting plasmaglucose level

4.7Sun 200972



RESEARCH



Table 2 (continued)



(%)AUROC(95% CI)


Diabetesincidence

(%)*Study

22.7NS15.39/98.470.8475.0/79.7Age, ethnicity, waist circumference,height, systolic blood pressure, family

4.7Sun 200972

history of diabetes, fasting plasmaglucose level, triglyceride level, highdensity lipoprotein cholesterol level

19.2Hosmer-LemeshowP=0.77

NS/NS0.72 (0.69 to0.76)

NS/NSNS3.5Talmud 2010 10

26.6Hosmer-LemeshowP=0.42

NS/NS0.78 (0.75 to0.82)

NS/NSNS3.5Talmud 201010

NSObserved topredicted risk

NS/NS0.84 (NS)NS/NSLevels of adiponectin, C reactiveprotein, ferritin, glucose, haemoglobinA1c, interleukin 2, insulin

3.2Urdea 200964

NSObserved topredicted risk

NS/NS0.84 (NS)NS/NSLevels of adiponectin, C reactiveprotein, ferritin, glucose, haemoglobinA1c, interleukin 2, insulin

3.2Urdea 200964

NSNS16.7 at 80%specificity, 24.6 at90% specificity/NS

0.79 (0.78 to0.81)

69.5 (62.6-73.9) at 80%specificity, 57.0(49.8-64.0) at 90%specificity/set at 80%and 90%

Age, BMI, hypertension, glucose,family history of diabetes, highdensity lipoprotein cholesterol level

5.4Von Eckardstein200050

47Hosmer-LemeshowP=0.006

NS/NS0.77 (0.74 to0.79)

79.2 (top 40%) 50.3 (top20%)/61.8 (top 40%)81.4 (top 20%)

Age, sex, family history of diabetes,smoking status, BMI, waistcircumference, hypertension, recallof doctor diagnosed coronary heartdisease

4.3Wannamethee201127


NS/NS0.82 (0.79 to0.84)

84.2 (top 40%), 63.8 (top20%)/62% (top 40%) 82(top 20%)

Age, sex, family history of diabetes,fasting plasma glucose level, smokingstatus, BMI, waist circumference,hypertension, recall of doctordiagnosed coronary heart disease,high density lipoprotein cholesterollevel, triglyceride level



NS/NS0.81 (0.79 to0.83)

85.1 (top 40%), 62% (top20%)/62.1 (top 40%),82% (top 20%)

Age, sex, family history of diabetes,smoking, BMI, waist circumference,hypertension, recall of doctordiagnosed coronary heart disease,high density lipoprotein cholesterollevel, γ-glutamyltransferase level,,haemoglobin A1c concentration


10.8NSNS/NS0.60 (0.56 to0.64) at 20years

35.6/75.7 (both at 20years)

NS5.8Wannamethee200565

15.6NSNS/NS0.85 (NS)NS/NSFasting plasma glucose level, BMI,high density lipoprotein cholesterollevel, parental history of diabetes,triglyceride level, blood pressure

5.1Wilson 200751

NS=not stated; BMI=body mass index.*Incidence of diabetes was measured differently by different authors, such as annually, every five years, every 10 years, or per 1000 patient years.†Sensitivity and specificity are based on authors’ preferred cut-off score.



RESEARCH



Table 3| Summary of authors’ assumptions and claims about their diabetes risk models or scores

Citationtracking

Data inpaper on

Authors’ statedconcerns abouttheir risk score

Authors’ claimsfor risk scoreover others

Authors’adjectives todescribe theirrisk score

Mechanismby whichuse of riskscore may

Authors’ assumptions

Study

What will beoffered topeople whoscore abovecut-off for“caseness”

Who will use riskscore, on whichsubgroups orpopulations

(GoogleScholar)

forstudiesof realworlduse

use ofrisk

score inrealworld

improveoutcome

64citations,notrelevant

Validatedon anothercohort insamefactory

Generalisability hasnot been shownbeyond Thaipopulation

“Almost as good as”and less expensivethan models thatrely on blood tests

Simple, “apractical tool,”low tech, no labtests,non-invasive

ClinicalFasting plasmaglucose test,“health educationand theopportunity toengage inhealthy lifestyles”

“Primary health care”will use score on“individuals who arelikely to developdiabetes”

Aekplakorn 20067

0NoneOnly predicts gettingdiabetes, does notpredict complications

NS“Pretty good”Clinical, publichealth

Blood test,preventivemanagementaccording toprotocol

General practitioners,for use on high riskpatients. Public healthclinicians, for use onhigh risk populations

Alssema 200852

1 citation,notrelevant

NoneSome missing data indataset

Better discriminationUpdated,refined, simple

Clinical, publichealth

Blood test,“integratedstrategies”(addressing riskof cardiovasculardisease as well)

Intended users notstated. Refinedprevious risk score

Alssema 201153


None2 hour glucose levelrarely used in practice

Better area underreceiver operatingcharacteristic curve,simple (requires 3variables for men, 4for women)

SimpleNonespecificallyhypothesised

Focuses onpopulation level,not clinical careof high riskpeople

Implicit target audienceepidemiologists andpopulation geneticists

Balkau 200836


NoneSample may not berepresentative (too“urban”)

Better discriminationcapacity, developedon large cohort

Simple,parsimonious

Clinical“Intensivediabetespreventioninterventions”

Clinical (“targetedinterventions”) andpublic health (“efficientallocation ofresources”)

Bozorgmanesh201054

2 citations,notrelevant

NoneNSBetter discriminationcapacity, developedon large cohort

Simple,superior,pragmatic,parsimonious,comprehensive

ClinicalNSClinicians in Iran andother Middle Easterncountries; unselectedMiddle Easternpopulation

Bozorgmanesh201166

0NAResponse 65%; shortfollow-up, predictivevalue reduces withtime

Likely to beacceptable topatients and doctors

Simple, clinical,parsimonious

ClinicalFormal test fordiabetes, forexample, oralglucosetolerance test,plus

Clinical practice (“to beordinarily available in aroutine clinicalsetting”), MiddleEastern countries

Bozorgmanesh201055

“Individualisedprimaryprevention”


NAAuthors unconvincedthat it adds value

NANo better atpredictingdiabetes thanrandom bloodglucose level

ClinicalImplicitly, generalpopulation(Australians).“Lifestylemeasures”

Intended users notstated. Does notconsider how scoreswill be used

Cameron 200856

6 citations,of whichone wasan impactstudy

Validatedon secondpopulationas part ofthis study

Developed on narrowage band hence agenot very significant infinal model

Betterdiscrimination,easier to measure(for example, waistcircumferencemorepracticable thanBMI for lay people)

Simple,non-invasive

Lay people“Interventions toprevent or delay[diabetes] onset”

Not stated but scorehas been converted toan online tool for selfassessment of risk bylay people

Chen 201037


NoneAUROC only 70%,diabetes not excludedat baseline

First to be validatedin Chinese (butothers claim thistoo)

SimpleClinical“Preventive andtreatmentstrategies”

“Clinical practice”(Chinese population)

Chien 200967



RESEARCH



Table 3 (continued)

Citationtracking(GoogleScholar)


Data inpaper onuse ofrisk

score inrealworld




Mechanismby whichuse of riskscore mayimproveoutcome


Study



0NoneNoneMenu of scores(some simple, some

SimpleClinicalNS“Clinical professionalsand general subjects,”

Chuang 201138

more complex withfor use in “middle agedbetterChinese adults living in

Taiwan” discrimination);large validationcohort

0NA (nottheir riskscore)

NoneValidated by anindependent teamon an independentcohort (unlike mostothers)

UsefulPublic healthNSImplicitly,epidemiologists andpublic health clinicians,for use in UKpopulation

Collins 201157

0NoneOnly moderately goodpredictive power(AUROC 71%)

Simple, usesabsolute risk, basedon prospectivecohort

SimpleLay peopleNS“To be used bylaypersons” to detectdiabetes and raiseawareness,“particularly in low-income countries”

Gao 200939

0NoneNot shown to be costeffective or to improvequality of life, needsexternal validation

Statistically betterthan other scoresfor use on a LatinAmericanpopulation

Quick and easyto use, fewlaboratoryinvestigations,cheap

Implicitly,clinical

Blood test,monitoring ofrisk, preventiveinterventiontargetingparticular riskfactors

Intended users notstated. For use onunselected LatinAmerican population

Guerrero-Romero201058


None, butauthorsemphasisethat itcould beused easily

Missing values (forexample, smoking,ethnicity); internalvalidation on EMISonly; better designwould be a

Includes deprivationand ethnicity, basedon data fromgeneral practicerecord, goodstatistical

Simple, gooddiscrimination,well calibrated,readilyimplementablein primary care,cost effective

Clinical“To identify andproactivelyintervene”

General practice andpublic health in areasof high socioeconomicand ethnic diversity;use in “clinical settings”and by lay publicthrough a “simple webcalculator”

Hippisley-Cox20098

prospective study ofinception cohort

properties, wellvalidated, “likely toreduce . . . healthinequalities”

0NoneNone mentionedMorecomprehensive,AUROC 0.85,longer follow-up,less bias (forexample, in how

NSNonespecificallyhypothesised

“Lifestyle adviceadvocatingphysical activity,healthy low fatdiet, and weightreduction”

Implicitly,epidemiologists (focusof paper isidentification andrefinement of riskfactors in a population)

Joseph 201040

incident diabeteswas diagnosed)


NoneLimited to age 45-65and to white or blackethnic groups

Prospectivelyvalidated, mayilluminate cause ofdiabetes bydemonstrating newassociations

Low cost,clinical, simple


“Preventiveinterventions”

“Insurers or publichealth agencies . . . tooptimise allocation ofpreventive medicineresources”

Kahn 200941

0NoneNeeds validating in alongitudinal study

Very simple,validated in severalsamples

SimpleClinical“Lifestylemodification”

To identify “olderpersons who shouldreceive intensivelifestyle intervention”

Kanaya 200559



RESEARCH



Table 3 (continued)




score inrealworld






Study



29citations,

NoneDeveloped inoverweight middle

Biologicallyplausible

Objective,quantitative

Nonespecificallyhypothesised

“for whom themostcomprehensive

For use on “individualsat highest risk ofdeveloping type 2diabetes”

Kolberg 200942

notrelevant

aged white people,hence transferabilitymay be limited

(“multi-biomarker”),convenient, fewerlogistical challenges

preventionstrategies shouldbe considered” to implementation,

better discrimination

343citations,of whicheightdescribedimpactstudies

Not in thispaper, butsee citationtrack

Possible circularargument—identifyingpeople based on samerisk factors that wouldhave prompted theirclinician to measurerandom blood glucoselevel in the first place

Prospective, largecohort. “The publichealth implicationsof the Diabetes RiskScore areconsiderable”

Simple,practical,informative,fast,non-invasive,inexpensive,reliable, safe

Clinical, laypeople

“Direct attentionto modifiable riskfactors.” Also,doing one’s ownrisk score mightprompt people tomodify theirlifestyle and

Intended users notstated. Implicitly, thosewho (like the authors)seek to undertakeintervention studies ofdiabetes prevention.For use with “thegeneral public”

Lindstrom 200368

prompt them toget their bloodglucose levelchecked

0NoneValidated in middleaged to older cohortso unproved benefit inyounger people. Didnot include familyhistory of diabetes, asnot on database

Validated on amainland Chinesepopulation, largecohort, prospective,stable predictionmodel

Practical,effective,simple, easilyused in clinicalpractice

ClinicalOral glucosetolerance test,education,“opportunity toengage inhealthy lifestylesat an early stage”

Clinicians. “initialinstrument foropportunistic screeningin general population”,“could enhancepeople’s awareness”

Liu 201143


NonePoor discriminatoryability

NANA (they don’trecommend it inthis group)

Clinical“Earlyrecognition andtreatment”

Implicitly, clinicians.Paper describesvalidation of a previousrisk score in a youngercohort

Mainous 200760


NoneInability to isolateMexicans

Recalibration andrevalidation ofFramingham basedscore in largeethnically diversepopulation

Highdiscriminativeability


NS“Clinicians . . . tostratify their patientpopulations”

Mann 201019


None“Further refinementsthat take into accountthe differential effectsof age are needed”

Better in short termthan fasting bloodglucose test but notin long term(younger people).Not as good as oral

None, all dataexpressed innumbers


NS“Clinical practice.” Topredict diabetes risk inJapanese Americans

McNeely 200361

glucose tolerancetest (older people)

0NoneNew and relativelyuntested, somemissing data

Higher predictabilityrate than use ofsingle risk factorsalone

Useful, novelNotspecificallyhypothesised

NSNSMehrabi 201044

163citations,but notrelevantas papercited for its

NADid not help to refinethe prediction ofdiabetes risk

NALess usefulthan datacollected at aroutine clinicalexamination

NA (authorssuggestfurtherresearch onkeysubgroups)

NANA—negative studyshowing that geneticfactors add nothing toclinical scores

Meigs 20089

negativefindings



RESEARCH



Table 3 (continued)




score inrealworld






Study




NoneIf health maintenanceorganisationpopulation has

Better AUROC“Extremelyaccurate,”simple

Clinical, publichealth,technology

“Interventions”and targeting ofhealthcareresources

Health maintenanceorganisations. Basedon analysis ofelectronic record data,

Nichols 200862

different incidence ofto identify members at type 2 diabetes fromhigh risk of developingdiabetes

validation cohort,score will beinaccurate


NoneWill need to bevalidated in otherprospective cohorts

Based on dataroutinely availableon general practicerecords

Simple,effective


Not explicitlystated butauthors suggestpotentialavenues forimpact studies

Primary care andpublic health clinicians.Use for “definingindividuals andpopulations for testing,treatment andprevention”

Rahman 200863


NoneNo external validationyet

Validated in olderpopulation

SimplePublic health“Preventivestrategies”

Intended users notstated. Use “to identifyhigh-risk populationsfor preventivestrategies”

Rathmann 201085


NoneCould be further testedon other populations.Family history andpoor diet not collected,relies on self reports

Uses data availableon populationregistries

SimplePublic health,clinical

“Newinterventionstrategies”

Public health cliniciansand health planners “toestimate diabetesincidence, to stratifythe population by risk,and quantify the effectof interventions”

Rosella 201069


NoneHigh losses tofollow-up, oral glucosetolerance test not doneat baseline

Good predictor forwhite andAfrican-Americanmen and women;may apply also toother ethnic groupsin United States

Simple, basedon readilyavailableclinicalinformation andsimplelaboratory tests

Clinical, publichealth,research

“Preventiveactions ofappropriateintensity”

Use “in clinicalencounters,” “bymanaged careorganizations . . . toidentify high-riskindividuals,” and toenrol to clinical trials

Schmidt 200546


NoneSelf reports may havebeen biased

Good AUROC(0.84), usedabsolute values forage rather thanbroad categories

Precise,non-invasive,accurate, useful

The publicNot explicitlystated

Intended users notstated. “Identifyingindividuals at high riskof developing T2D[type 2 diabetes] in thegeneral population”

Schulze 200770


NonePredictive for onset ofdiabetes in middle agebut not from birth,since diabetes wasexcluded frominception cohort

“A comprehensivebasic model,”significantlyimproved by routineblood tests but notchemical or geneticbiomarkers

Improveddiscrimination


NSNSSchulze 200947


Feasible tocollect

No better thanstandard clinicaldataset routinelycollected in UKgeneral practice (butmay be feasible inother health settings)

Relies only onsimple questionsabout lifestyle,which would beasked in a routinehealth check.AUROC (0.76) is as

Simple, feasibleClinical,administrative

“Could beincorporated intonew patienthealth checksand may providea more feasiblemeans of

Primary care: “couldinform . . . healthbehaviour information. . . routinely collectedin GP consultations orby administrative staff,”identify groups fortargeted prevention

Simmons 200771

good as manycomplex risk scores

identifying thoseat risk thanOGTT [oralglucosetolerance test], orselect those



RESEARCH



Table 3 (continued)




score inrealworld






Study



suitable forOGTT”


NoneNSUses commonlymeasured clinicalvariables

Predictive,multivariate

Research,clinical

“Identifyinghigh-risk cohortsfor preventiontrials”

Implicitly,epidemiologicalresearchers

Stern 199348


NonePossible missing dataLess expensive andmore convenientthan oral glucosetolerance testing

SimpleClinical, publichealth,technological,research

Clinical: “patientcounselling.”Public health: “toidentify targetpopulations forpreventiveinterventions”

“Could be incorporatedas it stands into clinicalpractice and publichealth practice with theaid of a calculator orpersonal computer”

Stern 200286


NoneLosses to follow-up,oral glucose tolerancetest not done atbaseline so somecases detected,especially early on,may be prevalent ones

Simple, uses readilyavailable clinicalinformation

Simple,effective,accurate

Clinical,technological,research

Further researchUse in clinicalencounter, bymanaged careorganisations toidentify high riskpeople, and to enrol toclinical trials

Sun 200972


NoneNASimple clinical riskscores performedmuch better thanassessment ofgenetic risk from 40polymorphisms

NA(revalidation)

Notspecificallyhypothesised

NSIntended users notstated (but study usedan existing risk scoreas a “control” fortesting a geneticprofile)

Talmud 201010


NoneNS“Better than anyother clinicalmeasure”, notover-fit, based onmultiple biomarkershence highlyplausible

Simple,accurate,convenient

Clinical“so that clinicianscan implementan effectivediabetespreventionprogram”

“Current clinicalpractice”; for“identifying individualsat highest risk ofdeveloping T2DM [type2 diabetes mellitus]”

Urdea 200964


NANegative studyNANA (negativestudy)

NANA (negativestudy)

NA (negative study, nobetter than fastingblood glucose testalone in this cohort)

Von Eckardstein200050


NoneNS“Useful predictor”(but not as good asFramingham score)

NA (lesseffective thanFraminghamrisk score)

Notspecificallyhypothesised

Not stated, unitof analysis is thepopulation

Intended users notstated

Wannamethee201127

0NoneDiabetes diagnosed byself reports

StepwiseSimple, routineNotspecificallyhypothesised

Blood testsIntended users notstated

Wannamethee200565


NoneNSVery good AUROC(85%)

Simple,effective, easy

ClinicalImplicitly, lifestyleadvice andmetformin

Implicitly, cliniciansWilson 200751

NS=not stated; NA=not applicable; BMI=body mass index; AUROC=area under receiver operating characteristic curve.



RESEARCH



Table 4| Components of seven diabetes risk models or scores with potential for adaptation for use in routine clinical practice

External validation

CalibrationAUROCRisk factors included in scoreScore/study name,country, reference CalibrationAUROCYear, country

Hosmer-LemeshowP<0.001, afterrecalibration P>0.10

0.842010,19 USANS0.80Age, ethnicity, waist circumference, height,systolic blood pressure, family history ofdiabetes, fasting plasma glucose levels,triglyceride levels, high density lipoproteincholesterol levels

ARIC (AtherosclerosisRisk in Communities),Germany, Schmidt200546

Not externally validated but has been studied as part of anintervention to improve outcomes87

Hosmer-LemeshowP=0.85

0.78Age, sex, ethnicity, parental history ofdiabetes, history of high blood glucose,use of antihypertensive drugs, smoking,physical inactivity, waist circumference

Ausdrisk, Australia,Chen 201037


0.722010,10 UK*NS0.74 withthreshold of

0.38

Age, sex, use of current corticosteroids,use of antihypertensive drugs, familyhistory of diabetes, body mass index,smoking

Cambridge risk score,UK, Rahman 200863


0.762010,53 Holland,Denmark,Sweden, UK,Australia*

NS0.85Age, body mass index, waistcircumference, use of antihypertensivedrugs, history of high blood glucose,physical inactivity, daily consumption ofvegetables, fruits, and berries

FINDRISC, Finland,Lindstrom 200368

Hosmer-LemeshowP<0.001, afterrecalibration P>0.10

0.782010,19 USANS0.85Fasting plasma glucose levels, body massindex, high density lipoprotein cholesterollevels, parental history of diabetes,triglyceride levels, blood pressure

Framingham OffspringStudy, USA, Wilson200751

Hosmer-LemeshowP<0.001, afterrecalibration P>0.10;Hosmer-LemeshowP≤0.001, afterrecalibration P=0.131;

0.83; 0.83;0.78; 0.78

2010,19 USA;2010,55 Iran*;2010,10 UK*;2010,66 Iran*

Hosmer-LemeshowP>0.2

0.84Age, sex, ethnicity, fasting plasma glucoselevels, systolic blood pressure, highdensity lipoprotein cholesterol levels, bodymass index, family history of diabetes infirst degree relative

San Antonio risk score,clinical model, USA,Stern 200249

Hosmer-LemeshowP=0.42;Hosmer-LemeshowP=0.264

Brier score: 0.053 men,0.041 women

0.80 men,0.81 women

2011,57 UKBrier score: 0.078men, 0.058 women

0.83 men,0.85

women

Age, sex, ethnicity, body mass index,smoking, family history of diabetes,Townsend deprivation score, treatedhypertension, cardiovascular disease,current use of corticosteroids

QDScore, UK,Hippisley-Cox 20098

AUROC=area under receiver operating characteristic curve; NS=not stated.*Validation used more, less, or substituted risk factors from original risk score or did not state the exact factors it used. See table 2 for further details.



RESEARCH



Table 5| Results of impact citation search (studies using diabetes risk models or scores as part of an intervention to improve outcomes)

CommentMain findings or expectedreporting date

Study design,intervention

Setting andsample

Research questionScore usedStudy(acronym)

Changes only reported on“completers”; those lost tofollow-up were not included inanalysis. Absolute changes weresmall and probably not clinicallysignificant—for example, mean 1

271/352 completed study.Showed statisticallysignificant reduction inweight, body mass index,and total cholesterol level,maintained at 36 months

Real world feasibilitystudy: eight lifestylecounselling sessions

Australia, 352high risk adults

Can diabetes risk bereduced by lifestylecounselling?

FINDRISC68Absetz 2009(GOAL study)88

kg weight loss. Change inFINDRISC score was not reported

Some but not all peopleencouraged to change lifestyle willachieve it, but most will struggle

Many found dietary changedifficult and stressful; somewho did not achieve weightloss felt despondent

Focus groups withweight losers and weightgainers studiedseparately

Australia, 30weight losersand 30 weightgainers fromGOAL study

What is theexperience of lifestylechange in peoplerecruited into diabetesprevention studies?

FINDRISC68Jallinoja 2008(GOAL study)89

Participants will be recruited inprimary care, but intervention willbe delivered as a publichealth/community basedprogramme

Results expected 2013.Main outcomes will bechange in weight, physicalactivity, diet, fasting glucoselevels, blood pressure, lipidlevels, quality of life, andhealth service utilisation

Real world feasibilitystudy: individualassessment followed bygroup sessions

Australia, 1550high risk adults(100 indigenouspeople)

Can diabetes risk bereduced by aprogramme ofintensive behaviourchange?

AUSD-RISK37Colaguiri 2010(Sydney DPP)87

Weight loss in intervention groupwas clinically significant (3.8 kg);fasting glucose in the control groupincreased, whereas that in theintervention group decreased.However, follow-up was short

Statistically significantchanges in weight, physicalactivity, diet, and fastingglucose levels at 12 monthscompared with controls

Randomised trial.Intervention groupreceived 12 grouplessons in lifestylemodification, controlshad leaflet

Germany, 182high risk adults

Can diabetes risk bereduced by lessons inlifestyle modification?

FINDRISC68Kulzer 2009(PREDIAS)90

Mean weight loss 2.52 kg. Authorsview findings as “convincingevidence that a type 2 diabetesprevention programme usinglifestyle intervention is feasible inAustralian primary health care with

Statistically significantimprovements in weight,fasting and two hourglucose levels, and lipidlevels at 12 months

Real world feasibilitystudy: six sessions ofnurse led groupeducation

Australia, 237high risk adults

Can risk factorreduction be achievedin a high risk non-trialpopulation?

FINDRISC68Laatikainen2007(GGTDPP)91

reductions in risk factorsapproaching those observed inrandomised controlled trials”

Authors report that “certainproblems and challenges wereencountered, especially in relationto the limited resources allotted topreventive health-care.”74A smallerongoing prevention programme

Preliminary results only.Numbers and detailedfindings not given.“Desirable changes” at 12months in risk factors andglucose tolerance in high

High scorers onFINDRISC had oralglucose tolerance andlipid levels tested; thosewithout diabetes wereoffered nurse led

Finland, highrisk adults (partof a nationaldiabetespreventionprogramme that

Can a populationapproach detect highrisk people, modifytheir risk througheducationalintervention, and

FINDRISC68Saaristo 2007(FIN-D2D)80 andLindstrom 2010(FIN-D2D)74

using FINDRISC along withrisk cohort. Incidentcommunity basedalso includedthereby reduce theoccupational health screening ondiabetes reduced (asindividual or grouppopulation

component)incidence of newdiabetes? an occupational cohort in an airlinemeasured by drugsessions, or both, based

company (FINNAIR diabetesreimbursement registrationon stages of change andprevention study) is also brieflyoutlined in Lindstrom paper74

data). Full results expected2012-13

tailored to individualprofile

Authors recognise that preventionon a large scale sits oddly withinthe existing treatment orientedhealth system. Key features ofTUMANI are prevention managersworking within the existing

Results expected 2012-13High scorers onFINDRISC had oralglucose tolerance testbefore being assigned a“preventionmanager” foreducation, support, andtelephone counselling

Germany, highrisk adults (partof a nationalpreventionprogramme)

Can an intensive,multifaceted publichealth interventionprevent incidentdiabetes in high riskpeople?

FINDRISC68Schwarz 2007(TUMANI)59

infrastructure, a structured qualitycontrol programme, and apopulation component—forexample, website and links tomass media

Findings to date suggest that halfof high risk patients were willing tofill out the FINDRISC questionnaireand follow-up with their generalpractitioner. Response rates toquestionnaire varied significantlyamong practices

16 032 people were mailed;response rate toquestionnaire 54.6%, ofwhich 17.5%were classifiedas high risk. Of these,73.1% booked aconsultation with their

General practitionersmailed questionnaires totheir adult patients. Highscorers were offeredoral glucose tolerancetest

Netherlands, 48generalpractices

Can a mailedquestionnaire fromgeneral practiceidentify high riskpeople to participatein a preventiveintervention?

FINDRISC68Vermunt 2010(APHRODITE)92

general practitioner. Fullresults expected 2014



RESEARCH



Figures

Fig 1 Flow of studies through review

Fig 2 Publication of diabetes risk models and scores 1990-2010. Eleven new risk models and scores had been publishedin the first five months of 2011



RESEARCH



Risk models and scores for type 2 diabetes: systematic review

Documents