Article Text
PDF
Abstract
Background Information from well-established dementia risk models can guide targeted intervention to prevent dementia, in addition to the main purpose of quantifying the probability of developing dementia in the future.
Methods We conducted a systematic review of published studies on existing dementia risk models. The models were assessed by sensitivity, specificity and area under the curve (AUC) from receiver operating characteristic analysis.
Results Of 8462 studies reviewed, 61 articles describing dementia risk models were identified, with the majority of the articles modelling late life risk (n=39), followed by those modelling prediction of mild cognitive impairment to Alzheimer’s disease (n=15), mid-life risk (n=4) and patients with diabetes (n=3). Age, sex, education, Mini Mental State Examination, the Consortium to Establish a Registry for Alzheimer’s Disease neuropsychological assessment battery, Alzheimer’s Disease Assessment Scale-cognitive subscale, body mass index, alcohol intake and genetic variables are the most common predictors included in the models. Most risk models had moderate-to-high predictive ability (AUC>0.70). The highest AUC value (0.932) was produced from a risk model developed for patients with mild cognitive impairment.
Conclusion The predictive ability of existing dementia risk models is acceptable. Population-specific dementia risk models are necessary for populations and subpopulations with different characteristics.
- dementia
- alzheimer’s disease
- risk model
- prediction
- systematic review
Statistics from Altmetric.com
Introduction
Dementia is a significant public health problem associated with disability, institutionalisation and mortality among elderly individuals. The prevalence of dementia in the world has been reported to be approximately 5%–7% among people aged 60 years and older.1 Accurate identification of individuals at high risk of dementia is very important for early diagnosis and intervention, such as close monitoring, improved care and risk factor-targeted intervention.2 Various risk models for identifying individuals who are at high risk of dementia or Alzheimer’s disease (AD) have been developed with different sets of known risk factors. A systematic review about dementia prediction models was published in 2010,2 but none of the reported models are universally accepted with high predictive accuracy. While research on dementia risk prediction has been developing rapidly in the past decade, new risk factors and biomarkers were identified to be associated with dementia or AD, and subsequently incorporated into newly constructed models to increase the predictive accuracy.3 Besides, the Alzheimer’s Disease Neuroimaging Initiative (ADNI) was initiated in 2003 to test whether serial MRI, positron emission tomography, the biomarkers and clinical and neuropsychological assessments can be combined to accurately describe the natural history and measure trajectories of AD progression.4 Many models were constructed using the data from the ADNI, especially the models for predicting mild cognitive impairment (MCI) to AD conversion.5–8
In this review, we assessed the predictive ability of the latest risk models using the sensitivity, specificity and the area under the curve (AUC) from receiver operating characteristics (ROC) analysis of the models based on the results of an updated literature search. AUC values of 0.9–1, 0.7–0.9 and <0.7 are considered to represent high, moderate and low accuracy.
Methods
Search strategy
We searched PubMed for articles published from 1995 to 1 April 2018 using combinations of the following terms: “dementia”, “Alzheimer disease”, “Alzheimer and disease”, “predict*”, “develop*”, “incident”, “sensitivity”, “specificity”, “ROC”, “c statistic”, “AUC” and “area under the curve”. Bibliographies of eligible studies were hand-searched for potential missing studies. Only articles published in English were considered for review.
Selection criteria
Articles were included if they simultaneously met the following criteria: (1) the sample was population-based or restricted to patients with MCI; (2) the article provided a risk model to predict dementia in non-demented individuals, prospectively; and (3) the article included measurements of sensitivity, specificity or AUC (or c-statistic).
Quality evaluation
The study quality was evaluated on selection, comparability and outcome with the Newcastle-Ottawa Quality Assessment Scale. Items describing a non-intervention cohort were excluded and therefore the total ranking was out of six (rather than nine). Only studies with more than 4 points were included in our review. Overall, 15 articles scored 6 points, 31 scored 5 points and 15 scored 4 points.
Results
Included studies
Figure 1 shows the results of literature searching and selection. A total of 8462 articles were found, and 141 were selected by title and abstract screening based on our inclusion and exclusion criteria. Another nine potential papers were further identified from the reference of relevant reviews. Finally, 89 were further excluded and 61 articles describing dementia risk models were included in this review.
Flow diagram of the literature search and studies selection process: flow of articles from initial database search to final selection whereby 8462 articles were reduced to a final 61 that were included in this review. AD, Alzheimer’s disease; MCI, mild cognitive impairment.
Description of studies included
The study characteristics and predictive models are summarised in online supplementary tables 1-4. Sample sizes of the studies ranged from 40 to 930 395. Follow-up duration ranged from 1 year to 39.1 years (figure 2). The number of final predictors incorporated into the models reported in these studies ranged from 1 to 224. The common variables used in dementia risk prediction are summarised in online supplementary table 5, including demographics, subjective cognitive complaints, cognitive test scores, lifestyle and health-related variables (figure 3). In four studies, the models were developed for middle-aged individuals. In 39 studies, the models were developed for elderly individuals. In 15 studies, the models were developed for predicting MCI to AD conversion, and dementia risk models were developed for patients with diabetes in 3 studies.
Supplemental material
Supplemental material
Supplemental material
Supplemental material
Supplemental material
Comparison of AUCs, numbers of variables and follow-up length in different studies. The length of the bars on the left represents the AUC values of the models in different studies. The length of the bars on the right represents the follow-up years in different studies. The details of the studies are available in online supplementary tables 1 and 4. *The follow-up length was not available or not applicable. AD, Alzheimer’s disease; AUC, area under the curve; MCI, mild cognitive impairment.
Most common variables comprised in the dementia risk models. The thicknesses of the lines represent the frequency of the variables used to construct the models. 3MS, The Modified Mini-Mental State; AD, Alzheimer’s disease; ADAS-Cog, Alzheimer’s Disease Assessment Scale-cognitive subscale; BMI, body mass index; CASI, Cognitive Abilities Screening Instrument; CERAD, Consortium to Establish a Registry for Alzheimer’s Disease; FCSRT-FR, free recall score from the Free and Cued Selective Reminding Test; IST, Isaacs Set Test; MCI, mild cognitive impairment; MMSE, Mini Mental State Examination; RAVLT, Rey Auditory-Verbal Learning Test; SNP, single nucleotide polymorphism; TMT-B, Trail Making Test (Part B); WMS, Wechsler Memory Scale.
Mid-life risk models for the general population
Four studies reporting mid-life risk models for the general population were included in our review. Sample sizes ranged from 14099 to 948010. Follow-up period ranged from 16.111 to 39.112 years.
CAIDE risk score
The Cardiovascular Risk Factors, Aging and Dementia (CAIDE) risk score comprising age, sex, education, cholesterol level, body mass index (BMI) and systolic blood pressure was developed for individuals in their middle age.9 This model was validated externally using retrospective cohort data from the Kaiser Permanente.10 The AUCs were similar between the two studies (0.75 in the Kaiser Permanente cohort vs 0.78 in the original cohort). Furthermore, the CAIDE risk score also performed well across different ethnicities (AUC 0.81 for Asian, 0.75 for black, 0.74 for white). However, the accuracy of the CAIDE risk score in predicting dementia or AD was poor in older-aged cohorts (mean age range 72.3–82.5 years).13 The AUCs ranged from 0.491 to 0.568 for dementia, and from 0.488 to 0.570 for AD.
Middle age self-report risk score
The middle age self-report risk score was developed with factors including age, education, work status, nature of work, work environment and the physicality of work in individuals with a mean age of 46.7 years.12 The AUC for this model was 0.77. It is worth pointing out that the diagnostic classification of dementia in this study was based on TELE, a telephone assessment for dementia, and telephone interview for cognitive status, rather than clinical assessments or established diagnostic criteria.
Another mid-life risk model
A model comprising sex, total cholesterol, systolic blood pressure, BMI, education level and number of APOE ε4 alleles showed moderate predictive ability (AUC 0.704 for AD in 10 years, 0.655 for AD in 18 years, 0.674 for dementia in 10 years, 0.644 for dementia in 18 years).11 The 10-year predictive ability for AD improved slightly when the blood level of N-terminal pro-brain natriuretic peptide was added to the model (AUC 0.724).
Late life risk models for the general population
Thirty-nine studies reporting late life risk models for the general population were included in our review. Sample sizes ranged from 18714 to 930 39515. Follow-up period ranged from 1.416 to 3017 years.
Cognitive test-based models
Cognitive test scores or subjective cognitive complaints were incorporated as predictor variables into 36 studies. Among those studies, 17 studies constructed risk models with neuropsychological test batteries or single cognitive test only, or combined with demographics. The AUCs ranged from 0.4918 to 0.9219. The Mini Mental State Examination (MMSE) is the most common cognitive variable. The models with MMSE score only had moderate predictive accuracy across cohorts. The AUCs ranged from 0.6814 to 0.8220. But the total MMSE score is associated less strongly with dementia and AD than the episodic memory subset.21 The free recall score from the Free and Cued Selective Reminding Test (FCSRT-FR) was used to predict dementia in three studies.19 22 23 The AUCs ranged from 0.81 to 0.88 in 3 years’ follow-up. The predictive performances of the Cognitive Abilities Screening Instrument,18 Clock Drawing Test,16 Cambridge Cognitive Examination24 and the total score of the Consortium to Establish a Registry for Alzheimer’s Disease neuropsychological assessment battery20 were also tested, and these models showed moderate accuracy (AUC ranged from 0.74 to 0.89). But none of them have been validated externally.
Late life risk models for primary care patients
Four risk models were reported in three studies15 25 26 based on information that can be obtained in the primary care setting. The Study on Aging, Cognition and Dementia (AgeCoDe) score comprising age, subjective memory impairment, delayed verbal recall, verbal fluency, MMSE and instrumental activities of daily living was developed for primary care patients.25 The model predicted AD in individuals aged ≥75 with 79.6% sensitivity, 66.4% specificity and an AUC of 0.84. Risk models for two age groups (60–79 and 80–95 years) were developed and validated using routinely collected data.15 The model for the cohort aged 60–79 years incorporated 14 variables. It performed well (AUC 0.84) when applied to the validation cohort. The model for the cohort aged 80–95 years incorporated 17 variables. However, the model performed poorly in the validation cohort with a low AUC of 0.56. The predictive ability of a model including age, education level, stroke, diabetes mellitus, BMI, depressive symptoms, and requiring assistance with money or medications was tested in four large cohorts (the Cardiovascular Health Study (CHS), the Framingham Heart Study (FHS), the Health and Retirement Study (HRS) and the Sacramento Area Latino Study on Aging (SALSA)).26 The accuracy for predicting 6-year incident dementia was low to moderate across the cohorts (AUC: CHS, 0.68; FHS, 0.77; HRS, 0.76; SALSA, 0.78).
SLAS risk score
The Singapore Longitudinal Ageing Study (SLAS) risk score comprising age, gender, education, depression, heart disease, social and productive activities, and MMSE score was developed to predict neurocognitive disorder (NCD) for individuals aged >55 years at baseline.27 This model predicted the risk of incident NCD in elderly participants moderately well in a 5-year follow-up (AUC 0.72).
Depressive symptom model
The association between specific symptoms of depression and later development of possible or probable Alzheimer’s dementia was investigated.28 Among eight symptoms of depression (depressed mood, loss of interest, change of appetite, sleep disturbance, psychomotor change, loss of energy, worthlessness, concentration difficulty), only loss of interest was significantly associated with later occurrence of AD. The depressive symptom model was then constructed with the symptom of loss of interest, APOE ε4, folic acid and education in individuals aged 75 years and older, and the AUC was 0.626.
Frailty index of non-traditional risk factors
The frailty index was constructed with 19 variables of health deficits not known to predict AD and dementia, such as eye trouble, chest problem and so on.29 The model exhibited relatively low predictive accuracy in participants of the Canadian Study of Health and Aging (AUC 0.64 for AD within 5 years, 0.66 for AD within 10 years, 0.64 for dementia within 5 years, 0.66 for dementia within 10 years).
Genetic risk models
A model was constructed with APOE ε4, APOE ε2, age, gender, polygenic score based on 20 genome-wide significant single nucleotide polymorphism (SNP) proxies, and polygenic score calculated using SNPs with AD association p values ranging from 0.0001 to 0.9 in the Cohorts for Heart and Aging Research in Genomic Epidemiology, European Alzheimer’s Disease Initiative and Alzheimer’s Disease Genetics Consortium sample.30 The best model showed moderate prediction accuracy (AUC 0.782).
Another genetic risk score including 10 risk genes (CLU, PICALM, BIN1, CR1, ABCA7, MS4A6A, MS4A4E, CD2AP, EPHA1 and CD33) was constructed in a study on risk genes for AD.31 However, adding this genetic risk score to a basic model of age, sex and APOE ε4 only marginally improved prediction of AD (AUC increased from 0.8148 to 0.8159).
The Australian National University AD Risk Index
The Australian National University AD Risk Index (ANU-ADRI) was constructed to assess the risk of later life AD13 using information on 12 risk or protective factors such as age, sex, education level, diabetes, traumatic brain injury, depressive symptoms, smoking, social networks, cognitively stimulating activities, alcohol consumption, physical activity and fish intake. The predictive ability was assessed in three independent cohorts of older adults. The AUCs ranged from 0.653 to 0.728 for any dementia, and from 0.637 to 0.740 for AD.
Other late life risk models
A model comprising age, sex, forgetfulness in daily living, the 4-item Instrumental Activities of Daily Living, the Isaacs Set Test, the Digit Symbol Substitution Test and the episodic memory subtest of the MMSE exhibited an AUC of 0.844 for dementia prediction in 5 years and an AUC of 0.814 for dementia prediction in 10 years.32
A model incorporating 13 conventional risk variables exhibited moderate predictive accuracy in individuals aged ≥65 over 10 years’ follow-up (AUC 0.77).33 The value of brain MRI-based markers was assessed, but no significant improvement was observed when markers such as white matter lesion volume (AUC 0.77), total brain volume (AUC 0.77), total hippocampal volume (AUC 0.79) or all three variables combined (AUC 0.79) were added to the conventional risk model.
Risk models for subgroups
Models for predicting MCI to AD conversion
Fifteen studies reporting models for predicting conversion from MCI to AD were included in our review. The AUCs ranged from 0.606 to 0.9325. Models in 11 studies were developed using data from the ADNI database. The best model included the following variables: Alzheimer’s Disease Assessment Scale-cognitive subscale, MMSE and 18F-fluorodeoxyglucose (18F-FDG) standard uptake values ratio of the posterior cingulate.5 This model produced an AUC of 0.932, a sensitivity of 96.4% and a specificity of 81.2%. Cerebral spinal fluid Aβ, p-tau and t-tau as newly identified variables were incorporated in four studies,6–8 34 and the AUCs ranged from 0.60 to 0.83.
Models for patients with diabetes
After evaluating 45 candidate predictors, the Diabetes-Specific Dementia Risk Score (DSDRS) for type 2 diabetes derived from age, education, microvascular disease, diabetic foot, cerebrovascular disease, cardiovascular disease, acute metabolic events and depression was constructed and validated. The model showed moderate prediction accuracy in 10 years (AUC 0.736 in creation cohort and 0.746 in validation cohort) in patients with diabetes aged ≥60 years.35 Another risk score for patients with type 2 diabetes was developed and validated.36 The model was developed with age, sex, duration of diabetes, BMI, fast plasma glucose, haemoglobin A1c, stroke, postural hypertension, hypoglycaemia, coronary artery disease and antidiabetes medications. The AUCs for 3-year, 5-year and 10-year dementia risks were 0.82, 0.79 and 0.76 in the derivation set, and 0.84, 0.80 and 0.75 in the validation set.
Based on the idea that both comorbid disease conditions (Dx) and prescription drugs (Rx) are important risk factors for dementia, the RxDx-Dementia Risk Index comprising age, gender and 31 RxDx disease conditions was developed.37 The AUC was 0.806 for patients with type 2 diabetes and hypertension, and 0.855 for patients with type 2 diabetes only.
Discussion
The aim of the risk models for the general population is to classify individuals into different risk categories and identify those with high risk. Individuals with high risk of dementia could lower their risk by modifiable risk factor manipulation. Patients with MCI have much higher risk of progressing to dementia than people in the general population. It is important to separate those who will progress to AD from those who remain stable. In fact, some of the patients with MCI are in the prodromal stage of AD. The accuracy of dementia risk models can be assessed in several ways. Predictive performance of dementia risk models is usually assessed by measures of sensitivity, specificity and AUC. However, these measures can vary depending on disease prevalence, disease severity, risk factors for disease, study sample size, diagnostic criteria, operational definitions and length of follow-up. These limitations made it difficult to compare the predictive performance of the dementia risk models. Besides, the predictive accuracy among different cohorts, the cost-effectiveness and the availability of its variables should also be taken into consideration before any recommendation can be made.
External validation of a model in new populations is a key step for extensive application. Eight models included in this review were reported to have been validated.7 9 10 13 15 19 22 23 26 35 36 Among those, there are one mid-life risk model, four late life risk models, one model for patients with MCI and two models for patients with diabetes. The CAIDE score was developed and validated among individuals of comparable age.9 10 There was no significant difference in the AUCs between the original cohort and the validation cohort, and in the AUCs among the racial groups in the validation cohort. The variables in CAIDE score are easy to obtain. So it could be used by people who want to estimate their own dementia risk not only by clinicians. The FCSRT-FR showed highly similar accuracy in three different cohorts as a single cognitive test model, which suggested good reliability.19 22 23 The predictive performance of ANU-ADRI for AD and dementia was tested in three cohorts,13 resulting in AUCs ranging from 0.666 to 0.734. The Dementia Screening Indicator was developed using data from four cohorts26 and was then tested in each of the four cohorts separately. The AUCs ranged from 0.68 to 0.78 in the four cohorts. The Disease State Index is the only model which has been validated externally for predicting conversion from MCI to AD.7 The model performed well in all the four cohorts (AUC range 0.74–0.82) as well as the combined cohort (AUC 0.76). However, the MRI, cerebrospinal fluid (CSF) and genetic variables in the model are not easy to obtain, which limited its extensive application. The DSDRS, a model developed for patients with type 2 diabetes, was also validated in a new cohort with similar predictive accuracy, despite older age and higher education of patients in the validation cohort.35 Another risk score for patients with type 2 diabetes was developed and validated in two cohorts. The model showed consistently moderate accuracy in the two cohorts,36 and the two models can be used conveniently by clinicians and patients with diabetes.
The cost of calculating the dementia risk models should also be considered. The models for general population in nine studies were developed comprising some expensive measures such as genetic and MRI variables. The AUCs ranged from 0.626 to 0.91. Generally, the predictive accuracy of the models with high-cost variables is higher than those without the variables. But the feasibility and cost constraints limited the usage of the models, especially in the primary care setting. The best model for primary care should be constructed using information already available or easily obtainable. Three studies about dementia risk models for primary care were reported before.15 25 26 Among those, the model using routinely collected data from the Health Improvement Network (THIN) database performed well in the prediction of dementia in the validation cohort.15
Candidate factors selected for model construction in different studies were highly variable. In some studies, the variables were selected by the OR of the risk factors. However, ORs cannot reflect the factors’ ability to classify subjects.38 The effect of a variable should be evaluated by its predictive sensitivity and specificity. But they are rarely reported in studies of dementia risk models. So it is hard to say the effect of each variable. The effect of follow-up time of some risk factors should be noticed. For example, the predictive accuracy of CAIDE risk score was increased after excluding BMI and cholesterol levels together in older-aged cohorts. It might be because high BMI is a risk factor of dementia with a follow-up time of over 20 years. But low BMI is a risk factor of dementia when the follow-up time is short.39 Besides, the effect of risk factors in different age groups is not the same. It has been found that higher BMI, cholesterol levels and blood pressure are associated with lower incidence of dementia in older-aged cohorts,40 which is different from the middle-aged cohorts. Age is the greatest risk factor of dementia. Dementia risk of an old person is higher than that of a young person with the same level of risk factors. Although some models were constructed without age as a variable, they were not validated in different age groups, so they cannot be recommended to general populations of all ages.
Several limitations of the current dementia risk models should be noticed. First, the majority of the current dementia risk models are not stratified by age. Indeed, only two models were developed specifically for individuals in mid-life (CAIDE and middle age self-report risk score).9 12 Second, there is dearth of data on the external validation of published risk models, while it is well known that external validation is critical in assessing a model’s capabilities and applicability. Most models have been developed in Caucasian populations, and it remains unknown if those models would perform equally well in other ethnic groups. The third limitation is that the cost and feasibility of the data gathered for calculating risk were not considered in most of the studies. In risk models, a balance should be maintained between prediction accuracy and ease of attaining the risk score. Hence, new risk factors should be cost-effective when they are considered as candidate to improve the predictive accuracy of risk models.
Conclusions
Many risk prediction models have been developed, but only a handful of them have been externally validated. The predictive ability of the existing dementia risk models is acceptable, but the lack of validation limited the extensive application of the models for dementia risk prediction in general population or across subgroups in the population. In the eight models which have been validated, CAIDE score is a good tool for mid-life dementia risk prediction. FCSRT-FR showed relatively better predictive accuracy than the other late life risk models. It can be recommended for widely use. The Disease State Index showed consistently moderate accuracy in different cohorts, so it is a good choice for predicting dementia risk for patients with MCI despite the high cost of calculating the model. The DSDRS could be recommended for patients with diabetes.
Future research should focus on improvement, calibration and validation of existing models while considering new variables, new methods and differences in risk profiles across populations and subgroups in the same population.
References
Footnotes
Contributors LT, J-TY and LF conceived the study. X-HH, X-PC and LF selected reports and extracted the data. X-HH and CZ analysed and interpreted the data. X-HH and J-TY wrote the first draft of the manuscript. All the authors critically revised the manuscript for intellectual content and approved the final version. LT and J-TY are guarantors.
Funding This work was supported by grants from the National Key R&D Program of China (2016YFC1305803), the National Natural Science Foundation of China (81471309), Taishan Scholars Program of Shandong Province (ts201511109 and tsqn20161079), Qingdao Key Health Discipline Development Fund, Qingdao Outstanding Health Professional Development Fund, and Shandong Provincial Collaborative Innovation Center for Neurodegenerative Disorders.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.