Jump to comment:

• Models for predicting risk of dementia: improvement still needed
  Silvan Licher, Pinar Yilmaz, Maarten J.G. Leening, M. Kamran Ikram and M. Arfan Ikram
  Published on: 17 July 2018

• Published on: 17 July 2018

  Models for predicting risk of dementia: improvement still needed

  • Silvan Licher, PhD Student Erasmus MC - University Medical Center Rotterdam, Rotterdam, the Netherlands
  • Other Contributors:
    • Pinar Yilmaz, PhD Student
    • Maarten J.G. Leening, Post-doc Preventive Cardiology and Cardiovascular Epidemiology
    • M. Kamran Ikram, Neurologist and Associate Professor Neuro-epidemiology
    • M. Arfan Ikram, Professor and Chair Epidemiology

  Hou et al. are to be commended for an in-depth systematic review of currently available dementia risk models that quantify the probability of developing dementia, covering both studies on community-dwelling individuals as well as clinic-based MCI studies.1 One of the key conclusions was that “the predictive ability of existing dementia risk models is acceptable, but the lack of validation limited the extensive application of the models for dementia risk prediction in general population or across subgroups in the population.” Based on recent insights, we believe that the discriminative ability of existing dementia prediction models in the general population is currently not acceptable for clinical use.

  We recently validated four promising dementia risk models (CAIDE, ANU-ADRI, BDSI, and DRS).2 In addition to external validation of these models in the Dutch general population, we also sought to investigate how these models compared to predicting dementia based on the age component of these models only. We found that full models do not have better discriminative properties than age alone. As such, we would like to make three suggestions to establish a reliable dementia prediction model.

  First, prediction models typically only report model performance on the basis of a full model.1-4 For dementia risk, however, age plays a pivotal role. Therefore, any new model should compare its predictive accuracy to age alone.

  Second, the setting in which a prediction...

  Show More

  Hou et al. are to be commended for an in-depth systematic review of currently available dementia risk models that quantify the probability of developing dementia, covering both studies on community-dwelling individuals as well as clinic-based MCI studies.1 One of the key conclusions was that “the predictive ability of existing dementia risk models is acceptable, but the lack of validation limited the extensive application of the models for dementia risk prediction in general population or across subgroups in the population.” Based on recent insights, we believe that the discriminative ability of existing dementia prediction models in the general population is currently not acceptable for clinical use.

  We recently validated four promising dementia risk models (CAIDE, ANU-ADRI, BDSI, and DRS).2 In addition to external validation of these models in the Dutch general population, we also sought to investigate how these models compared to predicting dementia based on the age component of these models only. We found that full models do not have better discriminative properties than age alone. As such, we would like to make three suggestions to establish a reliable dementia prediction model.

  First, prediction models typically only report model performance on the basis of a full model.1-4 For dementia risk, however, age plays a pivotal role. Therefore, any new model should compare its predictive accuracy to age alone.

  Second, the setting in which a prediction model is to be used will dictate which predictors are available. For instance, a ‘basic’ model would be practical in a primary care setting, by including easily and relatively low-cost obtainable information, such as non-laboratory measurements. In this setting, not only risk factors but also prodromal features of dementia may be useful, such as subjective memory complaints. Therefore, the ability to identify high-risk individuals many years before a clinical dementia diagnosis can be used in future prevention trials to intervene in the earliest phase of the disease process. In settings beyond primary care, an ‘extended’ model could be considered, which adds specialist assessments, such as cognitive testing, brain MRI, and possibly genetics.

  Finally, improvements in the use of more advanced modelling techniques and reporting of the underlying model properties are needed. For example, future studies could consider more complex effects of age on dementia risk in their candidate models, by using non-linear effects or interactions with other predictors. We also encourage the exploration of developing age stratified models, but only if sample size permits in order to prevent model overfitting. As the authors briefly pointed out, most of the included models only report discriminative properties (i.e., AUCs or C-statistics), yet information on model calibration is also required to assess model performance.5 Without data on model calibration, proper validation attempts are limited, but most importantly also hamper actual model application to clinical practice.

  In conclusion, updated and well-validated dementia risk models are still urgently needed. These models could be used in the near future to design (preventive) trials by reliably selecting high-risk individuals into such trials. Such a tailored approach could eventually benefit progress to delay or even prevent the onset of dementia.

  References
  1. Hou XH, Feng L, Zhang C, et al. Models for predicting risk of dementia: a systematic review. J Neurol Neurosurg Psychiatry 2018 doi: jnnp-2018-318212 [pii]
  10.1136/jnnp-2018-318212 [published Online First: 2018/06/30]
  2. Licher S, Yilmaz P, Leening MJG, et al. External validation of four dementia prediction models for use in the general community-dwelling population: a comparative analysis from the Rotterdam Study. Eur J Epidemiol 2018;33(7):645-55. doi: 10.1007/s10654-018-0403-y
  10.1007/s10654-018-0403-y [pii] [published Online First: 2018/05/10]
  3. Stephan BC, Kurth T, Matthews FE, et al. Dementia risk prediction in the population: are screening models accurate? Nat Rev Neurol 2010;6(6):318-26. doi: nrneurol.2010.54 [pii]
  10.1038/nrneurol.2010.54 [published Online First: 2010/05/26]
  4. Tang EY, Harrison SL, Errington L, et al. Current Developments in Dementia Risk Prediction Modelling: An Updated Systematic Review. PLoS One 2015;10(9):e0136181. doi: 10.1371/journal.pone.0136181
  PONE-D-15-14570 [pii] [published Online First: 2015/09/04]
  5. Collins GS, Reitsma JB, Altman DG, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162(1):55-63. doi: 2088549 [pii]
  10.7326/M14-0697 [published Online First: 2015/01/07]

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top