Article Text

Download PDFPDF

Discriminative and predictive power of an informant report in mild cognitive impairment
  1. V Isella,
  2. L Villa,
  3. A Russo,
  4. R Regazzoni,
  5. C Ferrarese,
  6. I M Appollonio
  1. Neurology Section, San Gerardo Hospital, Monza, Department of Neurosciences, University of Milan Bicocca, Italy
  1. Correspondence to:
 Dr V Isella
 Clinica Neurologica, Ospedale S. Gerardo, Via Donizetti 106, 20052, Monza (MI) Italy; valeria.isella{at}unimib.it

Abstract

Background: The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) requires caregivers to rate decline in patients’ cognitive and functional performance and has never been used for mild cognitive impairment (MCI).

Methods: We contrasted the discriminative and predictive power of the IQCODE with that of the Mini Mental State Examination (MMSE) and a verbal episodic memory measure, the Rey’s Auditory Verbal Learning Test (RAVLT), in 45 patients with MCI (mean (SD) age at baseline: 71.6 (4.7) years) and 30 outpatients with dementia (70.5 (6.3) years) attending the Neuropsychology Service, St Gerardo Hospital, and compared them with 55 cognitively intact elderly people (70.7 (7.1) years). Patients with MCI were followed up for at least 2 years or until conversion to dementia.

Results: In total, 24 patients with MCI (53.3%) had converted to dementia at follow up (mean (SD) duration of follow up 17.0 (7.3) months for converters and 35.0 (7.1) months for non-converters). At baseline, the ability to differentiate patients with MCI from healthy controls was similar for the IQCODE (area under the curve (AUC) 0.86) and the MMSE (AUC 0.84; z = 0.53, not significant). As predictors of conversion to dementia, a trend favouring the IQCODE (AUC 0.86) with respect to immediate (AUC 0.74) and delayed (AUC 0.75) recall on the RAVLT was apparent (z = 1.36, p = 0.087 versus immediate recall, z = 1.51, p = 0.064 versus delayed recall). The independent predictive ability of IQCODE and memory scores was evaluated through logistic regression, and the questionnaire alone yielded the best correct classification of 81%.

Conclusions: The IQCODE is an informant based measure of cognitive decline that may provide a relevant contribution to the diagnostic and prognostic investigation of patients with MCI.

  • AD, Alzheimer’s disease
  • AUC, area under the curve
  • DSM-IV, Diagnostic and statistical manual of mental disorders
  • FTD, frontotemporal dementia
  • IQCODE, Informant Questionnaire on Cognitive Decline in the Elderly
  • MCI, mild cognitive impairment
  • MMSE, Mini Mental State Examination
  • NOS, not otherwise specified
  • RAVLT, Rey’s Auditory Verbal Learning Test
  • ROC, receiver operating characteristic
  • SVD, subcortical vascular dementia
  • mild cognitive impairment
  • IQCODE
  • informant-based instruments
  • dementia
  • neuropsychology

Statistics from Altmetric.com

Latest recommendations on the investigation of preclinical dementia stress the importance of assessing cognitive decline rather than impairment, and suggest the validity of informant reports as an index of decline.1 In the field of neuropsychology, efforts to identify subjects with mild cognitive impairment (MCI) and to estimate their risk of conversion to dementia have mainly focused on the search for a direct measure of current cognitive status that is sensitive enough to detect minor impairments, with memory tests usually showing the best discriminative and predictive power, at least for the amnestic forms of MCI.2,3 Informant based instruments have received less attention. Despite some obvious disadvantages (such as the need for a collateral source), scales and questionnaires for caregivers have repeatedly demonstrated their validity for the screening of dementia (see Jorm4 for a meta-analysis on this issue), and when challenged with mild stages of cognitive deterioration. The CAMDEX informant interview,5 the Clinical Dementia Rating scale,6–8 the Brief Cognitive Scale,9 the Functional Activities Questionnaire10 and the Global Deterioration Scale11 have shown satisfactory discriminative and predictive ability in the preclinical stages of Alzheimer’s disease (AD). As well as accuracy, scales for caregivers also display a series of strengths with respect to neuropsychological tests,4,12 such as more direct correspondence with everyday functioning, independence from educational and premorbid intellectual level, better tolerability for patients, and applicability to non-compliant or non-testable cases.

Of the informant based tools, the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE)13 is one of the most well known and widely used. It comprises 26 self administered items and requires relatives or friends to compare the patient’s current cognitive and functional performances in everyday life with their level of functioning 10 years previously. A meta-analysis of studies comparing the IQCODE with the Mini Mental State Examination (MMSE)14 showed that the two scales have equivalent effectiveness at screening for dementia, with sensitivity and specificity values ranging from 69% to 89% and from 65% to 96%, respectively.4 Thanks to its “retrospective” character, the IQCODE presents the additional advantage of allowing assessment of cognitive and functional decline. However, the ability of an informant based instrument to discriminate patients with MCI from controls and to concomitantly predict likelihood of patients’ conversion to dementia of both AD and non-AD type has never been investigated.

In a previous exploratory study, we assessed the validity of the IQCODE in a small sample of patients with MCI, compared with normal elderly and patients with dementia, and achieved promising results.15 In the present investigation, we assessed the contribution of the IQCODE to the diagnostic and prognostic investigation in a larger sample, while comparing it with the discriminative and predictive power of two standards for the screening and prediction of dementia: the MMSE and an episodic verbal memory measure, the Rey’s Auditory Verbal Learning Test (RAVLT),16 which has properties (verbal content, free recall, and delayed testing) that are deemed to be particularly sensitive to mild cognitive deficits.17

MATERIALS AND METHODS

Participants

Patients with MCI were selected from new outpatients attending our neuropsychology service from March 2000 to July 2003 for subjective cognitive disturbances or referred by their general practitioner or by a neurologist suspecting cognitive impairment, and newly receiving a diagnosis of MCI according to Petersen’s outlines:18 self or informant’s cognitive complaint, one or more abnormal score at psychometric assessment with respect to normative data, and no significant interference with activities of daily living. In addition to unavailability or actual lack of a compliant informant, exclusion criteria were: presence of serious medical illness or physical disability, past or present history of neurological or psychiatric disturbances (including stroke, extrapyramidal disorders, and severe depression), and substance misuse. After all data were collected, further selection was subsequently made according to the availability of adequate follow up data; eligible patients had to have been followed up for at least 2 years, or until a diagnosis of dementia had been reached.

Standardised diagnostic criteria for dementia from the Diagnostic and statistical manual of mental disorders (DSM-IV)19 were applied for the selection of subjects to be included in the group with dementia (as well as for the classification of MCI converters) among outpatients of the neuropsychological service. Controls were healthy subjects taking part in our ongoing normative study of the Italian version of the IQCODE; they were male or female volunteers aged ⩾50 years. Clinical exclusion criteria were the same as for patients with MCI for both demented and healthy subgroups; controls also had to achieve a MMSE score ⩾24 after adjustment for demographic variables.20

Of the initial pool of 77 individuals meeting study inclusion and exclusion criteria for MCI, 18 (23.4%) could not be included because of inadequate follow up data, 10 (13%) because of lack of an informant, and 4 (5.2%) because of incomplete IQCODE protocols (defined as three or more answers missing from the 26 required in the questionnaire). Patients excluded did not differ from those enrolled in features such as age, education, and MMSE and memory scores (data not shown). The final study sample included 45 patients with MCI, 30 patients with dementia, and 55 normal controls. Enrolled patients had either degenerative (n = 36, 80%) or subcortical vascular (defined according to the outlines of Rockwood et al;21 n = 9, 20%) mild cognitive impairment. In total, 27 subjects (60%) were purely amnestic, 10 (22%) had memory deficits plus impairment in another cognitive domain, and 8 (18%) had impairment in a single or in multiple non-memory domains. The group with dementia comprised 17 patients (56.6%) with AD, according to criteria from the NINDS-ADRDA;22 5 (16.7%) with frontotemporal dementia (FTD), according to consensus criteria;23 3(10.0%) with mixed dementia; 2(6.7%) with subcortical vascular dementia (SVD), according to the modified criteria from NINDS-AIREN for probable vascular dementia;24 and the remaining three subjects (10.0%) were classified as having dementia not otherwise specified (NOS).

The caregivers of the patients with MCI were 58.4% female, had a mean (SD) age of 52.3 (13.3) years (range 26–76) and a education of 10.7 (4.6) years (range 4–17). They were mostly spouses (44.5%, n = 20), children (40.0%, n = 18), siblings or in laws (6.7%, n = 3), or friends (4.4%, n = 2). For two subjects, (4.4%) the informant’s relationship type was not registered. Over half (56%) lived with the patient. The caregivers of the patients with dementia were 63.1% female, with mean (SD) age of 53.7 (13.1) years (range 33–79) and education of 9.8 (4.0) years (range 5–17). They were mostly children (51.7%, n = 16), followed by spouses (37.9%, n = 11), and siblings or in laws (7.4%, n = 2). For one informant (3.0%), the relationship type was not registered, and 52% of caregivers lived with the patient.

Methods

We compared the ability of the IQCODE and the MMSE to discriminate patients with MCI from patients with dementia and age matched controls, and the ability of baseline IQCODE and RAVLT scores to predict progression from MCI to dementia.

At baseline, all patients underwent neurological and neuroradiological investigation, and performed an extensive psychometric battery including standardised tests for the evaluation of attention, verbal and visual episodic memory, language production and comprehension, visuospatial abilities, praxis, executive functions, and reasoning. Mood and behavioural disturbances were also assessed. Follow up of patients with MCI was carried out repeatedly, approximately every 12 months, according to our usual patient care schedule, with the same psychometric battery.

The IQCODE was administered only at baseline. It requires caregivers to rate changes that have possilbly occurred over the past 10 years in the patient’s ability to cope with several everyday activities, on a scale from “much improved” (1 point) to “much worse” (5 points); a score of 3 is given for a “no change” judgment. Total score derives from averaging the ratings over the total number of completed items and ranges from 1.0 to 5.0, with higher scores indicating worse decline. Completion takes approximately 10 minutes. The questionnaire was filled in autonomously by informants after receiving appropriate instructions, while patients were undergoing neuropsychological assessment. At the end of the patient’s testing session, scoring of the IQCODE was performed by an examiner (one psychology trainee) blinded to patients’ data, after responses had been checked with the caregiver to ensure full comprehension and completion of questions.

The MMSE and the RAVLT immediate recall (sum of five learning trials) and delayed recall (translation and normative data which are available for use in the Italian population25) were administered as part of the neuropsychological assessments both at baseline and at follow up. These results were available to the clinician who made the diagnosis at baseline and who judged conversion at follow up, whereas the IQCODE scores were not.

Statistical analysis

Statistical analysis was performed with SPSS (version 10.0; SPSS Inc., Chicago, IL, USA). Student’s t test, χ2 analysis, and analysis of variance with pairwise post hoc comparisons were used to compare means of discrete and continuous variables among the study groups, with two tailed standard significance level set at p<0.05. Correlation analysis was carried out using Pearson’s r coefficient.

Receiver operating characteristic (ROC) analysis was used to assess the accuracy of each instrument as a screen for mild cognitive impairment and as a predictor of transition to dementia. Areas under the curve (AUC) were compared using the method of Hanley and McNeil,26 considering a one tailed significance level of p<0.05. Sensitivity, specificity, and positive and negative predictive values of the instruments were also calculated at their best cut off points. Backward stepwise logistic regression analysis (conversion to dementia (yes/no) as dependent variable) was performed to establish the independent predictive ability of the IQCODE, the RAVLT immediate and delayed scores, and the demographic variables.

RESULTS

Comparisons between the sociodemographic and main neuropsychological features of the study groups are shown in table 1. No statistically significant difference in demographics was detected. The MMSE and IQCODE scores were significantly different for all study groups, with the worst performance shown by patients with dementia and intermediate values by patients with MCI.

Table 1

 Comparison between demographic and baseline neuropsychological variables of the study groups

ROC curves for the MMSE and the IQCODE as measures for discriminating between patients with MCI and normal controls are shown in fig 1A. Areas under the curves were overlapping: 0.86 (95% confidence interval (CI) 0.78 to 0.93) for the IQCODE, 0.84 (0.73 to 0.92) for the MMSE (z = 0.53, p>0.05).

Figure 1

 Receiver operating characteristics (ROC) curves for (A) the MMSE and the IQCODE as screening tests for mild cognitive impairment and (B) Rey’s immediate and delayed recall and the IQCODE as predictors of progression from mild cognitive impairment to dementia.

Sensitivity, specificity, and positive and negative predictive values obtained at the best cut off points for the two screening tools are shown in table 2. Overall accuracy was 76% for the IQCODE and 77% for the MMSE.

Table 2

 Sensitivity, specificity and positive and negative predictive values for the MMSE and the IQCODE as screening tools for mild cognitive impairment and for Rey’s immediate and delayed recall and the IQCODE as predictors of progression from mild cognitive impairment to dementia

At follow up, patients with MCI were subdivided into two groups, according to their clinical evolution: 24 subjects (53.3%) had converted to dementia (converters), while 21 (46.7%) had reverted to normal mental function (n = 6, 13.6%) or had remained with MCI (n = 15, 33.1%) (non-converters). Of the converters, 12 (50.0%) met criteria for AD, 3 (12.5%) for SVD, 3 (12.5%) for FTD, 2 (8.3%) for mixed AD/SVD, and 4 (16.7%) had dementia NOS. Average (SD) follow up duration was 17.0 (7.3) months for converters and 35.0 (7.1) months for non-converters

Comparison between sociodemographic and neuropsychological features of converters, non-converters, and patients with dementia is shown in table 1. There was no statistically significant difference in age, education, and disease duration; non-converters included a significantly larger percentage of men with respect to the other two groups. Baseline scores for MMSE and Attentional matrices from the MDB25 were significantly lower for demented than both MCI subgroups, while there was no difference between converters and non-converters. Baseline RAVLT immediate recall, IQCODE, and verbal fluency scores were significantly different among the three groups, with the worst performance for patients with dementia and intermediate values for converters; baseline RAVLT delayed recall score was similar for patients with dementia and for converters and significantly lower with respect to non-converters.

Logistic regression analysis performed on the MCI subgroup, with transition to dementia (yes/no) as dependent variable and including age, education, RAVLT immediate and delayed raw scores, and the IQCODE as predictors, showed that the questionnaire alone had the highest predictivity, resulting in an overall correct classification of 81% (36/45 subjects) (β = −4.506, Wald statistics = 10.394, p<0.001); all of the other variables were not retained in the final model.

ROC curves for RAVLT immediate and delayed raw scores and the IQCODE as measures predicting conversion from MCI to dementia are shown in fig 1B. Area under the curve was 0.86 (95% CI 0.74 to 0.98) for the IQCODE, 0.74 (0.58 to 0.89) for immediate recall, and 0.75 (0.60 to 0.90) for delayed recall; there was no statistically significant difference between the areas, although a trend favouring the IQCODE was apparent: z was 1.36 for the IQCODE compared with immediate recall (p = 0.087) and 1.51 for the IQCODE compared with delayed recall (p = 0.064).

Sensitivity, specificity, and positive and negative predictive values obtained at the best cut off points for the three measures are shown in table 2. Overall accuracy was 80% for the IQCODE, 73% for RAVLT immediate recall, and 67% for delayed recall.

“Predictive” ROC curves were also calculated for RAVLT adjusted scores: AUC was 0.70 (95% CI 0.54 to 0.86) for immediate recall and 0.75 (0.60 to 0.90) for delayed recall; there was a trend towards significance in favour of the IQCODE with respect to both RAVLT immediate (z = 1.58, p = 0.057) and delayed recall (z = 1.31, p = 0.095). Corresponding sensitivity, specificity, positive and negative predictive values obtained at the best cut off points are shown in table 2. Overall accuracy was 67% for immediate recall and 64% for delayed recall.

Correlation analysis was performed on the pool of patients with MCI for the IQCODE, MMSE, and RAVLT scores, considering patients’ main demographic and clinical features. Results are displayed in table 3. The IQCODE was significantly correlated with the MMSE (p = 0.027), and RAVLT immediate (p = 0.004) and delayed (p<0.001) recall. It was also correlated with age (p<0.001), but not with education and disease duration; the same was true for RAVLT scores.

Table 3

 Correlation between the IQCODE, the MMSE, and the RAVLT immediate and delayed recall scores with sociodemographic and clinical features of patients with MCI (n = 45)

DISCUSSION

In this study, we have shown that the IQCODE is a valid instrument for differentiating between normal elderly subjects and patients with subtle cognitive disturbances and for early prediction of the future development of dementia in MCI. We also provide reference values to classify patients at their referral for questionable cognitive decline (although cross validation would be needed before these can be used operationally): an IQCODE score <3.22 makes a diagnosis of MCI unlikely, while values >3.45 indicate patients with MCI with a high risk of transition to frank dementia within the following 2–3 years; intermediate scores point to a diagnosis of MCI with a more unpredictable evolution, deserving longer longitudinal monitoring.

The level of discriminative accuracy obtained by the IQCODE was basically identical to that of the MMSE, and high IQCODE scores also showed the instrument to be a very sensitive and specific marker of progression to dementia. Using ROC curve analysis, the questionnaire even yielded a slightly, though not significantly, higher predictive accuracy than immediate and delayed recall scores, mainly due to the low specificity displayed by both memory measures. Susceptibility of neuropsychological tests to low educational level are unlikely to account for such a finding, given that, in our MCI sample, patients’ years of schooling were shown to be unrelated not only to the IQCODE (in agreement with previous evidence suggesting its cultural fairness),27–30 but also to the RAVLT scores. The most likely explanation is the relatively short duration of study follow up. Evidence has been gathered from both group31 and single32,33 case studies that amnesia may remain isolated for up to 10 years in MCI. An alternative explanation is that memory ability is more affected by physiological aging than performance in everyday activities, but, even though we did find a significant correlation with patient’s age for memory tests, we found an even higher correlation for the questionnaire (as described previously27,29 and perhaps reflecting an increasing rate of cognitive decline in older subjects34). The higher proportion of false positives we observed for the RAVLT may then be due to the fact that, unlike informant ratings that directly assess change,29,35,36 recall ability as measured by tests might be confounded by (poor) premorbid intellectual level. The possible contribution of contingent factors interfering with patient performance during the testing session might be a further explanation.

One previous study by Estevez-Gonzalez et al37 addressed the issue of the validity of the RAVLT in preclinical dementia and also considered the IQCODE. However, their study sample is different; their controls were subjects with subjective memory complaint, rather than elderly from the general population, and non-converters all had a diagnosis of MCI at follow up and no patient reverted to normality. This may have determined greater overlap among the subgroups, leading to limited predictive validity for the questionnaire.

Another discrepancy with respect to Estevez-Gonzalez’s work deserves comment: correlation between the MMSE and immediate recall at the RAVLT was definitely higher than in our study (and statistically significant). A modest relationship between the two tests is, in fact, puzzling. Restriction of score range might be a possible explanation. In fact, the array of recall scores was much wider in Estevez-Gonzalez’s study, perhaps due to the fact that correlation analysis was performed on the whole study sample (including subjects with subjective memory complaints). The preclinical and conclamate patients with non-AD dementia included in our study population may have lost points in MMSE items other than those responsible for the relationship with memory tests (verbal registration and recall), whereas Estevez-Gonzalez et al only enrolled cases of pure amnestic MCI and impending AD.

One further unexpected but intriguing finding of the present study is represented by the worse predictive performance of memory tests after adjustment for age and education, which may perhaps be taken as evidence of the influence exerted by age on the risk of developing dementia (so that scores free of its effect may underestimate risk of conversion).

Non-AD patients were scarcely represented in our study population, so that we could not expressly evaluate the effects on IQCODE scores of motor disability possibly associated with cerebrovascular disease, or of neuropsychiatric disturbances accompanying frontal dysfunction (although one previous study showed that mood and psychotic disorders per se have limited influence on the IQCODE38). Given the impact these factors have on activities of daily living, they might affect the accuracy of the questionnaire, and may at least partly explain its relatively low specificity in our mixed study sample. In contrast, they did not seem to decrease sensitivity, suggesting that the questionnaire may be valid for a timely detection of different variants of preclinical dementia. This might be because, unlike single neuropsychological tests, caregiver rated scales tap multiple aspects of cognitive and functional competency. The significant correlation we found between the questionnaire and memory scores and correlations with other cognitive domains shown in previous investigations12,29,35 also suggest the composite nature of the general factor underlying the IQCODE.27,28

One possible caveat of the present study design concerns the neuropsychological standards chosen to define the discriminative and predictive power of the IQCODE. The MMSE is unquestionably the mainstay of dementia screening, and episodic verbal memory tests seem to measure up as the strongest predictors of transition from MCI to dementia (see Arnaiz et al3), but both instruments are tuned in to the amnestic, pre-AD cognitive profile, whereas our MCI sample was rather heterogeneous. Impairment of attention and executive functions, and relative sparing of episodic memory represent the most frequent clinical presentation of vascular and frontotemporal dementia.23,39,40 The MMSE and the RAVLT might thus have been “penalised” by the inclusion of subjects presenting with a non-AD neuropsychological profile. In truth, had this been the case, we would have expected a major number of false negatives, in particular for memory measures, while recall scores yielded poor specificity, but rather good sensitivity values. Such good sensitivity shown by the two measures indicates that their bias towards AD was limited.

Another limitation of the present data regards their generalisability, given that accuracy values (and cut off points) obtained by the diagnostic instrument under scrutiny depend on the prevalence of the disease in the setting where it is being validated. Our promising findings, however, encourage verification of the accuracy power of the IQCODE in a larger, multicentre clinical sample. Final shortcomings of the present study are the small sample size, the short follow up duration, and the non-consecutive mode of enrolment. Subjects excluded because they did not have an informant or a sufficiently long follow up appeared to be grossly comparable to those included into the study. However, patients with MCI presenting alone at first referral or lost at follow up might have had milder cognitive and functional impairment; selection bias towards relatively more severe cases may thus not be fully ruled out.

One future interesting adjunct to the present survey might be the assessment of the combined use of the questionnaire with other measures. The correlation we found between the IQCODE and the MMSE was statistically significant, but not very high (similar to findings by Jorm et al29 and Christensen and Jorm30), suggesting that there is actually limited overlap of the information they provide. On the other hand, regression analysis demonstrated that memory measures do not add much to the independent predictive ability of the questionnaire. The combined use of behavioural and non-cognitive markers of dementing illness (the IQCODE and imaging41 or neurobiological and genetic41–44 indexes) might perhaps be more promising.17 Finally, systematic investigation of informants’ features influencing their rating accuracy (which requires an ad hoc study design) might help in identifying the most reliable collateral sources.

REFERENCES

Footnotes

  • Competing interests: none

  • All necessary approval for the study was secured from the ethics committee of St Gerardo Hospital, Monza, and written consent was obtained from patients taking part in the study.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.