METHOD

Participants

Participants were 568 consecutive new patients attending the Bristol Memory Disorders Clinic between 7 June 1993 to 28 August 1997. During this period 195 (34.3%) had QD, 370 (65.0%) were diagnosed with a dementia, and three (0.5%) were found to have no cognitive impairment at all. Of the 195 QD patients, 135 (69.2%) were seen for at least one follow up visit.

Patients were referred to the Bristol Memory Disorders Clinic (BMDC) by their GP or other healthcare professional. Each patient underwent medical, psychiatric, and psychological screening to exclude any other treatable disease. Particular attention was paid to presenting symptoms, onset (sudden or insidious), progression (static, stepwise, or gradual), and presence of memory and other cognitive problems, as well as affective or behavioural difficulties. Past medical history was also evaluated, emphasising conditions that might be associated with cognitive impairment, medications, and substance misuse. Family history of depression and organic or neurological disease was also noted. A depression rating scale was used (Cornell Scale for Depression in Dementia).⁴² Patients were referred to a psychiatrist if there was any clinical suspicion of affective or psychotic illness or if they scored above the cut off on the depression rating scale.

Behavioural and functional deficits were measured in interview with a knowledgeable collateral source (generally a spouse or adult child) using the Stockton Geriatric Rating Scale.⁴³ The standardised Hachinski Ischaemic Scale^44,45 was administered. A comprehensive physical examination was undertaken including neurological examination. Laboratory blood testing, CT brain scans, and where clinically indicated single photon emission computed tomography or magnetic resonance imaging were also carried out.

The neuropsychological assessment used in the BMDC was designed and validated specifically for the clinic.⁴⁶ The assessment included measures of higher order cognitive function assessing short-term and working memory (Digit Span, forwards and backwards), abstract thinking (Similarities), and general non-verbal problem solving (Picture Completion) all Wechsler Adult Intelligence Scale-Revised (WAIS-R⁴⁷), measures of episodic memory assessing immediate and delayed story recall (Adult Memory and Information Processing Battery⁴⁸), list recall, verbal learning, and verbal recognition (Hopkins Verbal Learning Test: HVLT⁴⁹), and visual recognition (Middlesex Elderly Assessment of Mental State: MEAMS⁵⁰). Other tests assessed language expression, reception, and executive function using the Frenchay Aphasia Screening Test,⁵¹ letter fluency,⁵² and Weigl’s Colour Form Sorting Test⁵³ and measures of visuospatial function (Cube analysis, Visual Object Space Perception Battery⁵⁴) and psychomotor speed (Digit copying⁵⁵).

Each patient’s assessments were discussed in a multi-disciplinary case conference by trained clinicians. The diagnosis was based on a history of cognitive impairment relative to the patient’s premorbid abilities. Impairment in any cognitive function was defined as performance 1.5 or more SD below expected level of functioning using age adjusted means. For AD, this history was of gradual onset and progressive course. Diagnosis of dementia was made according to the Diagnostic and Statistical Manual of Mental Disorders-Revised IV Edition.⁵⁶ A diagnosis of probable AD was made according to the NINCDS-ADRDA criteria.⁵⁷ A diagnosis of QD was made if a patient was (a) experiencing memory impairment that affected their social or occupational functioning but without other cognitive impairment, or (b) experiencing cognitive impairment in one or more areas of functioning but this impairment did not result in significant change in social or occupational functioning. Thus, a diagnosis of QD indicated that there was objective evidence of cognitive impairment but that these impairments did not satisfy the criteria for dementia. The outcome measure was the final diagnosis at follow up.

One hundred and thirty five participants were followed up for 4.8 to 95.7 months (mean 24.5, SD 19.7, range 1–7 follow up visits, mode 1, median 2). At final follow up their diagnoses were reviewed, at which time 37 (27.4%) had gone on to develop dementia and 92 (68.1%) continued to suffer QD, additionally, six (4.4%) had recovered from all cognitive deficits and were deemed to be normal. Table 1 gives the descriptive characteristics of those who converted to dementia and those who continued to suffer from QD.

Statistical analyses

Univariate Mann-Whitney U and χ² statistics were used to compare groups. Prediction of outcome was evaluated using Cox regression proportional hazards analysis. Either variables were used as continuous covariates, or the neuropsychological variables were recoded categorically according to whether patients scored above or below the 5th centile cut off for age adjusted means. With the exception of the HVLT learning index these means were taken from published normative data. The Learning Index was based on a similar index developed by Bucks et al.⁵⁸ (See footnote*). All analyses were performed with SPSS for Windows version 11.0⁵⁹ using an α level of 0.05.

RESULTS

There were no significant differences between the groups in premorbid IQ, years of education, dependency, ischaemic score, or depression, nor was there a significant sex difference. Those who developed dementia reported a significantly shorter mean history of cognitive impairment, had significantly lower mean MMSE scores, and were significantly older than those who did not develop dementia (see table 2). Table 2 shows the distribution of diagnoses. In those who went on to develop dementia, a diagnosis of probable or possible AD was made in over half the cases. Of those who continued with QD a specific aetiology was identified in less than half.

Table 3 shows mean neuropsychological test performance at the initial visit. There were significant group differences in Similarities, all the Memory measures and Verbal Fluency.

Cox regression analyses were performed with progression to dementia as the event, and time to diagnosis or survival as the time variable. Only those variables on which the two groups performed significantly differently in univariate comparisons were entered into the equations. Because of small amounts of missing data, only those cases for whom there was complete data were entered into the analyses (QD n=80, D n=30). Three variants of the Cox regression analysis were conducted (Enter, Forwards Stepwise, and Backwards Stepwise) for both continuous and categorical predictors. This permitted comparison with the same sorts of models found in the literature as well as an exploration of the stability of the analyses. Table 4 shows the prediction results for each of the different analyses performed.

A number of the models compared favourably with published findings. However, using the neuropsychological test scores as continuous predictor variables produces problems because of colinearity of the predictor measures: that is, the tendency for test results to vary systematically with each other as a function of disease severity. This can be seen in the degree to which Models 1 to 3 differed in the variables that were shown to contribute significantly to the likelihood of converting to dementia. In Model 1, all variables were forced into the equation together. Only two were found to be potentially significant explanatory variables: Verbal Recognition and Verbal Fluency. In Model 2, a forwards stepwise procedure resulted in two different variables, HVLT Total List Recall and MMSE being left in the equation. In Model 3, using continuous covariates, a backwards stepwise procedure resulted in four variables being left in the equation, two of which were the same as the significant explanatory variables in Model 1, and none of which was the same as in Model 2. These were Age, Visual Recognition, Verbal Recognition, and Verbal Fluency. In the models that used categorical covariates there was consistency in the variables that were identified as significant predictors (Models 4–6) methods. In all three the significant covariates were Age and HVLT Total List Recall.

DISCUSSION

As in previous studies,^{8,14–17,24,35,36} a number of neuropsychological parameters seem to offer potential to distinguish between those who will and those who will not go on to dement. However, despite significant group differences in these measures, many did not prove to be consistently significant predictors of conversion to dementia. For example, despite a significant difference in mean MMSE score, this was not consistently a significant predictor of final diagnosis of dementia, indeed it was only a significant predictor in one model. This finding reflects the generally poor status of the MMSE as a significant predictor noted in other studies.^1,2,38,60

As has previously been found, there were significant differences between the QD and D groups on univariate analysis of all memory tests.^{7,16,18,23,30,60} However, the predictive power of memory test performance was less convincing. Recall performance, whether Immediate or Delayed, was not found to be significantly predictive of final diagnosis, although, Verbal Recognition was found to be a significant predictor in Models 1 and 2. Other neuropsychological test variables that appeared as significant predictors in some models were Verbal Fluency (Models 1 and 3) and Visual Recognition (Model 3).

In all analyses using categorical scores (Models 4–6) only low Total List Recall performance was significantly associated with increased risk of converting to dementia. Otherwise, only increasing Age was consistently found to be moderately positively associated with increased risk of converting to dementia: a finding consistent with a number of other studies.^18,38,61

By comparison with the Devanand results,² a smaller proportion of our sample converted to dementia (27.4% compared with 41.3%). Although this may relate to the slightly shorter mean interval of follow up (25 months compared with 30 months) between the studies, there was no significant difference between our groups in the length of follow up to diagnosis or survival making this explanation unlikely. There was also a smaller proportion of subjects with diagnoses of probable or possible AD; in our sample 56.6%, in Devanand et al 87.1%.

Despite this difference in base rates of conversion, in terms of predictive accuracy, these results seem to replicate Devanand et al. Indeed, the sensitivity, specificity, positive and negative predictive values appeared quite good. Sensitivity values ranged from 53.4% to 64.4%. Specificity values ranged from 66.7% to 83.3%. Positive predictive values (PPV) were equally good, although the best was associated with only modest sensitivity (Model 3) and should, therefore, be discounted. Good sensitivity is often gained at the expense of poor specificity, with the same trade off occurring for PPV and negative predictive values (NPV). For example, the best PPV (Model 1: 87.0%) was associated with poor NPV (46.9%). Thus, for every two people who were predicted to continue with QD, this would be incorrect for one of them. In addition, only two thirds of the people who actually did convert to a diagnosis of dementia were correctly identified (sensitivity 64.4%) and one in four of those who did not were misidentified (specificity 76.7%). Thus, a technique that at first sight looks promising, is really rather less than satisfying.

Review of the literature reveals a number of explanations for this general difficulty with predictive studies. These include issues of test sensitivity,^1,6 as well as the problem that many of the tests are highly intercorrelated. This colinearity between independent variables is one of the primary reasons for poor regression models. Unfortunately, few studies confront this issue. For example, despite reporting a failure to find significant associations attributable to colinearity in their predictor variables Devanand et al² went on to conduct a discriminant function analysis. Discriminant function uses another type of regression model that is no less susceptible to the effects of colinearity than their original analysis. Many other studies simply force all variables into their regression models (Enter method) rather than allowing them to be placed into or out of the final equation according to a statistical criterion as in a stepwise model. With highly intercorrelated data, in a stepwise model what ends up in the equation will therefore depend on the order with which the variables were put into or taken out of the equation. This is demonstrated clearly by the instability shown between Models 1 to 3.

Some researchers have sought to resolve this correlation problem by conducting principal components analysis (PCA) to reduce their dataset to a smaller number of uncorrelated components.^24,25,28,62 Fabrigoule and colleagues^24,25,28 isolated a component that they interpreted as reflecting a disturbance of control processes. Indeed, a number of researchers have argued that studies should be focusing on attentional control processes in their search for a specific marker of incipient dementia.^24,25,28,62 Findings regarding the extremely specific nature of the central executive deficits found in mild AD patients, particularly their marked difficulty with dual task performance^63,64 would tend to support this view. However, reported inconsistency regarding the components identified by PCA,^28,62 and the difficulty of applying PCA analysis to routine clinical data make such approaches currently unsuitable.

An additional problem with these types of studies is that many of the groups have different mean years of education or predicted premorbid IQ scores. A number of studies have controlled for this difference by regressing out years of education or IQ before analysing the predictive power of the test scores.^7,8,25,32,34 Unfortunately, analysis of covariance is often not an appropriate method for providing statistical control over differences between groups in neuropsychological research, as it can remove the variance of interest that is associated with age or education factors.⁶⁵ One final problem with these types of studies is that while some exclude people with depression, others do not. However, evidence relating to the additional effect of depression on risk of developing dementia is equivocal.^61,66 Moreover, in this study, mean IQ, years of education and mean number of depressive symptoms did not differ between the two groups.

It could be argued that any study predicting conversion from an outpatient population, such as that taken from a memory clinic will be problematic given the lack of generalisability to the general population of older adults. However, a large number of studies using both community and clinic based samples have failed consistently to identify a class or class of neuropsychological measures that produce satisfactory prediction rates. Thus, the issue seems less to be one of generalisability of the findings and more of generalisability of the method; which we feel has yet to be established.

Perhaps, it is currently ambitious to try to predict long term outcome from assessment at a single time point. Indeed, a number of studies have attempted to evaluate the predictive power of change scores.^33,62,67 While some studies have found evidence of significant changes in test performance up to five years before diagnosis,^67,68 others have shown marked changes in test performance only shortly before the diagnosis is made.^26,33 Furthermore, at least one set of researchers have questioned the validity of any predictive studies.⁶²

A range of alternative avenues for predicting conversion to dementia are currently being explored in the literature (see Almkvist and Winblad⁶⁹ for a review). These include volumetric analyses of medial temporal lobe and hippocampal structures,^20,70–75 functional neuroimaging analyses of blood flow,⁵ or of cortical glucose metabolism,³⁷ genetic risk factors (for example, apoE4,)^{15,35,36,76–78} biological markers in cerebrospinal fluid,¹ and nerve growth factor.³² Indeed, Collie and Maruff⁵ and Reischies and Hellweg⁷⁸ recommended a convergent approach, combining measures (see for example, Albert⁷⁴ and Daly et al³⁶). While a convergent approach may improve predictive validity, it makes predictions of this type even further beyond the reach of the average geriatric outpatient service. In addition, it does not avoid the serious difficulties identified with determining what is normal for cognitive or neuroanatomical measures for any particular age group.

While it is tempting to seek a combination of neuropsychological and clinical measures that can predict the development of dementia, analysis of our clinical data suggests that it would be more fruitful to expend our energies on searching for a new, specific and unique cognitive, biological or anatomical marker, than to pursue these types of predictive models. Unfortunately, statistical significance does not, as yet, offer clinical utility.