Cerebral small vessel disease (SVD) is an important cause of cognitive impairment and of vascular dementia, rivalled only by Alzheimer’s disease as a cause of dementia worldwide.¹ Disease of small end arteries that penetrate from the pial surface to the deep white matter and deep grey matter nuclei leads to both focal lacunar infarcts and more diffuse regions of axonal loss, demyelination, and gliosis.^2,3 The neuroradiological correlate of these diffuse neuropathological abnormalities is referred to as leukoaraiosis and is seen as diffuse hypointensity on computed tomography (CT), or as corresponding hyperintensity on T2 weighted magnetic resonance imaging (MRI).⁴

Cerebral SVD is associated with cognitive decline, particularly in the presence of leukoaraiosis. Many studies have shown an association between leukoaraiosis and cognitive function, both in patient groups⁵ and in asymptomatic older adults.^6–8 The importance of leukoaraiosis as a substrate of vascular dementia is underlined by a recent clinical trial in vascular dementia, where 64% of the participants had extensive white matter lesions.⁹ Although leukoaraiosis is most commonly caused by SVD, it is a radiological entity and can also been caused by other pathologies. The term “ischaemic leukoaraiosis” has been introduced to identify a group of patients with leukoaraiosis in whom cerebral SVD is likely to be the underlying pathology.¹⁰ Ischaemic leukoaraiosis is defined as radiological leukoaraiosis in combination with a clinical lacunar stroke.¹⁰

The pattern of cognitive impairment found in cerebral SVD differs from that found in Alzheimer’s disease. Deficits of episodic memory, the hallmark of Alzheimer’s disease, are often mild or absent.¹¹ In contrast, executive dysfunction is a characteristic feature.^12–14 Executive function refers to those higher cognitive processes by which performance is optimised in situations requiring the simultaneous operation of several cognitive processes.^15,16 Executive skills are used to construct effective plans of action, particularly in the face of complex tasks. Component processes include planning, the marshalling of attention (including the switching of attention between different aspects of a task or problem (attentional set shifting)), and selection of appropriate responses. Analysis of psychometric data suggests that executive function can be broken down into components in this way, and has shown that tests can be relatively specific for certain components.¹⁶ Working memory, which refers to the ability to hold and manipulate information mentally, is also required for these processes and is best considered as a component of executive function quite distinct from episodic and long term memory.¹⁷ This classification is supported by neuropsychological and lesion studies, and the relation between working memory and a “central executive system” has been expressed in the working memory model.¹⁵

Most established brief assessment tools, such as the mini-mental state examination (MMSE),¹⁸ have been developed in the context of Alzheimer’s disease. These tools are insensitive even to gross deficits of executive function¹⁹ and are therefore likely to be insensitive in cerebral SVD. There is a need for brief cognitive testing instruments tailored to the detection and measurement of executive dysfunction in cerebral SVD, for use both in the clinic and for serial assessment of cognitive decline in clinical trials.

In this study, a brief assessment of executive function was developed and evaluated in a group of patients with ischaemic leukoaraiosis. Tests were chosen so that the protocol was less than 30 minutes in length, relatively easy to administer, and sensitive to a range of aspects of executive functioning. One essential property of such a brief assessment tool is the discrimination of cognitive changes caused by disease from those of normal aging. Disentangling disease from aging effects is particularly challenging for a brief executive assessment, as various studies of cognitive aging suggest that executive functions may decline earlier and more rapidly than other cognitive domains.²⁰ We therefore set out to determine the ability of our brief executive assessment to discriminate between patients with cerebral SVD and healthy elderly controls. In addition, the ability of the assessment to discriminate between groups was also compared with that of a more comprehensive neuropsychological battery to determine whether any other important deficits of executive dysfunction, intellectual function, or memory were overlooked by the brief executive assessment.

METHODS

Subjects

Ischaemic leukoaraiosis was defined as diffuse white matter hyperintensity (corresponding to grade 2 or 3 on the Fazekas scale²¹) on T₂ weighted MRI in patients with a history of a clinical lacunar event.²² Thirty two subjects with ischaemic leukoaraiosis were recruited from a specialist cerebrovascular disease outpatient clinic. All had had a transient ischaemic attack or a minor stroke in the past, which was the reason for referral to the clinic, though none had had an event in the three months preceding study entry. Exclusion criteria were: contraindications to MRI; carotid stenosis (>50%); cardioembolic sources of stroke; large subcortical infarcts (greater than 15 mm maximum diameter), as these infarcts often have a large vessel or embolic aetiology; cortical abnormalities on MRI; and a history of previous neurological or psychiatric disease, as these may have an influence on cognitive function. Subjects fitting these criteria and who agreed to participate were recruited consecutively.

Seventeen normal community volunteer control subjects, free of past or present neurological or psychiatric disease, were also recruited. These were from a database of healthy volunteers held by the clinical age research unit at King’s College Hospital and were drawn from the same geographical population as the patients.

The groups were matched for age (ischaemic leukoaraiosis, range 50 to 84, mean (SD), 70.0 (8.8) years; controls, range 56 to 84, mean 72.4 (7.4) years; p = 0.32), sex distribution (ischaemic leukoaraiosis, 22 men, 10 women; controls, 11 men, 6 women; χ² test, p = 0.77), and educational history (ischaemic leukoaraiosis, 11.0 (3.5) years, controls, 11.9 (1.9) years, p = 0.28).

The study was approved by the local research ethics committee and subjects gave informed, written consent.

Clinical and MRI assessment

All subjects underwent full clinical and neurological examination and MRI. MRI included dual echo (proton density and T2 weighted), FLAIR (fluid attenuating inversion recovery), and T1 weighted images. Multiple contiguous 3 mm slices were prescribed in a true axial plane to provide complete brain coverage. Control subjects had no more than five focal hyperintensities on T2 weighted images, corresponding to grade 1 on the Fazekas scale. This criterion was chosen as pathological verification of this scale indicates that an ischaemic basis for this grade of white matter change is unlikely.²¹

Neuropsychological assessment

Neuropsychological testing was carried out in a single session lasting two to three hours. The battery included tests from a putative “brief executive assessment” and a range of other tests. Tests from the brief executive assessment and additional tests were administered together as one battery, in a standard order in which elements of the brief executive assessment and the additional tests were mixed. Examiners and subjects were thus blinded to the categorisation of the tests and were unaware of the intention to divide tests in this way. The brief executive assessment, additional tests, and their respective times of administration are summarised in table 1.

Brief executive assessment

A brief assessment was designed based on knowledge of the pattern of deficits from previous neuropsychological studies in subjects with white matter disease.^5–8,23 Tests were only considered suitable if they were quick and simple to administer, and the brief executive assessment was designed to take no more than 30 minutes, even in severely impaired subjects. Four tests that examine different aspects of executive functioning were included:

• The trail making test was included as a measure of cognitive set shifting and mental flexibility.²⁴ In part A of the test the subject is asked to draw a line to join numbered points scattered randomly over a sheet of paper in numerical order. In part B of the test the test sheet contains points marked by both numbers (1,2,3 …..) and letters (A,B,C….). The subject is asked to join the points with a line, alternating between numerical and alphabetical order (that is, in the sequence 1,A,2,B …….). Part B of the test therefore differs from part A in the requirement to shift between different mental rules, which is described as cognitive set shifting, and is an aspect of executive function. Part B therefore differs from part A in having an executive component, but shares similar non-executive aspects (visual scanning of the paper and motor function in drawing joining lines). Theoretically, subtracting the time for part A from that for part B corrects for differences some of the non-executive aspects, so that this B−A score is considered more specific for executive performance.

• Verbal fluency assesses generativity.²⁵ The subject is asked to produce as many words as possible beginning with particular letters (F, A, and S) in one minute. Executive function is important, as successful performance of this task involves the creation of appropriate strategies for word retrieval.

• Digit span backwards was included as a measure of working memory performance. Working memory refers to the ability to hold and manipulate information “on line” for short periods during cognitive processing, and is often included under the rubric of executive functioning.^15,16 The digit span test includes both forwards and backwards conditions where a subject is given a number sequence and is asked either to repeat it, or to repeat it in reverse order. The backwards condition is heavily dependent on working memory and therefore provides a measure of working memory performance.

• Digit symbol. This is a measure that reflects both performance IQ and executive functioning.²⁶ Each digit (1–9) is ascribed a unique symbol, which the subject is presented in the form of a key. In this test, subjects are presented with a series of digits and are asked to fill in the corresponding symbols in a space below. Subjects are asked to fill in as many consecutive spaces as possible in 90 seconds. This test requires subjects to switch between rules for each digit and therefore requires mental flexibility and has parallels to other set shifting tasks.

Both the trail making and digit symbol tests are timed executive tasks, so that these tests are in theory sensitive both to executive function and the speed of mental processing.

The MMSE was administered as a standard measure of dementia severity and to reflect current clinical practice in screening for cognitive impairment.¹⁸ The MMSE was not included in the brief executive assessment for analysis.

Additional tests

Various additional standard neuropsychological tests were administered, both to extend the assessment to other cognitive domains and to provide additional non-timed measures of executive functioning (Wisconsin card sorting test). Verbal memory was assessed with the Wechsler memory scale (logical memory and paired associate learning subtests²⁷). Verbal and performance IQ were assessed with relevant subtests from the Wechsler adult intelligence scale–revised (WAIS-R)²⁶: verbal IQ was calculated from vocabulary, comprehension, and similarities subtests; performance IQ was calculated from block design and object assembly subtests. To assess the contribution of motor deficits to differences between groups, a specific test of motor function (the finger tapping test) was performed. In addition, the motor control condition (part A) of the trail making test was included.

Statistical analysis

To determine the ability of each test from the brief executive assessment to discriminate ischaemic leukoaraiosis from healthy aging, sensitivity and specificity were calculated using all possible cut off scores. Receiver operating characteristic curves²⁸ were plotted for each test and, in addition, optimal cut off scores were established by adopting a minimum specificity of 75%. Results based on these optimal cut off scores are presented.

Discriminant function analysis was carried out to determine the best possible discrimination that could be achieved for categorising patients in the ischaemic leukoaraiosis or control groups. Separate discriminant functions were calculated for the brief executive assessment and for the whole neuropsychological battery.

This was done by entering relevant test scores into a discriminant function model in SPSS (SPSS Inc, Chicago, Illinois, USA). Discriminant function analysis works by calculating a discriminant score D, from the scores on individual tests as follows:

D  =  aR + bS + cT + d

where R, S, and T are scores on three individual tests and a, b, c, and d are constants. SPSS then iteratively recalculates values of a, b, c, and d until D provides maximum discrimination between predefined groups.

This is achieved by minimising Wilk’s λ, which expresses the proportion of variance in the discriminant function, D, not explained by group differences. A good discriminant function is therefore reflected by a low value of Wilk’s λ, with a λ of 1.0 indicating a discriminatory ability no better than chance. Wilk’s λ was used to compare the discriminatory ability of the separate discriminant functions calculated from the brief executive assessment and the whole battery.

RESULTS

Sensitivity and specificity of brief executive assessment tests

Table 1 shows the neuropsychological raw scores for both groups. Table 2 shows the sensitivity and specificity of each test in the brief executive assessment for differentiating between patients with ischaemic leukoaraiosis and controls. The highest combinations of sensitivity and specificity were obtained with trail making B−A (88% and 76%, respectively) and digit symbol (91% and 76%). In contrast, the MMSE performed less well, with a sensitivity of 63% and a specificity of 76%. Figure 1 shows a sensitivity–specificity plot for trail making B−A in comparison with the MMSE.

The motor control task had markedly lower sensitivity and specificity than trail making B−A and digit symbol (finger tapping test; sensitivity 63%, specificity 76%). In addition the motor control part of the trail making test (part A) was also less effective than part B−A, which is most sensitive to executive function (part A; sensitivity 50%, specificity 76%).

The analysis was repeated in subjects with an MMSE of 28 or above to determine whether the brief executive assessment could identify subtle cognitive deficits. Eighteen patients with ischaemic leukoaraiosis and 16 control subjects had an MMSE of 28 or above. In this subgroup, both trail making B−A and digit symbol remained sensitive and specific, with values of sensitivity of 83% and 89%, respectively, and of specificity, 81% and 81%.

Discriminant function analysis

Table 2 also gives sensitivity and specificity values for the optimal discriminant function that could be derived using the complete brief executive assessment. The ability of the discriminant function to separate groups was significant (Wilk’s λ = 0.54, p<0.001). High levels of sensitivity and specificity were obtained, and the discriminant function was able to categorise 88% of all subjects correctly, and 91% of those with an MMSE of 28 or above.

The best discriminant function was given by D, where:

D = 0.116[digit symbol]−0.075[digit span back]−0.011[verbal fluency]−2.791

The coefficient for trail making B−A was less than 0.001, so this was excluded from the model.

Figure 2 shows the discriminant function derived from the brief executive assessment plotted against age. The overlap between ischaemic leukoaraiosis and control groups was no greater in older subjects and the discriminant function appears to separate groups across the whole age range of subjects. There was no correlation between the discriminant function and age in either group (ischaemic leukoaraiosis, r = −0.16, p = 0.39; controls r = 0.06, p = 0.82). Even in subjects aged 75 years or over, the discriminant function placed 14 of 16 subjects in the correct group with a sensitivity of 100% and specificity of 75%.

Age/education corrected scores

Age and education corrected scores for trail making B had a discriminant ability approximately equivalent to trail making B−A (for which specific age and education corrections are not available; table 2). Correction of verbal fluency scores for age and education did not improve their performance (sensitivity 44%, specificity 76%, area under the receiver operating characteristic curve 0.69, 95% confidence interval 0.53 to 0.84). Substituting these age/education corrected measures for uncorrected verbal fluency and trail making B−A scores in the discriminant function did not improve discriminant performance (Wilk’s λ = 0.54, p<0.001).

Comparison of the brief executive assessment with the whole battery

The additional tests did provide extra qualitative information in some subjects. The most frequent example was identification of deficits in memory retrieval by the delayed condition of the Wechsler memory scale. In addition, some patients performed block design and object assembly tasks poorly and in others verbal reasoning was impaired on “similarities and comprehension” subtests of the Wechsler adult intelligence scale.

To determine how well the brief executive assessment could separate the two groups compared with a comprehensive neuropsychological assessment, a similar discriminant function was calculated for the whole neuropsychological battery (both brief executive assessment and additional tests). Figure 3 shows sensitivity–specificity plots for both the brief executive assessment and whole battery. There is little difference between the plots, indicating that little extra discriminatory information was provided by the additional tests. This was supported by a value of Wilk’s λ that was only marginally less than that obtained with the brief executive assessment (for whole battery, Wilk’s λ = 0.42; for the brief executive assessment, Wilk’s λ = 0.54).

To explore the possibility that the best possible brief assessment might consist of a mix of executive and non-executive tests, a discriminant function was also generated after adding those suitable non-executive tests. Only paired associate learning and WAIS-R block design were considered suitable in terms of time and ease of administration. The addition of these tests did not improve the performance of the brief assessment (with an optimal cut off score, 88% of subjects were correctly assigned to either ischaemic leukoaraiosis or control groups; Wilk’s λ = 0.53).

DISCUSSION

In this study, we have shown that a brief assessment of executive function provided good sensitivity and specificity for discriminating between the cognitive changes of ischaemic leukoaraiosis and those of healthy aging. The brief executive assessment was equally effective in subjects with normal or near normal performance on the MMSE, suggesting that these tests will be a useful adjunct in early disease. It was also able to discriminate between groups across the whole age range studied (50 to 84 years). The discriminant function analysis showed that much of the information about group differences that is provided by a comprehensive neuropsychological assessment could be obtained by the administration of a brief subset of selected tests sensitive to aspects of executive function. The most useful individual tests were trail making and digit symbol. The addition of one of these tests alone to the MMSE would offer a considerable improvement in sensitivity of bedside assessment of cognitive dysfunction.

Sensitivity and specificity values were calculated solely as a means of comparing the value of different tests, and clearly do not apply to the diagnosis of ischaemic leukoaraiosis in practice. The utility of the testing strategy described here will not be as a diagnostic tool, but rather as a means of identifying and measuring cognitive sequelae in patients known to have cerebral small vessel disease. A brief cognitive assessment has several potential applications in settings where detailed neuropsychological assessment is impractical. In the clinical setting, a brief executive assessment would have a role both in screening for cognitive impairment and in identifying cognitive decline during follow up. An effective brief executive assessment would also have applications in clinical trials in vascular dementia or vascular cognitive impairment. Most clinical trials have adopted cognitive assessments that were originally developed to assess treatment effects in Alzheimer’s disease,⁹ such as the ADAS-Cog. Executive dysfunction has a minimal impact on these scales, so that in cerebral SVD these tools may fail to detect beneficial treatment effects. Executive dysfunction is a major cause of morbidity, so detecting treatment effects, even if they are limited to this cognitive domain, should be an important aim of clinical trials.

The entities of vascular cognitive impairment and vascular dementia are heterogeneous,²⁹ encompassing cognitive impairment from multiple large vessel atherothrombotic or cardioembolic strokes as well as SVD. This heterogeneity means that it is unlikely that a single brief cognitive assessment could be designed that would be applicable to all disease subtypes. This study focused on a particular subgroup of patients with cerebral SVD. The issue of cognitive assessment is particularly important in this group because, while SVD accounts for only about 20% of all ischaemic strokes, its impact as a cause of vascular dementia seems to be far greater, with 64% of patients enrolled in a recent clinical trial having extensive white matter abnormalities.⁹

Brief assessment tools are most effective when they focus on a consistent core neuropsychological feature of a condition. One of the difficulties of designing a brief assessment in ischaemic leukoaraiosis is that the neuropsychological hallmarks are less well established than they are in Alzheimer’s disease. One theme that has emerged from studies, both in patient groups^5,30 and in asymptomatic subjects with white matter lesions,^6–8 is impairment of executive function. The most discriminating single tests from the brief executive assessment were trail making and digit symbol. Both of these tests involve cognitive set shifting, a process whereby we are able to move flexibly between different “cognitive sets” or rules. Although both tests are timed, the results suggested that the ability to discriminate between groups was not because of detection of a difference in motor performance, as a test of pure motor speed—finger tapping—was only as effective as the MMSE at discriminating between groups. Furthermore, the trail making test includes a control condition, part A, that incorporates all the non-executive requirements of the test (visual scanning and drawing between numbers), except the need to shift cognitive sets. Trail making part A was far less effective at discriminating between groups than the “executive” parts of the test. Based on the confidence intervals for the area under the receiver operating characteristic curve, no particular measure from trail making B emerged as clearly superior, though the sensitivity and specificity values suggested that trail B corrected for age/education or the difference score, trail B−A, would be most useful. Theoretically, trail B−A is more specific for executive disturbance as it includes correction for motor performance by subtracting the part A score. However, in practice, trail B alone may have better reliability as it relies on just one measurement and is likely to be less prone to measurement error.

Although pure motor speed does not appear to account for performance of trail making and digit symbol, an alternative factor that may be important is mental processing speed. As all of the brief assessment tasks were timed, slowing of information processing could also account for our results without the need to invoke executive processes specifically. Previous studies in similar patient groups have shown impairment on non-timed executive tests such as the Wisconsin card sorting test, but even with non-timed tasks the effect of processing speed cannot be completely excluded, as slow performance places greater demands on the retention of information in working memory and possibly other memory systems.

The next important step in developing assessment methods will be to apply the brief executive assessment developed in this study to larger and more varied groups of subjects with cerebrovascular disease. In particular, the cut off scores and discriminant functions calculated need to be validated in larger, independent samples, and the consistency of the optimal cut off scores across different samples needs to be evaluated. Given the number of cognitive variables and relatively small sample size, the discriminant function coefficients are particularly liable to error, and more accurate estimates in large samples are needed. The cut off scores provided in table 2 are equivalent to performance in the bottom quartile for normal age matched controls. Several additional useful cut off scores would emerge from studies in larger populations, including a clearer picture of the range of impairment in performance seen in patients with ischaemic leukoaraiosis and associated centile scores for different groups. This type of normative data will aid the interpretation of scores for this brief assessment.

Further studies should include comparisons with neurodegenerative dementias, especially Alzheimer’s disease. Although non-executive tests added little to the ability to discriminate between small vessel disease and aging in this study, non-executive elements may contribute significantly to discrimination from other groups. A recent study comparing the neuropsychological profiles of subcortical vascular disease and Alzheimer’s disease suggests that tests of verbal memory, semantic memory, and visuospatial cognition are all potentially useful for discriminating SVD from Alzheimer’s disease.¹³ Prospective studies are also now required to determine the sensitivity of the brief executive assessment, and other tests, to cognitive change over time.