Background Inattention is a core clinical feature of delirium and yet the particular aspects of attentional impairment associated with this feature are poorly understood. Objective methods for assessing inattention are also lacking. A new set of computerised tests of attentional deficits designed for use in patients with delirium have been developed. Test performances in patients with delirium, dementia and cognitively normal controls are compared.
Methods Eight novel tasks measuring sustained visual attention were administered to 20 older patients with delirium using the Edinburgh Delirium Test Box, a purpose built, computerised neuropsychological testing device. Comparison groups of 18 patients with Alzheimer's dementia and 20 cognitively normal patients of similar age were also assessed. Delirium was diagnosed using the Confusion Assessment Method. General cognitive impairment was measured using the Mini-Mental State Examination.
Results Patients with delirium had significantly lower scores than both comparison groups on all sustained attention tasks (p values from 0.003 to <0.001). Performance of the dementia patients generally did not differ from the cognitively normal group. Receiver operating characteristic analyses indicated that all tasks showed good or excellent accuracy for discriminating between delirium and dementia (AUC values 0.80–0.94), and between delirium and cognitively normal (AUC values 0.89–0.99) patients.
Conclusions Patients with delirium showed marked deficits in sustained visual attention, as measured by objective neuropsychological testing. These attentional deficits were mainly mild or absent in patients with dementia and in cognitively normal controls. Objective testing of sustained visual attention has promising utility in detecting delirium, and in discriminating delirium from dementia.
Statistics from Altmetric.com
Delirium (or ‘acute confusional state’) is a severe neuropsychiatric disorder characterised by acute and fluctuating deficits of attention, arousal and cognition. It is highly prevalent in the elderly, hospitalised population1 and is independently associated with increased mortality,2 duration of hospital stay,3 and future cognitive and functional decline.4 Episodes of delirium can be highly distressing for patients, relatives and caregivers.5 Moreover, the duration of the delirious episode correlates with mortality rates 1 year later.6 Early detection and treatment of delirium are therefore crucial for improving patient outcomes.7
Despite the importance of early diagnosis of delirium, it is substantially under detected in clinical settings,8 with as much as 75% of delirium going undiagnosed.9 There are several factors that may contribute to this. Firstly, many of the most commonly used delirium assessment methods rely on subjective clinical judgements and are only suitable for use by experts with considerable training and experience in diagnosing delirium.10 Many of the staff who routinely come into contact with delirium therefore lack the specialist knowledge required to use these assessment methods reliably.11 Secondly, delirium diagnosis is impeded by the degree to which its symptoms overlap with other conditions prevalent in elderly populations, such as depression and the dementias,12 making differential diagnosis difficult. Thirdly, the psychopathological features underlying the symptoms of delirium are poorly characterised, thereby impeding our ability to asses them.13
In a recent review of delirium assessment tools, Kean and Ryan highlighted the need for reliable assessment methods that do not depend on subjective interpretations of patient behaviour.10 One way in which such assessment tools can be developed is by designing objective cognitive tasks that are sensitive to the key neuropsychological markers associated with a disorder.14 A disturbance of attention (or ‘inattention’) is one of the core diagnostic features of delirium.15 While performance on more complex attentional tasks may also be impaired in Alzheimer's dementia, simpler attentional functions, such as sustaining attention to a task over time, appear to be relatively preserved in this disease.16 Neuropsychological tests that are sensitive to these attentional deficits may therefore offer utility for detecting delirium and discriminating it from Alzheimer's dementia.
While the psychopathology underlying inattention in delirium has not yet been well characterised,13 deficits of sustained attention have frequently been implicated. For instance, patients with delirium are known to perform poorly on tasks of auditory vigilance that involve listening to strings of letters and making a response each time a given target letter is heard.17 18 Evidence for similar deficits in the visual domain has also been shown using computerised tasks of sustained attention that require patients to monitor and respond to series of visual stimuli presented on a screen.19 However, in addition to assessing sustained attention, good performance on these tasks also requires relatively rapid perceptual processing of visual information, and the speeded formulation and execution of motor responses. It is therefore possible that patients with deficits in any of these domains may also perform poorly on the tasks, thereby limiting their specificity to detecting attentional deficits in delirium. Indeed, previous studies have shown that patients with Alzheimer's dementia are also impaired on these tasks.20 21
The aim of the current study was therefore to develop simple tests of sustained visual attention that did not involve rapid perceptual processing or responding, thereby making them both sensitive and specific to detecting the symptoms of delirium. In order to ensure that the tasks were suitable for use with patients with delirium, a hand held device (entitled the Edinburgh Delirium Test Box) was purpose built to be suitable for bedside testing of confused patients. We hypothesised that patients with delirium would show significant impairments on one or more of these tasks, indicating the presence of deficits of sustained visual attention. Patients with dementia or no cognitive impairment were expected to have few or no problems with these tasks, making the tasks both sensitive and specific to detecting delirium symptoms.
Three groups of patients were recruited for this study: (1)patients with current delirium and no known dementia, (2)patients with dementia and no current delirium and (3) cognitively normal (control) patients with no known dementia and no current delirium. All patients provided written informed consent prior to taking part in the study which was approved by the Scotland A Research Ethics Committee. As both delirium and dementia are associated with older age, only patients aged over 65 years were approached to take part. Exclusion criteria for all patients were visual or auditory impairments severe enough to affect the reliability of the cognitive testing, and the inability to provide informed consent.
Twenty patients with delirium were recruited from acute medical and surgical wards of a general hospital. Potential participants were first identified by ward staff who named eligible patients who had shown evidence of an acute change in cognitive function since being admitted. The presence of delirium was then formally assessed by one of the trained researchers (CF or HZ) using the Confusion Assessment Method (CAM).22 This diagnostic instrument records the presence of four core features of delirium: (1) attentional impairment, (2) confused thinking, (3) disturbed consciousness and (4) an acute onset or fluctuating course of symptoms. A diagnosis of delirium requires that features 1 and 4, and either 2 or 3 are present.
Scoring of the CAM was informed using a battery of measures specifically validated for this purpose.23 This comprised the Mini-Mental State Examination (MMSE),24 the Delirium Symptom Interview25 and cognitive assessments that required the patient to repeat strings of numbers in a forward or reverse order, and to list the days of the week and the months of the year backwards. Hospital staff and patients' case notes provided additional diagnostic information.
Eighteen dementia patients were recruited from an outpatient memory clinic. All dementia patients had been diagnosed by a geriatrician (JS) as having either Alzheimer's dementia or mixed Alzheimer's and vascular dementia according to ICD-10 criteria.26 Most patients in this group were tested in a clinic room at the hospital. One patient was tested at home.
Cognitively normal (control) patients were recruited from the same hospital wards as the patients with delirium. Patients were included if they had no evidence of dementia or delirium, as judged by the delirium assessments, their medical history and their scores on the MMSE (using a cut-off score of 24/30). Three patients were subsequently excluded from this group for not meeting these criteria, resulting in a final group size of 20.
The presence of delirium was assessed in all patients using the assessment battery described previously. The severity of delirium symptoms was also assessed in all patients using the Memorial Delirium Assessment Scale (MDAS).27 The range of possible scores for the scale is 0–30, with higher scores indicting more severe symptoms. Importantly, patients' performance on the computerised sustained attention tasks was not used to inform either the CAM diagnosis or the MDAS severity rating of delirium.
Sustained attention tasks
Eight sustained attention tasks were administered to patients less than 1 h after their delirium assessment. The tasks were administered using a computerised battery powered device entitled the Edinburgh Delirium Test Box (EDTB) (figure 1), which was custom made by a local electronics company (Eagle Designs: http://www.eagle-designs.co.uk/). The Test Box was devised to be suitable for bedside testing of patients with delirium: being robust and portable, and having large and clearly visible illuminable response buttons. The Test Box was also designed to be simple and rather old-fashioned in appearance, so as not to appear intimidating or off-putting to the patients . All tasks required patients to focus their attention on one or more of the target locations on the Test Box for periods of up to 72 s at a time, and can therefore be considered to be tests of sustained attention.
In order to minimise floor and ceiling effects, eight tasks of differing complexity were designed. Complexity was manipulated by having either one or two potential target locations for the patient to attend to, and by adding task irrelevant distracting stimuli to some tasks. Two different response modes (verbal report and manual button pressing) were also used to minimise potential confounding effects of group differences in either verbal or motor ability.
The tasks were administered to patients in the same order in which they are described below. Tasks with simpler instructions were presented before those with more complex instructions. All eight sustained attention tasks consisted of one unscored practice trial followed by five scored trials. Practice trials were repeated, and additional clarification of instructions was provided when necessary. Each task could be completed within approximately 4–5 min although completion time was longer for some patients.
The first four attention tasks all required patients to count and verbally report how many times one or both of the buttons on the Test Box had illuminated in each trial. In task 1, the Test Box was placed in front of the patient, in a portrait orientation. The patient was instructed to keep watching the button that was located closest to them and to count how many times it lit up. Once the patient has indicated that they understood what to do, the trial was begun. The specified button then illuminated for between five and 14 times in each trial. All illuminations were colourless and lasted for 1000 ms. The duration of the intervals between the illuminations varied between 2050 and 4050 ms. The end of each trial was signalled to the experimenter by a small red dot illuminating on the central display. The experimenter then asked the patient to report how many times the button had illuminated. The task was scored from 0 to 5 according to the number of correct responses given.
Task 2 was the same as task 1 but with the addition of irrelevant distracting stimuli being presented in the central LED display during most trials. These distracting stimuli comprised a series of four or eight alternating red flashes of two lower segments of the LED display. Each flash lasted for 200 ms. There was no temporal gap between one flash ending and the next flash being presented. The distracting stimuli were presented at various time points within each trial, with some occurring at the same time as a target illumination and others occurring between illuminations. The distracting stimuli were designed so as to be clearly distinct from the target button flashes and also to be difficult to perceive as discrete events. Patients were informed about the presence of these distracting stimuli before beginning the task and were instructed to ignore them.
For task 3, the Test Box was positioned in landscape orientation in front of the participant. The task was the same as task 1 except that, for this task, either of the two buttons could be illuminated on each occasion. Patients were informed before starting the task that either button could light up on each occasion and were instructed to count the total number of illuminations. Task 4 was then the same as task 3 but with the addition of the same type of distracting stimuli presented in task 2.
The stimuli in tasks 5–8 followed the same patterns as for tasks 1–4. However, instead of verbally reporting the number of illuminations, patients were now required to press a button each time it illuminated. In these tasks the buttons remained illuminated for 1000 ms or until they were pressed by the participant. Any button press occurring within 100–2000 ms of an illumination was counted as a valid response. Any button presses made outside of this time window, or additional button presses made to the same target stimulus, were classed as being invalid. At the end of the trial the numbers of valid and total responses were displayed in the central display and recorded by the experimenter. The trial was scored as correct if the correct number of valid responses had been made, resulting in a possible score of 0–5 for each task.
Given that the distributions of most of the cognitive test data were non-normal and that variances were not always homogeneous between groups, non-parametric statistics were used to compare all experimental data between groups. Kruskal–Wallis tests were first used to look for overall differences among the three groups. Mann–Whitney U tests were then used to perform pairwise group comparisons. Receiver operating characteristic (ROC) analyses were also conducted for the eight attention tasks to assess the ability of each to discriminate delirium from dementia and cognitively normal patients. Statistical significance was taken as a two sided p value <0.05.
The three groups were well matched in terms of age and sex distribution (table 1). Both the delirium (U=0.0, p <0.001) and dementia (U=15.5, p <0.001) patients scored lower than the cognitively normal patients on the MMSE, indicating higher levels of cognitive impairment. Patients with delirium also scored significantly lower than dementia patients on the MMSE (U=83.0, p<0.01), showing that levels of general cognitive impairment were highest in the delirium group. MDAS scores were higher in the delirium group than in both the dementia (U=3.5, p<0.001) and cognitively normal control groups (U=37.0, p<0.001), showing the expected greater severity of delirium symptoms in this group.
Sustained attention tasks
All patients who began the sustained attention tasks were able to complete task 1. However, as the tasks progressed, the number of patients unable to complete the tasks due to drowsiness, agitation, discomfort or time constraints increased. Furthermore, two patients in the delirium group did not have use of both of their hands at the time of testing and so were unable to complete all of the button pressing tasks. The final numbers of patients in each group therefore differed among tasks (figure 2).
Patients in the delirium group scored lowest in all eight of the sustained attention tasks. Their scores were significantly lower than both the cognitively normal control group and the dementia group (figure 2). Patients with dementia did not differ from controls on any task except task 4 in which they made fewer correct responses than the controls. Task 4 required patients to count illuminations occurring at two locations while ignoring the distracting stimuli, making it conceptually the most complex and challenging of the tasks.
As patients with delirium had lower MMSE scores than the dementia patients, it is possible that their poorer performance on the attention tasks was due to their increased level of overall impairment rather than the presence of delirium per se. In order to assess this possibility, separate post hoc analyses were carried out on the data of 12 delirium patients who had the highest MMSE scores in their group and 12 dementia patients with the lowest MMSE scores who had all completed the first two sustained attention tasks. The patients in these two subgroups did not differ in age and the delirium subgroup still showed the expected higher ratings of delirium severity on the MDAS (table 2). However, there was no difference between the MMSE scores of these two subgroups. Importantly, this subgroup of delirium patients still scored significantly lower than the subgroup of dementia patients on both of the sustained attention tasks compared (table 2). While the relatively small sample sizes involved with these post hoc comparisons is acknowledged, these results provide some evidence that the poorer performance of the delirium group on the sustained attention tasks therefore is unlikely to be accounted for by group differences in levels of general cognitive impairment as measured by the MMSE.
Receiver operating characteristic analyses
The sensitivity of each task for detecting delirium was determined at each possible cut-off score. The specificity at each cut-off score was also determined separately for discriminating delirium from dementia and from cognitively normal controls. Sensitivity and specificity values for optimal three highest cut-off scores for each task are shown in table 3. Positive and negative likelihood ratios are also presented for these cut-off points.
ROC curves were plotted to assess the overall accuracy of each task for discriminating delirium from each of the other patient groups. The AUC of all plots fell between 0.802 and 0.995, indicating that all tasks had either good or excellent accuracy for discriminating delirium from dementia or cognitively normal patients (table 3).
Patients with delirium performed poorly on a series of simple cognitive tasks assessing sustained visual attention. They scored significantly worse than patients with dementia, even when the two groups were matched on overall cognitive impairment. The results of this study demonstrate the potential utility of sustained attention tasks for assessing the presence of delirium for research and clinical purposes. These results also support and extend previous findings suggesting that the neural mechanisms underlying sustained attention are impaired in delirium.
All eight of the sustained attention tasks used showed either good or excellent ability to discriminate delirium from dementia. This suggests that the tasks are both sensitive and specific to the symptoms of delirium and that they may be useful for diagnostic purposes. The EDTB was custom built to be suitable for bedside testing of patients with cognitive impairment; being robust, portable and having large prominent features. The tasks also have simple and standardised administration procedures and can be objectively scored, making them less dependant on clinical judgement or expertise. The EDTB therefore fulfils many of the criteria that are desirable for a diagnostic aid that can be used by a wide range of staff in a hospital environment.10 Larger scale validation studies are now underway to better evaluate the performance of the individual tasks in different patient groups and against existing measures of attention and delirium severity.
The results of this study also provide further insight into the psychopathological mechanisms that underlie delirium symptoms. Previous work has suggested that patients with delirium are impaired at sustaining attention to auditory information.17 18 Recent findings also suggested that similar impairments may occur in the visual domain19 although potential confounding explanations of patients' deficits could not be ruled out. The results of the current study support and extend these findings by showing that patients' deficits on sustained attention tasks are apparent even when the tasks require neither the rapid processing of visual information nor the formulation of speeded responses. Taken together, these studies provide strong support for the notion that inattention in delirium can at least partly be explained as a deficit in sustaining attention to visual or auditory information. The neurophysiological structures and processes that underlie sustained attention, such as the right prefrontal and superior parietal and cortices,28 may therefore be implicated in the pathophysiology of delirium.
The tasks used in this study were designed to be as cognitively simple as possible so that any deficits observed most likely represent genuine deficits of sustained visual attention. Nevertheless, it is impossible to develop any task to be a completely pure measure of any one neuropsychological function and so it is important to consider alternative explanations that could account for the observed patterns of behaviour. As patients were impaired at these tasks regardless of whether verbal or motor responses were required, their poor performance cannot easily be accounted for by specific impairments in either of these response modes alone. Furthermore, as patients with delirium were impaired on these tasks relative even to dementia patients who were matched in terms of overall cognitive impairment, it is unlikely that their deficits simply reflect more general difficulties related to understanding or remembering task instructions. The effects of impaired arousal or consciousness, however, are more difficult to account for, and so it is possible that patients' performance on the tasks may have been worsened by these more fundamental impairments in some cases. Further studies in which patients' levels of arousal are systematically recorded during task performance would be useful for investigating this possibility.
There are some other limitations of the study that should also be acknowledged. Firstly, whereas cognitively normal and delirium patients were recruited from, and tested in, hospital inpatient wards, patients in the dementia group were assessed in an outpatient memory clinic. This recruitment strategy was chosen due to the high prevalence of mixed dementia and delirium that occurs in inpatient settings29 and the difficulties associated with differential diagnosis, thereby making case ascertainment much more reliable. Nevertheless, it is possible that patients with dementia might perform differently on these tasks when tested in an inpatient ward environment. Similarly, the performance of patients with mixed delirium and dementia on these tasks remains to be ascertained.
The requirement for patients to provide written informed consent before taking part in this study also limits the degree to which these findings can be generalised.30 In particular, it is not possible to determine how well the EDTB tasks are able to discriminate between delirium and dementia across the full spectrum of severities (eg, mild delirium versus severe dementia). This issue reflects a more general ethical concern relating to research with cognitively impaired populations, in which both the need to protect the autonomy of vulnerable individuals and the need to preserve their rights to benefit from research advances must be considered.31 The fact that these tasks have now shown promising utility in detecting delirium in patients able to provide written consent, therefore, provides some justification to next examine their performance in more impaired patients, such as those able only to provide verbal assent.
Another limitation of the study is that some patients failed to complete all eight of the sustained attention tasks. This is perhaps unsurprising given that the full set of sustained attention tasks took approximately 35-40 min to administer, which is demanding for a somewhat frail and elderly population. However, all patients who attempted the tasks were able to at least complete task 1, indicating that one task alone is well tolerated by most patients and would be suitable for clinical or research purposes.
In summary, the EDTB tasks used in this study show promise in their ability to detect delirium and discriminate it from some types of dementia. The results also support and extend previous observations that patients with delirium have specific impairments of sustained attention, thereby adding to our knowledge of the neuropsychology of the disorder. Additional studies are now required to assess how well these tasks perform in other clinical patient populations, and to further develop our understanding of the specific neuropsychological deficits that are associated with delirium.
The authors thank Mr Jonathan Adler of Eagle Designs for building and developing the Edinburgh Delirium Test Box, and Mr John Sutherland and Dr Roanna Hall for providing the photograph featured in this article. The authors also thank the patients and staff of the geriatric medicine and orthopaedics units of the Royal Infirmary of Edinburgh and those at the memory clinic in the Royal Victoria Hospital, Edinburgh.
Funding This work was funded by a Medical Research Council Clinician Scientist Fellowship to AMJM. The work was undertaken by the University of Edinburgh Centre for Cognitive Ageing and Cognitive Epidemiology, part of the cross council Lifelong Health and Wellbeing Initiative. Funding from the Biotechnology and Biological Sciences Research Council, the Engineering and Physical Sciences Research Council, the Economic and Social Research Council and the Medical Research Council is gratefully acknowledged. None of the funders of this work had any role in the design of the study, the collection, analysis or interpretation of the data, in the writing of the manuscript or the decision to submit it for publication.
Competing interests This Edinburgh Delirium Test Box and testing protocols described in this manuscript are currently the subject of a patent application.
Ethics approval This study was conducted with the approval of the Scotland A Research Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.