Objective The validity and functional predictive values of the apraxia tests in the Birmingham Cognitive Screen (BCoS) were evaluated. BCoS was developed to identify patients with different forms of praxic deficit using procedures designed to be inclusive for patients with aphasia and/or spatial neglect.
Method Observational studies were conducted from a university neuropsychological assessment centre and from acute and rehabilitation stroke care hospitals throughout an English region. Volunteers from referred patients with chronic acquired brain injuries, a consecutive hospital sample of patients within 3 months of stroke (n=635) and a population based healthy control sample (n=100) were recruited. The main outcome measures used were the Barthel Index, the Nottingham Extended Activities of Daily Living Scale as well as recovery from apraxia.
Results There were high inter-rater reliabilities and correlations between the BCoS apraxia tasks and counterpart tests from the literature. The vast majority (88.3%) of the stroke survivors were able to complete the screen. Pantomime and gesture recognition tasks were more sensitive in differentiating between individuals with left hemisphere damage and right hemisphere damage whereas the Multistep Object Use test and the imitation task had higher functional correlates over and above effects of hemiplegia. Together, the initial scores of the four tasks enabled predictions with 75% accuracy, the recovery of apraxia and independence level at 9 months.
Conclusions As a model based assessment, BCoS offers a quick and valid way to detect apraxia and predict functional recovery. It enables early and informative assessment of most stroke patients for rehabilitation planning.
Statistics from Altmetric.com
Limb apraxia is one of the family of apraxic disorders generally defined as a non-motoric deficit reflecting impaired purposive upper limb movement.1 It is a common and important2–5 but relatively poorly understood6 7 sequelae of stroke which can persist well after the acute phase.8 9 There is also evidence that apraxia is a significant predictor of functional motor skills10 more so than aphasia or primary motor deficits.11 Given this link to functional performance, an early assessment that informs treatment is invaluable. However, a review of current cognitive assessments suggests that there is no accepted screening for apraxia.12
Cognitive models describe the functional architecture of the praxis system and illustrate the different ways that apraxia can arise. One influential model is that put forward by Rothi and colleagues13 (see Cubelli and colleagues,14 Chainay and Humphreys15 and Goldenberg16) which has also been used to account for data from functional imaging studies of healthy participants.17 Figure 1 illustrates the cognitive mechanisms of the different commonly used praxis assessment tasks using a model incorporating recent extensions of Rothi's original model.17 In particular, the addition of the ‘body part coding’ mechanism for the imitation of meaningless gestures, which converts the visual appearance of the demonstrated gestures by an examiner to spatial relationships between discrete body parts.16 17
A cognitive theory driven approach to assessment is appealing. It can provide a diagnosis of specific deficits and can lead to appropriate targeting of treatment. For this, several batteries of apraxia tests have been developed recently.18 19 However, there can be problems for day to day clinical application. A comprehensive apraxia assessment can be time consuming,13 and the absence of consistent scoring criteria across different tasks may hinder meaningful interpretation of the performance profile. Moreover, as spatial neglect is prevalent among stroke patients with right hemisphere lesions20 and aphasia among stroke patients of left hemisphere lesions,21 and both impairments are linked to sites in the posterior parietal cortex also linked to apraxia,1 then assessment procedures need to facilitate the participation of patients with aphasia and spatial neglect to maximise involvement.
To address these issues, a novel set of model based apraxia tests were created as part of the cognitive screening battery (the Birmingham Cognitive Screen (BCoS)). This paper describes the development, validation and utility of these tests. An initial validation study, carried out with a set of patients with chronic lesions, examined inter-rater reliability and construct validity for the assessments relative to other published tests. Subsequently, we assessed the utility of the apraxia tests in BCoS for patients in their early stage of recovery from stroke, along with the functional relations between the BCoS measures and assessment of functional activities (eg, the Barthel Index).
Phase 1 validated inter-rater reliability and the construct validity of the BCoS apraxia tests with a sample of participants with established brain injuries. Phase 2 examined the utility and ecological validity among participants at a subacute stage after stroke.
Phase 1 participants were individuals with acquired brain damage who were either self-referred or referred by others (eg, hospital consultants) and attended the School of Psychology regularly for neuropsychological assessments and rehabilitation (hereafter referred to as ‘chronic acquired brain injury’—chronic ABI—participants). They were selected to represent a wide range of praxis abilities. Eighteen chronic ABI participants took part in the construct validity studies. Eight individuals took part in the inter-rater study and had their performance videoed for scoring by different raters. Two trained examiners independently scored performance on the BCoS tests using the instructions given in the examiner booklet. No communication took place between the raters, and their scores were subsequently compared.
Phase 2 data collection was conducted by trained examiners from the university and from the West Midlands Stroke Research Network who conducted weekly visits to the stroke wards in 12 local hospitals as part of a cognitive screen trial (the Birmingham University Cognitive Screen Trial, http://www.bucs.bham.ac.uk). In addition to assessments of apraxia, BCoS measures performance across four other cognitive ‘domains’: language, spatial and controlled attention, memory and number processing. All tests in the battery are designed to optimise time efficient data collection and to maximise patient inclusion. Any stroke survivors who were medically stable, within 3 months of their latest stroke and able to give consent, were approached. Six hundred and thirty-five participants recruited between November 2006 and September 2010 were entered into the study (hereafter referred to as ‘hospital patients’). Lesion information from hospital based axial CT scan reports was available for 80% of the hospital patients and revealed definitive lesions in 93% of patients who were scanned. A subset of 253 hospital participants were invited to take part in a 9 month follow-up assessment.
Healthy controls (n=100) were recruited according to the 2001 UK population census age × sex × education level distribution. The 18 university participants and 14 controls were also assessed with conventional apraxia tests.
Informed consent was obtained according to the approved ethics protocols of the UK National Research Ethics Committee and the Birmingham University ethics procedures.
Selection of the praxis screen tests in BCoS was governed by the need to minimise administration time while maximising inclusion and attempting to be as sensitive as possible. The aim was to assess the cognitive processes that support praxis and so tests covered: (1) input processing of visually conveyed gestures, (2) the coding of body part and position, (3) access to stored knowledge about the meanings of gestures (action semantics) and (4) access to appropriate motor output routines which transform spatiotemporal concepts of gestures into motor innervations, to produce gestures (gesture output processing) (figure 1). To assess these processes, BCoS contains three praxis tasks: pantomime to auditory/written words input (pantomime), forced choice recognition of pantomime (recognition) and imitation of meaningless gestures (imitation). Table 1 summarises analyses of deficits with different performance profiles. In addition to the assessments using single actions, a further test of Multistep Object Use (MOT) (assembling and then switching on a torch) was included to assess patients' ability in the use of objects in an everyday situation. This allowed us to assess step sequencing to achieve a goal along with the ability to select an appropriate object among distractor objects. The MOT task has added functional relevance as studies have shown dissociable abilities between everyday object use and praxis tasks.15 22 All tasks are illustrated in figure 2 (see appendix I, available online only, for administration and scoring details). Moreover, several principles were followed in the selecting and designing of both the tasks and items to make the screen rigorous (box 1).
Principles in apraxis tasks selection and design
To be inclusive of different gesture types which could be differentially impaired in patients—hence both transitive (object related) and intransitive (symbolic) gestures were included in both the pantomime and recognition tasks, while the imitation task required coding and production of both hand position and finger postures.
As many apraxic patients also have hemiplegia, only actions that could be performed with one hand were selected for the pantomime and imitation tasks.
For aphasic patients, instructions used high frequency words and were presented multimodally where possible—in written form, verbally and with photographic illustration to maximise understanding; in addition, forced choice responses were used for the recognition task, with no verbal production demands on patients.
All visual stimuli (objects, pictures, words, gestures) were presented vertically across the midline in front of the patient, to rule out the potential confounding effects of spatial neglect.
Conventional praxis tasks
To assess pantomime, the Florida Apraxia Screening Test23 was adapted. For gesture recognition, the principles of Peigneux et al's design24 were adopted. For the conventional imitation test, De Renzi et al's Movement Imitation Test2 protocol was used. For the object use assessment, a procedure based on De Renzi's Single Object Use test (SOT)25 was employed (see appendix I, available online only, for administration and scoring details).
Within the protocol of the BCoS study, hospital participants were assessed initially with the Barthel Activities of Daily Living (ADL) Index,26 a well established and validated functional measure for acute stroke patients.27–30 At the 9 month follow-up assessment, the Nottingham Extended ADL scale (NEADL)31 was added to measure higher level community based daily activities. The presence of hemiplegia was recorded when participants could only use one hand to complete the bilateral MOT task (see appendix I, available online only).
Descriptive analyses were conducted for all measures by group. Inter-rater reliability was examined by Pearson's correlations and the kappa test. Sensitivity and specificity were evaluated using binary classification, as described by Greenhalgh.32 Log linear regression was conducted for the analysis of multiple categorical variables in a model. Between two groups, χ2 tests were used to compare categorical data and t tests were used to compare interval data.
The functional analyses focused on the effect of the presence and recovery of apraxia on ADL outcomes. The presence of apraxia was defined by an impairment in any one of the praxis tasks. Based on the assumption that some action processes recruited by each task are specific to that particular task, we also conducted task by task analyses to examine task specific effects. To investigate the relations between recovery from apraxia and functional outcome, the follow-up participants were divided into three groups: (1) those with no apraxia (no impairment in any one task); (2) those recovered from apraxia (failed at least one task on initial assessment but passed all tasks on follow-up); and (3) those with persistent apraxia (failed at least one task even on follow-up). First, discriminatory function analyses were used to assess the predictability of the four apraxia task scores on recovery from apraxia at 9 months. This was followed by correlations and ANCOVA analyses for ADL outcomes. Post hoc comparisons were conducted with the Sidak correction.
Participant characteristics and praxis abilities
Table 2 details the demographic characteristics and praxis abilities in each of the study groups. The 100 controls were divided into three age groups (<65 years, 65–74 years, >74 years), each with gender and education profiles comparable with those of the 2001 census population profile. Cut-off scores for each of the BCoS apraxia tests were set at the 5th percentile of control performance (table 3).
Phase 1 studies
The pantomime and imitation scores given by the raters were significantly correlated (r=0.94 and 0.86, respectively). Using the cut-off points from control performance, there was 100% agreement between the raters on impairments for the two tasks. For the MOT, the percentage agreement on individual error scores was 92.7%, resulting in a κ value of 0.82 (p<0.001). The total scores given to the performance of the chronic ABI participants on each task were highly correlated across the raters (0.99; p<0.001). Three of eight participants failed the torch task and the resulting agreement on impairment was 100%.
Sensitivity and specificity
The standard deviations in the conventional praxis tasks (table 2) suggested that the chronic ABI participant group represented a wide range of praxis abilities. The variability was maintained in performance on the BCoS praxis tasks. This confirms that the praxis screen can distinguish different levels of patient performance. All screen tasks correlated well with their conventional counterparts, except for the MOT (against the SOT) (table 4). Apart from the MOT, all of the tasks in the screen demonstrated substantial sensitivity and specificity values (table 4). However, the wide 95% CIs call for further studies with a larger sample size. Corresponding to the poor correlation between the SOT and MOT tasks reported above, low sensitivity (50%) was demonstrated. These data suggest that performance on the MOT and SOT tests might be supported by different cognitive processes (see discussion below).
Phase 2 studies
One of the aims of the BCoS assessment was to maximise the inclusion of patients, including patients with aphasia and/or visuospatial deficits. Out of the 635 hospital patients, 585 (92.1%) were assessed on at least one of the apraxia tests and 561 (88.3%) completed all four apraxia tasks. Only one patient was unable to participate in some praxis tasks due to aphasia and no patient had a problem participating due to visuospatial neglect. This confirmed that the BCoS is indeed aphasia and neglect friendly.
Assessments were performed within the first week post stroke for 22.0% of patients and 81.6% were assessed within 6 weeks post stroke (this is the period proposed by the UK National Institute for Health and Clinical Excellence Quality Standards for Stroke 2010). The mean ‘period after onset’ on testing was 25.2 days (SD 21.3).
To allow comparisons across test sessions and individual tasks, table 5 summarises the task specific findings for the subgroup who completed all four tasks and were followed-up (n=231). In this group, 46% were impaired in at least one praxis task. Therefore, using only one praxis task would result in 13–30% under detection of impairments. On the other hand, there were individuals who were impaired in only one out of the four praxis tasks (per cent impairment in single task). Such unique impairments occurred in each of the four tasks and among 22% of the patients in the initial session and 18% of the patients in the follow-up session, indicating dissociations of processes underlying each task.
Praxis performance data were analysed and contrasted across patients with unilateral lesions (left vs right) who were tested for apraxia. Left hemisphere damage (LHD) was identified in 74 patients and right hemisphere damage (RHD) in 84. A four way log linear analysis of assessment session (initial vs follow-up), tasks (the four praxis tasks), lesion (left vs right) and impairment (impaired vs not impaired) produced a final model (that showed significant interaction only at the three way level of tasks × lesion × impairment) (χ2(3)=13.97, p=0.003). This suggested that the profile structure was maintained across the assessment sessions. Across assessments, only pantomime and recognition tasks consistently showed a differential impairment profile between the LHD and RHD groups (table 5).
Prediction of recovery
Of those who showed signs of apraxia in the first assessment, 107 participated in the follow-up assessment where 51 (48%) showed no further sign of deficit across all tasks (recovered) and 56 (52%) showed persistent signs of apraxia. Based on their initial scores for the four praxis tasks, discriminant function analysis was conducted to predict praxis recovery (recovered vs persistent, see definitions above) at 9 months. Using the four initial scores, 75% of cases were correctly classified in the recovered group (centroid=0.489) and the persistent group (centroid=−0.445). The discriminating power was significant (Wilks' λ=0.82, χ2(4)=20.62, p<0.001). The loading correlations between the predictor variables and the discriminant function were 0.86 for the pantomime task, 0.54 for the imitation task, 0.51 for the MOT and 0.28 for gesture recognition, suggesting that the first three tasks contributed more in differentiating between the recovered and persistent groups than the gesture recognition task.
The presence of apraxia at the initial assessment (an impairment in any one of the BCoS apraxia tasks) related to lower mean Barthel score both measured initially (mean (SD)=10.92 (5.67) vs 15.04 (5.13)) and at 9 months (16.00 (4.83) vs 17.83 (3.51)). With hemiplegia partialled out, the effect of apraxia on the Barthel remained significant (initial: F (2,513)=57.85, p<0.001; 9 months: F (2,214)=8.81, p=0.003). The same result arose for the NEADL at 9 months (mean (SD)=11.08 (6.96) vs 14.40 (5.94), F (2,218)=13.05; p<0.001).
Moreover, with hemiplegia again controlled for, the scores of each initial praxis assessment were correlated with the initial Barthel, follow-up Barthel and the NEADL (table 5). At the p=0.001 level, these showed low but consistent correlations between the MOT and the imitation with functional assessments (Barthel at initial assessment and NEADL at follow-up), explaining between 7% and 10% of the variance.
For those who were assessed, 124 (54%) had no apraxia, 51 (22%) recovered from the signs of apraxia and 56 (24%) showed persistent signs of apraxia. Classifying the patients as having no apraxia, recovered apraxia or persistent apraxia, ANCOVAs examined the impact of apraxia on everyday abilities at different stages of recovery, with the effects of hemiplegia (covariate) partialled out (table 6). Both the recovered and persistent apraxia groups had significantly lower initial Barthel scores compared with the non-apraxic group (p<0.001) but they did not differ from each other (F<1.0; indeed the recovered patients, if anything, had lower initial scores). The difference disappeared for the follow-up Barthel scores. However, the persistent apraxia group were still significantly lower in the NEADL score at follow-up relative to the no apraxia group (p=0.002).
The past 20 years have seen substantial advances in our theoretical understanding of the neural and cognitive mechanisms of apraxia.13 16 17 33 The apraxia tests within the BCoS were developed to capitalise on these advances to enhance identification and understanding of praxis deficits in clinical settings. The current study shows promising results for these tests in terms of inter-rater reliability and construct validity although further studies on this are desirable as the current validation and reliability tests were based on relatively small numbers. To further ensure standardisation of practice, the study team have additionally developed videos with testing demonstrations and example performances to give consistency to training.
The tasks were designed to overcome the assessment barriers imposed by aphasia, neglect and hemiplegia. As a result, it was possible to administer the screen to the vast majority of the hospital participants very early on after their stroke. Signs of impairment in the individual praxis task ranged from 16% to 33%. Taking into account performance across all tasks, BCoS identified signs of praxis deficits in 46% of the acute hospital stroke survivors (62% in the LHD, 37% in the RHD group). These figures are somewhat higher than other studies.2 3 5 The difference can partly be accounted for by our relatively more acute patients (mean days post stroke=25.2) whereas the other studies recruited participants at a later stage; in addition, our use of procedures to include patients with neglect, aphasia and hemiplegia may have led to a greater range of deficits being sampled. In our 9 month follow-up assessment, about half of the apraxic patients had recovered which would have brought the impairment rates to a more comparable level to the other studies. Moreover, in an attempt to capture all types of praxis deficits, we included a larger range of tasks than the previous studies, all of which have functional implications (see discussion below). The rate of impairment in our sample was particularly high in RHD patients compared with other studies. This could be due to our inclusion of the meaningless gesture imitation and the MOT. With the imitation task, the meaningless nature of the gestures and the inclusion of trials where finger postures had to be reproduced may lead to significant contributions from the right hemisphere for high level visuospatial analysis (although the LHD patients remained worse).34 35 Meaningful gestures (via the semantic route, requiring retrieval from the long term memory store) and the hand to head postures may rely more heavily on the left hemisphere processing of body structure.34 With the MOT, impairment rates did not differ across the two unilateral lesion groups. This task simulates a naturalistic situation requiring multiple cognitive abilities (eg, sequencing, selection of correct objects from distracters, online spatial analyses in assembling different components of the torch, as well as higher level attentional control). Therefore, it is no surprise that both hemispheres contribute to satisfactory performance.36
Our findings on the lack of correlation between the SOT (a common test of apraxia) and the MOT (a more naturalistic assessment) present in the BCoS confirms earlier findings37 and supports the notion that MOT and SOT call on at least some distinct processes.38 For example, the contextual information from related object in the MOT might cue relevant actions for some patients who are impaired in the SOT, while the presence of distracters and the demand on sequencing abilities might lead to more opportunities for errors in other patients (eg, those with frontal lobe lesions). It can also be argued that unlike MOT, SOT can be performed without access to semantic memory, working memory and attentional processing; impairments in these ancillary cognitive mechanisms could differentially affect MOT performance.
Measures of apraxia related to functional outcome, both initially and at follow-up. This supports the argument for early diagnostic testing of apraxia after stroke. One might wonder if early assessment would overdiagnose problems (eg, failure of praxis assessment due to the effects of post stroke fatigue39) which could be spontaneously resolved at a later stage. Against this, our data showed that information obtained from the early assessment of praxis was indicative of sustained deficits at 9 months over and above the effects of hemiplegia. The mean initial Barthel scores of the apraxic group (10.92) fell within the moderate dependency level whereas those of the non-apraxic group (15.04) fell within the low dependency.40 This difference has implications for the costs of health and social care in the first year after stroke.41 In our follow-up assessment, patients with persistent apraxia were still disadvantaged in the more community based, higher level ADL detected in the NEADL scale (eg, making a hot drink). Interestingly, patients with persistent problems were not necessarily worse at the initial assessment, suggesting that functional recovery is not simply related to the overall level of impairment.
As well as the value in predicting recovery from apraxia and functional outcomes when used together, each of the apraxia tasks can provide information about different aspects of a patient's action related ability which can be used for clinical reference and/or targeting rehabilitation. The ecological relevance of the MOT has already been discussed above. For stroke survivors with aphasia (ie, difficulties in producing and/or understanding speech), measures of gesture production and recognition would allow the therapist to ascertain if patients can reliably produce and understand gestures as an alternative means of communication. In addition, imitation of meaningless gestures can inform the rehabilitation team about a patient's ability to relearn movement patterns for everyday activities when patient can no longer access the meaning and functions of such movements from stored knowledge.42 Therefore, the BCoS apraxia tests have not only theoretical significance but also functional relevance. To further inform clinical applications, future investigations are required to (1) evaluate the use of training videos on improving the reliability between raters, (2) investigate the different recovery patterns between LHD and RHD patients, (3) examine the interactions between praxis abilities and other cognitive functions as well as (4) the relationship between different praxis abilities and specific different types of ADL activities.
The authors thank all of the patients and controls who participated in the BUCS study, the University of Birmingham BUCS research team, the rehabilitation teams who assisted at the collaborating sites (listed on http://www.bucs.bham.ac.uk) and the West Midlands Stroke Research Network staff without whom the study would not have been possible.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Download Supplementary Data (PDF) - Manuscript file of format pdf
Funding This study was funded by the Stroke Association, UK, and the West Midlands Stroke Research Network.
Competing interests None.
Ethics approval The study was approved by the UK National Research Ethics Committee (Essex 1).
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.