Article Text

PDF

Evaluating methods for estimating premorbid intellectual ability in closed head injury
  1. Kathryn J Watta,
  2. Ronan E O’Carrollb
  1. aDepartment of Clinical Psychology, St John’s Hospital, Howden Road West, Livingston, West Lothian EH54 6PP, Scotland, UK, bDepartment of Psychology, Stirling University, Stirling, Scotland, UK
  1. Dr Ronan E O’Carroll, Department of Psychology, Stirling University, Stirling FK9 4LA, UK. email: reo1{at}stir.ac.uk

Abstract

OBJECTIVES The present study examines the utility of three measures of premorbid intellectual functioning in closed head injury, the National adult reading test (NART), the Cambridge contextual reading test (CCRT), and the spot the word test (STW).

METHODS In the first experiment, a group of 25 patients with closed head injury were compared with 50 healthy controls and 20 orthopaedic trauma controls. In the second experiment, the strength of correlation between the premorbid measures and current intellectual level were assessed in 114 healthy adults.

RESULTS The head injured group performed significantly more poorly than both control groups on measures of current intellectual ability. However, no significant differences emerged between the groups on any of the premorbid measures. In the large control sample, both the NART and the CCRT accounted for about 50% of the variance in current verbal intelligence. However, by contrast, the STW only accounted for 29% of the variability in verbal intelligence. Adding demographic variables to the prediction of current intellectual level increased the amount of variance explained to 60% for the NART, 62% for the CCRT, but only 41% for the STW.

CONCLUSION The results provide supportive evidence for the use of the CCRT and NART in estimating premorbid intellectual functioning in patients who have sustained closed head injuries, but suggest caution when employing the STW.

  • NART
  • CCRT
  • STW
  • assessment
  • cognitive
  • premorbid

Statistics from Altmetric.com

Neuropsychological assessment of patients with acquired brain injury is critical because such cases are common causes of enduring significant disability. Although neuropsychological tests are designed to provide precise information about the level of cognitive functioning at a given point in time, test results must be contrasted with the estimated level of functioning before the injury. The discrepancy between current and premorbid estimates of function is essential, and provides an indication of the degree of deterioration compared with an individually controlled preinjury standard. Traditionally, premorbid ability has been estimated via indirect sources. For example, the vocabulary subtest of the Wechsler adult intelligence scale (WAIS)1was widely used as it was assumed to be relatively resistant to the effects of acquired brain damage. However, several recent studies have suggested that vocabulary is not immune to such effects.2-4 Similarly, intellectual level has been estimated using equations derived from demographic variables—for example, Wilson et al 5 produced a formula based on the WAIS standardisation sample which predicted 54%, 53%, and 42% of the variance in WAIS full scale verbal and performance IQ, respectively. Crawford and Allan6 recently showed that demographic variables accounted for 54% of WAIS-R variance in a large (n=200) United Kingdom sample. However, other studies have suggested that the percentage of variance explained may be less impressive when using demographic equations alone in clinical samples.7 8 For example, Perez et al 9 have recently looked at the potential of demographic regression equations to predict premorbid intellectual level in patients with head injury. They reported that the correlation between the equations and WAIS-R verbal IQ wasr=0.52. However, clearly such an approach only accounts for 27% of the variance in observed intelligence, leaving a vast amount of error variance. In attempting to improve on vocabulary and demographically based methods of estimating premorbid intellectual level, Nelson and McKenna10 suggested that reading ability may provide a more accurate estimate. Johnstoneet al 11 used reading ability from the wide range achievement test-revised (WRAT-R) to estimate premorbid ability in 97 outpatients who had had traumatic brain injury. They found the WRAT-R to be a useful measure of premorbid ability, and highlighted the greatest relative reductions in cognitive flexibility after head injury. Nelson and O’Connel12 developed the new adult reading test, later to be renamed as the National adult reading test (NART13), and revised as the revised National adult reading test14 based on their finding that reading ability—namely, accuracy of oral pronunciation—was relatively well preserved in dementing patients until relatively late on in the disease process. The NART comprises oral reading of 50 irregular words, and accuracy of pronunciation is scored. Various studies have indicated that NART performance is highly correlated with general intellectual level and relatively resistant to the effects of varying forms of brain damage (see reviews7 15) Crawford et al 4 carried out a large study which looked at NART performance in various organic conditions, including the performance of 18 subjects with closed head injury. No significant difference in NART performance was found between the closed head injury group and well matched controls. Subsequently, Moss and Dowd16 reported a single case study of a man who sustained a severe head injury in a road traffic accident, on whom an intelligence test had, fortuitously, been administered before the accident. Neuropsychological evaluation including the NART to estimate premorbid IQ was carried out after the accident for medicolegal purposes. The NART was found to produce a very accurate measure of premorbid IQ in this case, supporting its validity in predicting premorbid intellectual level in patients who have sustained a closed head injury.

The NART has become firmly established as the method of choice for estimating premorbid intellectual level in English speaking countries throughout the world. However, modifications and alternative measures have been proposed in recent years. Crawford et al 17 18 have proposed that adding demographic variables improves the amount of explained variance when using the NART. Furthermore, Beardsall and Huppert19 have criticised the presentation of irregular words in isolation. They proposed that many people who fail on the NART may be able to accurately pronounce such words in context. They developed the Cambridge contextual reading test (CCRT), in which NART words were placed in context.19Both normal and demented subjects significantly improved their performance when words were placed in context (meaningful sentences).

However, estimating premorbid intellectual level on the accuracy of pronunciation of irregular words obviously has limitations. This procedure cannot be used for those who have dyslexia, visual acuity problems, or articulation problems. Furthermore, both the NART and CCRT may underestimate the intelligence of self educated people who have obtained their vocabulary knowledge through private study with the result that they may be familiar with the word but do not know how to pronounce it. Accordingly, Baddeley et al 20 developed the spot the word test (STW). The STW consists of 60 pairs of words and has two parallel forms to overcome the difficulties of practice effects. The subject merely has to choose between each of the two words in the pair to determine which is a real word and which is a pseudo-word.

The aim of the current study was to compare these three methods of estimating premorbid intellectual ability in patients who have had a closed head injury. Two experiments were carried out.

Subjects and methods for experiment 1

Three subject groups were recruited—healthy controls, patients with head injury, and orthopaedic patients without head injury. The sample size was determined by the number of patients with head injury and orthopaedic patients who agreed to participate within the time frame of the study.

Twenty five patients who had sustained a closed head injury were recruited. All subjects had been injured at least 9 months before the start of the study and were between 18 and 55 years of age. The head injury group consisted of 15 patients (60%) who had had a severe head injury (Glasgow coma scale on admission 3–8), six (24%) who had sustained a moderate head injury (Glasgow coma scale 9–12), and four (16%) who had sustained a mild head injury (Glasgow coma scale 13–15). The average time since sustaining the head injury was 23.8 (range 9–51) months. A second group of patients were recruited who had been admitted to orthopaedic wards. Exclusion criteria for the orthopaedic control group included a history of neurological trauma such as head injury resulting in loss of consciousness. The orthopaedic control group consisted of 20 patients aged between 20 and 60 years of age and were included to provide a control for the non-specific effects of injury or trauma. Finally, a healthy normal control group of 50 subjects was recruited from community groups, non-medical health service personnel, and members of the general public aged between 18 and 65. Again, healthy control subjects were included only if they did not have a history of neurological trauma resulting in unconsciousness or a history of psychiatric illness. All control subjects were screened to exclude the possibility of acquired cognitive impairment using the mini mental state examination21; subjects had to score 24/30 or greater. The demographic details of the healthy control group, head injury group, and the orthopaedic control group are presented in table 1. Socioeconomic status was derived from a subject’s occupation using the Office of Population and Censuses and Surveys 1991 classification of occupations. All subjects underwent a visual acuity screening test which consisted of reading aloud 10 simple regular words. Thereafter, in a single session, all subjects were administered the following measures in the same standard order: the NART-R,14 the CCRT,19 the STW,20and a short form of the WAIS-R.22 A previous study had indicated that there was no significant order effect in relation to NART and CCRT administration.23 We adopted the short form of the WAIS-R recommended by Crawford et al,24 which consists of the subtests of comprehension, similarities, block design, and object assembly. The mini mental state examination (MMSE),21 FAS controlled oral word association,25 and the Stroop task26 were then administered; the last two measures of executive function were chosen as they are commonly reported as being particularly sensitive to the effects of closed head injury.27 Finally, measures of affective status were taken using the hospital anxiety and depression scale.28 The project was fully approved by the local clinical psychology and psychiatry ethics committee.

Table 1

Demographic information

STATISTICAL ANALYSIS

All data were tested for normality of distribution using the Kolmogorov-Smirnov test. No variables differed significantly from the normal distribution, therefore parametric analyses were employed. One way analysis of variance (ANOVA) was employed for between group comparisons on demographic and affective measures and χ2analysis was employed for categorical data. Pearson’s correlation coefficient was used to test the strength of associations between variables. A MANCOVA was used to test for between goup differences in current and premorbid measures controlling for potential confounding variables. Finally, forward stepwise multiple regression equations were used to determine the best predictors of current intellectual status in the healthy controls. All statistical analyses were conducted using SPSS version 6 for the Apple Macintosh.

Results for experiment1

Univariate analysis of variance disclosed that there was no significant differences between the groups for age and χ2analysis showed no differences for social class. However, the healthy controls had significantly more years of full time education and, unsurprisingly, the head injury group comprised a relative excess of men (table 1). There were no significant differences between the groups on HAD anxiety. However, the head injury group showed significantly raised HAD depression scores (table 2). HAD scores can be classified as normal (<8), borderline (8–10), or morbid (11+). Using these categories for HAD anxiety, the relative percentages for the head injury group were 52% normal, 16% borderline, and 32% morbid. For orthopaedic controls, the figures were 45%, 55%, and 0%, and for healthy controls, 60%, 28%, and 12% respectively. Turning to HAD depression, for the head injury group, the precentages were 52% normal, 24% borderline, and 24% morbid; for orthopaedic controls, 95%, 0%, and 5%; and for healthy controls, 94%, 6%, and 0% respectively. To test whether these between group differences were likely to be important sources of confounding variance in analysing the neuropsychological measures, a correlational analysis was carried out on the entire sample between years of full time education and depression scores and the measures of premorbid and current intellectual functioning (table 3). Significant correlations emerged between education and depression scores for both the premorbid and current measures of cognitive functioning.

Table 2

Between group comparisons of HAD anxiety and depression levels

Table 3

Correlational analysis between years education and depression and the measures of premorbid and current intellectual functioning

After the results of the correlational analyses, a further set of correlations were run restricted to the 25 patients with head injury to investigate the relation between depression and premorbid measures. The correlations between HAD depression and the premorbid indices were as follows: CCRT, r=0.405, p=0.044; STW, r=0.386, p=0.057; and NART,r=0.37, p=0.069. Similarly, correlations were run between depression and the premorbid indices for the control groups combined and the results were as follows: CCRT, r=0.213, p=0.077; STW,r=0.141, p=0.244; and NART,r=0.148, p=0.221. This shows that the association between depression severity and premorbid measures was largely accounted for by the patients with head injury and not the control subjects. Accordingly, for all subsequent between group comparisons of premorbid and current intellectual measures, years of full time education, sex, and HAD depression scores were entered as covariates. Two separate multiple analyses of covariance were carried out, firstly for the premorbid measures (NART, CCRT, and STW errors) and secondly, for the current measures of intellectual functioning (WAIS-R full scale, verbal, and performance IQ, MMSE, verbal fluency and Stroop). The overall MANCOVA for the premorbid measures was non-significant and similarly, none of the univariate between group comparisons for NART, CCRT, or STW were significant (table 4). No significant correlations emerged between Glasgow coma score and each of the premorbid measures (NART v GCS,r s=−0.15; CCRTv GCS,r s=−0.18; STWv GCS,r s=−0.19). By contrast, the overall MANCOVA for the current measures of cognitive function was highly significant, as were all the univariate comparisons for the WAIS-R measures, MMSE, verbal fluency, and Stroop (table 4). Post hoc Scheffe analysis showed that the head injury group’s scores on all of the measures of current intellectual functioning were significantly lower than both the healthy controls and the orthopaedic control group, and that there were no significant differences between the two control groups.

Table 4

Between group comparisons of measures of premorbid and current intellectual functioning and psychological distress controlling for the effects of years education, sex, and depression (mean adjusted scores)

To test the relative benefit of placing the NART stimulus words in context (the CCRT condition), a within subject comparison of NART versus CCRT performance was conducted for closed head injury versus controls. Given that no significant differences in cognitive test performance were apparent between the healthy and orthopaedic control subjects, they were combined to form a single control group. “The closed head injury group made significantly fewer errors in the CCRT versus NART condition (CCRT errors=22.0 (SD 7.5), NART errors=26.7 (8.7), mean difference score=4.7, pairedt=9.35, p<0.0001). Similarly, the combined control group made fewer errors in the CCRT condition (CCRT errors=17.0 (7.8), NART errors=20.2 (8.7), mean difference score=3.2, pairedt=10.84, p<0.0001). However, the closed head injury group derived a significantly greater benefit than the control group when the words were placed in context; a repeated measures MANOVA showed an overall effect of subject group (F=9.20, p=0.003) an overall test effect (NART v CCRTF=184.1, p<0.0001), and a significant group by test interaction (F=5.92, p<0.017).

Discussion of experiment 1

The results of experiment 1 indicate that after controlling for the effects of years of full time education, sex, and depression level, no significant differences emerged between the groups on any of the measures of premorbid intellectual functioning. This result suggests that all three putative estimates of premorbid intellectual functioning are not significantly affected by closed head injury or orthopaedic injury. However, all measures of current cognitive functioning (WAIS-R, MMSE, verbal fluency, and Stroop) were significantly impaired in the head injury group (table 4).

This study is the first to show that both CCRT and STW seem to be relatively resistant to the effects of closed head injury. It is of interest that placing the NART stimulus words within context (CCRT condition) resulted in a significantly greater benefit for the patients with closed head injury versus controls. This finding is similar to those derived from evaluations of the CCRT in demented patients.23 In terms of correlates of premorbid indices, whereas the association between years of full time education and premorbid estimates is unsurprising (table 3), it was of interest that significant correlations emerged between each of the premorbid estimates and HAD depression score—that is, higher depression ratings were associated with increased error scores, particularly in the head injury group. Previous studies had suggested that NART performance is not affected by major depression.29 30 However, both these studies were carried out on patients with major clinical depression, not patients with closed head injury who had raised depression levels. It is of theoretical interest that the combination of closed head injury and raised depression levels may be associated with impaired performance on estimates of premorbid intellectual functioning. It is not possible to determine the direction of causality from the present study. It is possible that the combination of head injury plus depression impairs performance. Alternatively, subjects with genuine low premorbid ability may become more depressed after a head injury. Clearly, further research is required to consider this issue. Although we have shown the relative insensitivity of the three putative estimates of premorbid intellectual ability to closed head injury, the relative ability of these measures to predict general intellectual level remains to be determined. This was considered in experiment 2.

Subjects and methods for experiment 2

In this study, the association between the premorbid estimates and measures of current intellectual functioning were determined in 114 healthy control subjects recruited from the general population (including the 50 healthy controls and 20 orthopaedic controls used in experiment 1). All healthy subjects fulfilled the recruitment criteria for experiment 1, and were administered the NART, CCRT, STW, and the short form of the WAIS-R described in experiment 1.

Results of experiment 2

The summary data on the 114 control subjects are presented in table 5. The simple correlations between each of the premorbid estimates (NART, CCRT, and STW) and the WAIS-R indices (VIQ, FSIQ, and PIQ) are shown in table 6. Both the NART and CCRT correlated relatively well with current verbal IQ (for both,r>0.7), whereas the STW correlated significantly less well (r=0.54). Not surprisingly, the correlations with full scale IQ were less strong and correlations with performance IQ were marginal. To determine whether the addition of demographic variables to the premorbid estimates would improve the amount of explained variance in predicted full scale IQ, verbal IQ, and performance IQ, multiple linear regression analyses were undertaken using the protocol adopted by Crawfordet al 18 Using a forward stepwise procedure, WAIS-R full scale, verbal and performance IQ were individually regressed against the premorbid measures (NART, CCRT, or STW) with years of education, social class, age, and sex added into the equation. The results are shown in table 7. The addition of demographic variables increased the amount of explained variance in current verbal IQ for both the NART and CCRT to about 60%. However, the addition of socioeconomic status as a predictive variable only increased the STW prediction of verbal IQ to 41% of explained variance in current verbal IQ. In predicting full scale IQ, the addition of years of full time education increased the predictive ability of the NART and the CCRT to between 42 and 45%. However, for the STW regression equation, the best predictor of full scale IQ was years of full time education which accounted for 26% of the variance, and only then was STW added in as a predictor variable accounting for an additional 9% of the variance. None of the premorbid estimates was a useful predictor of current performance IQ.

Table 5

Summary data on 114 controls (43 females, 71 males)

Table 6

Correlations of premorbid with current measures of intelligence in 114 control subjects

Table 7

Stepwise regression analyses of premorbid and demographic variables against measures of current intellectual functioning in 114 control subjects: amount of variance explained

Table 8

Regression equations for the prediction of WAIS-R verbal IQ with standard errors (SE)

Discussion of experiment 2

The main finding to emerge from experiment 2 was that both NART and CCRT correlated relatively highly with current WAIS-R verbal IQ. However, the correlation between STW and verbal IQ was disappointingly low, only accounting for 29% of the observed variance in current verbal intelligence. To be a clinically useful measure for the estimation of premorbid intellectual level in closed head injury, the test in question must (a) be relatively resistant to the effects of closed head injury and (b) also be highly correlated with current measures of verbal intelligence. Although performance on the STW certainly seems to be relatively unaffected by closed head injury, its ability to predict current intellectual level appears to be somewhat limited. In the initial publication on the STW, Baddeleyet al 20 reported correlations between the STW and Mill Hill vocabulary IQ of 0.87 and between STW and the NART of 0.871. In the present study, the correlation between STW and the NART was relatively impressive (r=0.69). However, its correlation with what many would value as the gold standard of current verbal intelligence, the WAIS-R, was modest. The addition of demographic variables into the regression equation increased the ability of the NART to predict current verbal IQ to 60% and the CCRT to 62%, but only improved the amount of explained variance for the STW to 41%.

General discussion

The present study is the first to compare three currently employed estimates of premorbid intellectual functioning in closed head injury. Performance in all three measures seems to be relatively unaffected by closed head injury, suggesting that all three may provide valid estimates of premorbid intellectual level. It is interesting to note that in this closed head injury sample depression level was significantly associated with impaired performance on each of the premorbid measures, suggesting that patients with closed head injury with high levels of depression may perform relatively poorly on the premorbid estimates, thus potentially providing a misleadingly low estimate of premorbid ability. Further work in this area is required. The correlations between each of the putative premorbid measures and measures of current intellectual level were obtained in 114 healthy control subjects. Both the NART and CCRT correlated relatively well with current measures of WAIS-R verbal IQ (table 6). However, the correlation between STW and current verbal intelligence was disappointingly low. The addition of demographic variables into the predictive equation resulted in significant improvements in the predictive ability of both the NART and the CCRT, leading to over 60% of the variance in current verbal intelligence being accounted for. However, the STW in isolation only accounted for 29% of the explained variance in current verbal intelligence and addition of socioeconomic status as a predictor only increased the combined predictive ability to 41%—that is, over 50% of the variance was left unexplained. The results of the present study suggest that the NART and CCRT may be useful estimates of premorbid verbal intellectual level in patients with closed head injury and that such patients derive particular benefit from the CCRT condition. However, the potential confounding effect of raised depression levels warrants further investigation. Furthermore, before the CCRT can be endorsed for clinical use, conversion tables which equate CCRT errors into WAIS-R equivalent scores need to be developed, based on a representative and large sample of the British population. Finally, the low correlations between STW and WAIS-R suggest that there is considerable scope for error in estimating the premorbid IQ an individual patient with closed head injury in clinical practice using the STW.

Acknowledgments

We thank the subjects who participated and Norma Brearley for her careful preparation of the manuscript. We also thank Roslyn Law for help with recruitment of healthy control subjects.

References

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.