Using an independent data set, the utility of the Glasgow Head Injury Outcome Prediction Program was investigated in terms of possible frequency of use and reliability of outcome prediction in patients with severe head injury, or haematoma requiring evacuation, or coma lasting 6 hours or more, in whom outcome had been reliably assessed at 6 to 24 months after injury. Predictions were calculated on admission, before evacuation of a haematoma, or 24 hours, 3 days, and 7 days after onset of coma lasting 6 hours or more. Three hundred and twenty four patients provided 426 predictions which were possible in 76%, 97%, 19%, 34%, and 53% of patients on admission, before operation, 24 hours, 3 days, and 7 days respectively. Major reasons for non-feasible predictions were that patients were paralysed/ventilated as part of resuscitation or management. Overall, 75.8% of predictions were correct, 14.6% were pessimistic (outcome better than predicted), and 9.6% optimistic (outcome worse than predicted). Of 197 patients (267 predictions) whose eventual outcome was good or moderate, 84.3% of predictions were correct. For death or vegetative survival (96 patients with 110 predictions), 83.6% of predictions were correct but for severe disability (31 patients with 49 predictions), only 12.2% were correctly predicted. The utility of the Glasgow Head Injury Outcome Prediction Program compares favourably with other outcome prediction algorithms for patients with head injury.
- head injury
Statistics from Altmetric.com
An estimate of prognosis after head injury is central to clinical decisions. Reliable prediction of outcome could allow realistic counselling of relatives and appropriate allocation of resources (to those predicted to survive or recover with a good outcome), while assisting treatment limiting decisions in those predicted to have a poor outcome (death or vegetative survival).1 2Prediction of an uncertain outcome may also be useful in preventing an over confident clinical decision.3
Several models for outcome prediction using easily available clinical and radiological data have been described,4-7 but most have been validated only on data from patients used to construct the algorithm or other subjects treated in the same department. The Glasgow Head Injury Outcome Prediction Program is probably the best known model8 9 and we report on an independent assessment of this program in a series of patients with head injury admitted to the Department of Clinical Neurosciences, Western General Hospital, Edinburgh.
Patients and methods
Data were prospectively collected from 324 patients with head injury admitted to the Department of Clinical Neurosciences between January 1989 and April 1994. The age range was under 1 year to 87 years, with median age 30.0 years.
In the Glasgow system, algorithms to predict outcome at 6 months are available for (a) patients in coma on admission, (b) patients with an intracranial haematoma (of any grade of severity) requiring evacuation, and (c) patients in coma (no eye opening, no comprehensive verbal response, and not obeying commands) for 6 hours or more.
For all patients, the first prediction calculated was an admission prediction or a preoperative prediction.
For those patients in coma for 6 hours or more, outcome may be predicted at 24 hours, 3 days, and 7 days after the onset of coma. The groups are not mutually exclusive and patients may have more than one set of predictions calculated, at these different “epochs”.
Each algorithm predicts outcome and provides probabilities of three outcome groups: (1) dead or vegetative (D/V), (2) severely disabled (SEV), (3) moderately disabled or good recovery (M/G), with the total probability equal to unity.
Data which can be used for prediction include age, Glasgow coma score (GCS)10 pupil response, CT findings (extradural haematoma only, extra and intradural haematoma, intradural haematoma only, diffuse injury with normal pressure, or diffuse injury with evidence of raised pressure), temporal changes in GCS, motor response pattern, presence or absence of a lucid interval, type of haematoma requiring evacuation, and eye signs such as presence or absence of spontaneous eye movements, and oculovestibular and oculocephalic responses.
The program uses a Bayesian model which has the ability to cope with missing data, a common problem in patients with head injury,9 but still be able to provide an outcome prediction. In practice, a minimum of three items of data were needed to calculate a set of predictions.
Predictions were not available to clinical staff and were not used to guide management.
ASSESSMENT OF OUTCOME
Outcome was assessed using the Glasgow outcome score (GOS),11 assessment being after a minimum of 6 months up to a maximum of 24 months after injury.
Interview of the patient and relatives by a research psychologist was the “gold standard”,12 otherwise the relative's questionnaire based on that of McKinlay et al 13 was used. General practitioner assessment was used only if neither was available.
The frequency with which predictions could not be calculated was determined. The reason was sought from clinical notes or from research data.
Calculated predicted outcomes were compared with actual outcome. A prediction was “correct” if the outcome group with the highest probability corresponded with the actual outcome. If the predicted outcome was worse than the actual outcome, the prediction was “pessimistic” and if better “optimistic”.
A “confident” prediction is one in which one of the outcome groups (D/V, SEV, or M/G) is predicted with high probability (0.97).14 A “confident” prediction may be correct, optimistic, or pessimistic.
POTENTIALLY SUITABLE PATIENTS
Of the 324 patients, 183 were potentially suitable for an admission prediction, 141 for a preoperative prediction, and 180, 145, and 124 suitable at 24 hours, 3, and 7 days respectively.
The progressive reduction in the total potential numbers reflects the declining numbers of survivors.
FREQUENCY OF PREDICTIONS
Overall, predictions were able to be calculated in 76% of potentially suitable patients on admission, 97.2% before haematoma evacuation, but only 19.4%, 33.8%, and 53.2% of eligible patients at 24 hours, 3 days, and 7 days after the onset of coma respectively.
The use of sedation, paralysis, and intubation (making clinical assessment impossible) was the most common reason for inability to calculate a prediction (62% at admission and 94.5%, 89.6%, and 74.1% at 24 hours, 3 days, and 7 days).
ACCURACY OF PREDICTIONS
Comparing predicted with actual outcome, predictions were correct in 76.3% of admission predictions, 82.5% of preoperative predictions, and 77.1%, 63.3%, and 69.7% of predictions at 24 hours, 3 days, and 7 days (table 1).
Overall, 75.8% of predictions were correct, 14.6% pessimistic, and 9.6% optimistic (table 2).
Of 137 patients with a preoperative prediction, 59 had a severe, 50 a moderate, and 28 a mild brain injury. Of these, 42 (71.2%), 45 (90%), and 26 (92.9%) respectively were correctly predicted.
For all predictions, 83.6% of those with actual D/V outcome were correctly predicted and 84.3% of those with actual M/G were correctly predicted. SEV was the smallest outcome group (11.5%) and the least often correctly predicted (12.2%).
Of 426 predictions, 79 (18.5%) were confident predictions of which 71 (90%) were correct, two were optimistic, and six pessimistic. Most confident predictions (51) were for the preoperative epoch. For the different categories of severity of injury (severe/moderate/minor), six (10.2%), 21 (42%) and 20 (71.4%) were correctly predicted.
For all patients and using dichotomous outcome groups (D/V and SEV/M/G), the number of confident predictions was increased to 160 (37.6% of total predictions) of which 153 (95.6%) were correct. Six of 34 predictions of D/V were pessimistic but five of these patients were actually severely disabled and only one achieved a moderate/good recovery.
Barlow and Teasdale reported that clinicians involved in the management of head injury regard an estimate of prognosis soon after injury as an important influence on subsequent management.15 Clinical estimates of outcome, however, are variable and may be inaccurate,16 and provision of predictions with the Glasgow program can influence management.17
It is unlikely that any clinician has the equivalent experience of the 3000 patients on whom the Glasgow Prediction Algorithm is based. The program was shown to be at least as accurate as experienced clinicians' predictions when the same cases are used.9
A major concern of any prediction is if patients predicted to have poor outcome actually survive with good outcome. In a previous assessment of the program, a confident prediction of death/vegetative survival with moderate disability/good recovery was rare but this was an internal assessment.18 Pessimistic assessments were uncommon in this study (15% of all predictions and 8% of confident predictions) and even more rarely (0.6%) were reflected in survival in an independent state using confident predictions and a dichotomous outcome.
Differences between predicted and actual outcome may be accounted for by several factors including case mix, differences in management and in assessment of initial state, the effects of alcohol and drugs in the acute stage, and differences in assessment of outcome. Many of the patients in this study were admitted to accident and emergency departments distant from the neurosurgical unit where assessment of conscious level may have varied and patients in coma were usually paralysed and intubated preventing assessment of GCS after transfer. The extent of this problem will vary between different neurosurgical units.
For patients with a haematoma requiring evacuation soon after admission, the prediction program more correctly and confidently identified the final likelihood of good/moderate outcome for patients with a minor and moderate injury than for a severe injury. This is not surprising as minor and moderate injury outcomes are clinically easier to predict because most will have a good prognosis. The computer is likely to be least accurate with those patients most difficult to predict clinically. Like other investigators,5 19 20 we found severe disability to be the hardest outcome to predict correctly.
Previous authors have reported the performance of different methods for outcome prediction.
Using logistic regression, data about admission state and a dichotomous outcome (at 3 months) of either D/V or SEV/M/G after severe head injury, Stablein et al 7achieved an overall correct prediction rate of 91%, compared with 84% in this series (severe head injuries only). Choi et al 5 achieved an overall prediction rate of 77.7% using a prediction tree for outcome after severe head injury. Admission data and 12 month outcomes were used, but the assessment was performed with data from patients used to construct the prediction tree. Using the same prognostic features, stepwise logistic regression, and stepwise discriminant analyses Choi et al 5 achieved an overall accuracy of 74.2% and 74.6% respectively. At least 64 patients of 555 total (11.53%) had pessimistic predictions of death, vegetative, or severe disability yet achieved moderate disability or good outcome. Correct prediction rates would be expected to be lower if patients independent from the algorithm are used.
There is debate as to whether the Bayesian model used in the program is the best statistical model for outcome prediction,7 21but comparison of statistical methods22 using a common data base demonstrated that the information used to make the prediction was more important than the statistical method.
Clinicians do not subject their own outcome predictions to analysis with the degree of rigour in this study and there must therefore be concern that some clinical predictions that lead to treatment limiting decisions might produce outcomes that become “self fulfilling prophecies”. The Glasgow computer program is likely to outperform the average clinician for accuracy of prognosis, and although not infallible, it will often prevent grossly overconfident or overpessimistic predictions that might otherwise result in erroneous management decisions.3
Accurate prediction of outcome in the individual patient remains difficult to achieve. The Glasgow program provides the clinician with probabilities of different outcomes, rather than a single explicit outcome category, and its usefulness is limited by the frequency with which predictions could not be predicted. Most clinicians will require extreme degrees of certainty from a computer before acting upon the prediction alone.21 Although there will always be the possibility that a “confident prediction” of death/vegetative survival will be followed by a moderate/good recovery, this study confirms that such extreme cases are rare. Therefore, in clinical practice, the result of a prediction program should not be used as the sole basis for decision making, but as a compliment to clinical estimates of prognosis and as an aid to decision making.
Overall, the utility of the Glasgow Head Injury Outcome Prediction Program compares favourably to that of other outcome prediction algorithms, even when applied in a different neurosurgical unit, to patients independent of the original data set.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.