Results

LOTHIAN STROKE REGISTER SERIES

The patients were assessed at a median interval of 72 weeks after the onset of their index stroke (interquartile range: 43 to 104 weeks). Of the 152 patients who participated in this study, 92 were able to complete the questionnaires by themselves without help; the remaining 60 patients could only be assessed by interview.

One hundred and forty seven of the 152 patients (97%) completed both modified simple questions. Three patients did not complete the modified dependency question and another two patients did not complete the problems question. Fifty four (37%) patients replied “yes” to the dependency question and were classified as dependent; eight (15%) of these 54 dependent patients replied “no” to the question about problems from their stroke (but in fact five of these eight had been rated as having an Oxford handicap score of at least 2 or more before their index stroke).20 21 The remaining 93 (63%) patients were independent in everyday activities; of these, 52 (56%) reported being left with problems after their stroke (independent, but not recovered), whereas the other 41 (44%) denied any problems (recovered).

Validity

The sensitivity, specificity, and accuracy of the modified simple questions for the assessment of outcome in the domains of mobility, self care, social functioning, pain, and psychological functioning are reported in table 1. The modified dependency question had excellent sensitivity, specificity, and accuracy for the assessment of activities of daily living as defined by the Barthel index. Predictably, it was less accurate at predicting psychological dysfunction as defined by the HADS. The problems question proved highly sensitive for the detection of problems in all of the domains assessed, but it had low specificity. Its specificity was better (63%) when used to identify patients with problems in any of the unidimensional measures, table1.

The median score and interquartile range for the standard instruments are shown for groups defined by their responses to both questions (table 2). The median scores with the standard instruments were ordered appropriately and were statistically distinct. The mean utility scores with the EuroQol differed significantly among groups of these patients classified by their responses to the simple questions (table 3). The best outcome in all domains (including overall health related quality of life with the “thermometer”, table 3) was reported by the fully recovered group of patients.

Validity of the proxy assessments—The study nurse, using the OPCS disability measure, rated six patients as having significant difficulties in communication. These six patients did not therefore provide data for the assessment of agreement between the patients and their proxies. We obtained proxy assessments of the patients' outcome with the modified simple questions for 121 (82%) of the 147 patients. Agreement between the patient and their proxy for the dependency question was very good (agreement=103/121 (85%); κ=0.70, 95% confidence interval (95% CI) 0.57–0.83). Agreement between the patient and their proxy for the problems question was good (agreement=99/121 (82%); κ=0.58, 95% CI 0.42–0.74). Agreement between the patient and their proxy for the overall outcome classification with the modified simple questions was good (agreement = 87/121 (72%); κ=0.57, 95% CI 0.45–0.69). There was an overall trend towards proxies assessing the patients' functioning as worse than that assessed by the patients themselves (21 patients were rated worse by their proxies than by themselves versus 13 patients who were rated better by their proxies than by themselves, sign test p>0.05).

INTERNATIONAL STROKE TRIAL SERIES

Relation between the simple questions and the EuroQol

Figure 2 shows the distribution of responses to the EuroQol questionnaire for the patients classified by their responses to the modified simple questions. As before, each of the groups have distinct responses for all of the domains of the EuroQol and the pattern of responses was very similar to that observed in the other (Lothian Stroke Register) cohort (histogram not shown, but available from authors). For this larger cohort of patients, the mean estimates of the EuroQol utilities were also ordered appropriately and were statistically distinct for each of the three levels (table 3). The mean estimates of overall health related quality of life showed a similar pattern (table3).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Health related quality of life in each domain of the EuroQol among patients categorised as dependent, independent, or recovered by their response to the modified simple questions (ISTS).

Relation between the simple questions and the SF-36

Although the responses to the modified simple questions seemed to be closely related to the patients' health related quality of life as defined by the EuroQol, we then confirmed the relation by relating the simple questions with the SF-36 (table 4). For all but two of the domains (physical role functioning and mental health), the median scores for each domain were ordered appropriately and differed from each other. In the physical role functioning domain, the dependent and independent patients both scored zero out of a possible 100. This provides evidence for a floor effect in this domain of the SF-36. Similarly, the median score in the mental health domain of the SF-36 was the same for patients who were classified as independent or recovered.

Test-retest reliability of the modified simple questions

In the study of the reliability of the EuroQol and SF-36 questionnaires, 443 patients returned repeat assessments of outcome.15 These assessments included responses to the modified simple questions. A further 688 patients also returned repeat assessments of outcome which included the modified simple questions as part of the comparison of the EuroQol and SF-36 questionnaires.16 Thus, 1131 patients potentially completed the simple questions at test and retest (fig 1). Test-retest reliability was very good for both modified questions and the classification of overall outcome with the modified simple questions (tables 5, 6, and 7 ). There was a mean delay of 30 days between completion of the initial and repeat questionnaires (SD 12 days).

Discussion

MODIFIED DEPENDENCY QUESTION (“DO YOU NEED HELP FROM ANYBODY WITH EVERYDAY ACTIVITIES?”)

We found that the modified dependency question was a valid measure of dependency in activities of daily living after stroke. It had excellent sensitivity, specificity, and accuracy for identifying dependency after stroke. Although the current study was not primarily designed to directly compare the validity of the modified dependency question with that of the original, it had a greater sensitivity than the original at identifying dependency (by Barthel index) after stroke (85% v 61%); however, its specificity was slightly worse than that of the original question (85%v 96%).3 The modified dependency question was also highly accurate at identifying poor mobility and social functioning after stroke. It was, as would be predicted, less accurate at identifying patients with psychological dysfunction. These indirect comparisons suggest that, compared with the original, the modified version of the question had concurrent validity that was at least as good as the original and better face validity. We found the modified dependency question also had excellent test-retest reliability (κ=0.81, 95% CI 0.77–0.85); this compares very favourably with the interrater reliability of the original question (κ=0.51, 95% CI 0.36–0.67).6 This difference may reflect the improved wording of the new question. Alternatively it may just reflect the differences between interrater and test-retest reliability or the method of questionnaire administration. Denniset al employed two raters who administered the simple questions on different occasions by face to face interview,6 whereas we administered the questions by post.

PROBLEMS QUESTION (“HAS THE STROKE LEFT YOU WITH ANY PROBLEMS?”)

We completely reworded the recovery question because patients found the original wording ambiguous and our assessments of its validity also disclosed uncertainty about which aspect of outcome it addressed.6 18 The reworded question aims to detect those patients who are left without appreciable problems resulting from the stroke, rather than to identify patients who recover to the point of having no problems at all, whatever their cause. This alternative emphasis, although relatively subjective and non-specific, may be more relevant to both patients and their families. The reworded question had excellent sensitivity and moderate specificity at detecting problems in all the domains examined (mobility, self care, social functioning, and psychological functioning). Indeed, this analysis may underestimate the sensitivity of the problems question as we would expect some patients to report problems on the standard measures which are not attributable to the stroke. These measurement properties differ clearly from those of the original recovery question and support the validity of the reworded question. The reworded question also had better reliability than the original (κ=0.78 (95%CI 0.73–0.83) v κ=0.61 (95%CI 0.46–0.76)).6 However, it is not clear whether this improved reliability resulted from better wording or from differences in the study design. It is also not clear from the current data how patients with problems after their stroke would have responded to the question “have you made a complete recovery from your stroke?” (the original version).

USE OF BOTH MODIFIED QUESTIONS TOGETHER TO ASSESS HEALTH RELATED QUALITY OF LIFE

The modified simple questions may be used to classify surviving patients into one of three hierarchical outcome groups. About half of the independent patients were left with significant problems as a consequence of the stroke. This illustrates the ceiling effect with simple measures of disability that focus exclusively on activities of daily living— that is, if the dependency question was the only measure of outcome, then other significant limitations (for example, problems with communication, household maintenance, and social and psychological functioning) would not be captured. Additional assessments of health are therefore required in independent patients who might otherwise be considered as having achieved a “good outcome” after their stroke. The statistically distinct profile of scores for this group of patients with the standard instruments support the validity of this hierarchical classification.

The validity of this classification is further supported by its close relation with patients' responses to the assessments of health related quality of life. The same pattern of responses was found, in both cohorts of patients, for assessments with the EuroQol and in the IST cohort for assessments with the SF-36. Therefore, the combined use of these modified simple questions, to classify patients as dependent, independent, and fully recovered, seems to offer a simpler approach to the assessment of global health related quality of life, than with even the EuroQol. These assessments also had very good test-retest reliability, which was of a similar order of magnitude to that found for the assessments of overall health related quality of life with the EuroQol or general health with the SF-36.15 Moreover, agreement between patients and their proxies for their responses to the modified simple questions was also moderately good (κ=0.57). There would be several potential advantages associated with the use of these simple questions for the assessment of health related quality of life after stroke. Firstly, because of their brevity and simplicity they are likely to place less of a burden on patients, and so may be more feasible than complex measures. Short and simple measures, with a high frequency of response, may reduce non-response bias and enhance the interpretation and generalisability of results. Secondly, analysis of data based on a single classification of outcome by the simple questions would be simpler to both perform and interpret, than analysis of multidimensional data from a health related quality of life instrument. Thirdly, treatment effects described in terms of the proportion of patients reporting a change in response to the simple questions would have more immediate interpretability than a change in numerical score or utility. Finally, a single categorical classification of outcome by the simple questions allows the analysis to include the dead patients (dead, dependent, independent, and recovered)—dead patients could not be easily scored with the EuroQol or SF-36.

However, there are also disadvantages to replacing detailed assessments of health related quality of life with the cruder information provided by these questions. Firstly, the modified simple questions could only be used to provide a broad and global assessment of health related quality of life. Therefore, the opportunity of obtaining information about outcome in specific domains—for instance, psychological functioning—would be lost. This would reduce the opportunity to explain how a treatment improves overall health related quality of life—for example, by improving psychological functioning or by improving mobility. Secondly, the modified simple questions were developed as a disease specific instrument. They do not, therefore, provide outcome information which can be used to compare directly the effects of different treatments in groups of patients with different diagnoses. However, such comparisons could still be performed, albeit indirectly, by using the data from table 3 to map patients' responses to the simple questions into health related quality of life utilities with the EuroQol.

SIMPLE VERSUS MORE COMPLEX METHODS OF MEASUREMENT

Simple measures of outcome, such as the modified simple questions, may be criticised for lacking measurement precision because they categorise stroke survivors into only three groups: dependent, independent with residual problems, and independent with no problems. This would be a reasonable point of view if the aim of the questions was to make a precise measure of outcome in an individual patient. However, these assessments were designed to estimate the difference in outcome between large groups, each comprising thousands of patients. Individual patients differ so greatly from each other that a rather crude yet valid measure can provide a precise and unbiased estimate of treatment effects in such large groups of patients.22 23Furthermore, detailed measures can be surprisingly unreliable if studies only include a few subjects, as the effect of random variation is likely to be larger than the effect of the treatment.

Simple categorical data, of the type generated by the modified simple questions, can convey a surprisingly large amount of information. For example, if an imaginary and really accurate measurement scale has a possible range of 0 to 100, a measure which can identify those with a score of 0–50 and those scoring 51–100 would reduce the variance of non-measurement by about 75%. In other words, three quarters of the information which would have been obtained using the full 100 point scale (at great effort, time, and cost) can be obtained by a simple dichotomy. An equal three way split reduces the variance by 89% (Peto R, personal communication). Finally, more recent data has also demonstrated that patients respond significantly more often to simple measures of outcome versus longer more complex measures.14

Conclusions

Patients completed these modified simple questions either by interview or questionnaire in the presence of the study nurse. Although the study nurse did not routinely ask patients about their interpretation of either question, no significant ambiguities were noted. These modifications therefore seem to have improved both the face and content validity as well as the reliability of these simple questions. The current study has provided strong support for the concurrent validity of the modified questions when used either individually or together to classify patients into one of three hierarchical groups which are relevant to the patients' overall health related quality of life. Proxies were able to provide a moderately accurate and unbiased assessment of the patients' outcome after stroke with these modified simple questions. Therefore, the “modified” versions of the simple questions seem to be valid, and although they have not been formally compared, they seem preferable to the original. The modified simple questions are a simple, pragmatic, and clinically relevant measure of outcome after stroke. They have a wide range of potential applications, from large formal epidemiological studies (and trials) to local audit studies.