Results

SAMPLE CHARACTERISTICS

Of the 150 patients entered into the study, one did not complete the battery of questionnaires and was excluded from the analyses. Of the remaining 149 people there were no missing data for any items. Table 1 presents the demographic and diagnostic characteristics of the study sample, of which 70% were married, 33% were employed, and 49% required assistance with their daily care. Table 2 shows the mean SF-36 scores for our sample alongside those of two other multiple sclerosis populations.

APPROPRIATENESS

Table 3 presents the score distributions for the SF-36 dimensions and summary scales. In the total sample, scores in all dimensions span virtually the entire range; however, floor effects in three dimensions (physical function, physical and emotional role limitations) and ceiling effects in two dimensions (emotional role limitations and pain) exceed the recommended criteria of 20%.34 When patients are subdivided into groups according to disease severity the distribution of scores within each subgroup, in some cases, alters markedly. For example : (a) the physical function scores span only the bottom 20% of the range for the severe group; (b) the means fall substantially outside the midpoint of the scale for physical function and physical role limitations in the moderate and severe groups (mean PF moderate=22.9, severe=4.4; mean RLP moderate=28.5, severe=10.5); (c) floor effects increase markedly for physical function and role limitations (both emotional and physical), particularly in the severe group. The lowest possible score is reported by 84% of severe patients for physical role limitations and 36% of patients for emotional role limitations.

Table 3 demonstrates that the differences in score distributions between the total sample and the subgroups were less marked in the summary scales. Of importance, no ceiling or floor effects were present.

Table 4 presents the score distributions for instruments measuring related health constructs. In the total sample scores on the FIM, LHS, and the global rating scale of QoL span virtually the entire scale range; the mean scores were near the midpoint; and the floor and ceiling effects were minimal. This indicated that the scales were appropriate for the total study sample. When patients were subgrouped according to EDSS score the appropriateness of these instruments, while not ideal, remained satisfactory. Although the scores were restricted to a smaller range of the available scale, the floor and ceiling effects remained well below the recommended criteria of 20%.34 By contrast, in both the total sample and each of the subgroups, the mean scores on the GHQ fell below the midpoint of the scale and the ceiling effects were above the recommended upper limit.

RELIABILITY

Internal consistency reliability for each of the eight dimensions and the component summary scales of the SF-36 was high with α coefficients ranging between 0.77 to 0.94.

CONVERGENT AND DISCRIMINANT CONSTRUCT VALIDITY

Intercorrelations between the SF-36 dimensions

Table 5 reports intercorrelations between SF-36 dimensions. Importantly, none of the correlations were strong (r=0.09–0.61) demonstrating that each dimension was measuring a related but distinct construct. As predicted, related dimensions were more strongly associated than less related dimensions. For example, physical function showed a stronger correlation with physical role limitations (r=0.57) than with emotional wellbeing (r=0.09) or emotional role limitations (r=0.14). Similarly emotional wellbeing showed a stronger correlation with emotional role limitations (r=0.54) than pain (r=0.29). Interestingly, emotional wellbeing showed a much stronger correlation with energy and vitality (r=0.61) than did physical function (r=0.18).

Correlations between SF-36 dimensions and instruments measuring related health constructs

As predicted, associations between SF-36 dimensions and instruments measuring related health constructs were strongest between those measuring similar concepts. For example, physical function correlated strongly with the FIM (r=0.68) and the EDSS (r=−0.82); and emotional wellbeing correlated substantially with the GHQ (r=−0.59). By contrast, associations were weak between instruments measuring unrelated constructs. For example, emotional role limitations was only weakly associated with the FIM (r=0.04), and pain was only weakly associated with the EDSS (r=−0.07). It is notable that the social function dimension correlated more strongly with scales measuring emotional constructs (for example, GHQ r=−0.56) than physical constructs (for example, EDSS r=−0.29; FIMr=0.34).

Group difference construct validity—As expected, statistically significant differences between the patient subgroups occurred in three SF-36 dimensions (social function, physical function, and physical role limitations; p<0.05–0.0001). This finding demonstrates the ability of these dimensions to discriminate between different levels of disease severity. Significant differences were also demonstrated between all subgroups for the physical summary scale (p<0.001), but only between the mild and the moderate group in the mental summary scale (p<0.03).

Hypothesis testing—As predicted: (a) patients requiring carer assistance reported lower scores in the physical role limitations dimension than those who are independent (p<0.0001, mean scores=13.5 and 43.7 respectively); (b) patients with relapsing-remitting multiple sclerosis reported higher scores in the physical summary scale than those with secondary progressive multiple sclerosis (p<0.0001, mean scores=35.7 and 27.4 respectively); (c) patients scoring ⩾5.0 points on the GHQ reported lower scores on the mental summary scale than those scoring <5.0 points (p<0.0001, mean scores=40.4 and 52.2 respectively).

RESPONSIVENESS

Forty four patients participated in inpatient rehabilitation for an average of 20 days ((SD 6) range 13–39). Effect sizes for the SF-36 dimensions ranged from negligible to small (effect sizes 0.01–0.30). The dimensions demonstrating the largest effect size were the emotional role limitations (effect size 0.27) and pain (effect size 0.30). Of the eight dimensions, only pain (p=0.006) and physical function (p=0.01) demonstrated a statistically significant change in scores between admission and discharge. By contrast effect sizes on the FIM, LHS, and GHQ were all moderate in magnitude (effect size 0.56, 0.58, and 0.51 respectively) and statistically significant differences were demonstrated between scores on admission and discharge for each of these measures (p<0.002).

Discussion

This study provides information about a widely used generic measure of health status—the SF-36. The SF-36 was constructed to compare functional health and wellbeing across patient and general populations, and to evaluate and compare the benefits of alternative treatments.24 The focus of this study was to examine its performance as an outcome measure in multiple sclerosis.

The generalisability of our results is supported by the fact that the demographic and diagnostic characteristics of our sample population are typical of those described in the literature.35Furthermore, as demonstrated in table 2, the distribution of SF-36 scores is very similar to the results of previous multiple sclerosis studies16 17 25 suggesting that our sample is representative of the general multiple sclerosis population.

The test-retest reliability of the SF-36 was not investigated in this study but the results of others, both in the United Kingdom11 and the United States,12 report excellent results at 2 weeks. In agreement with some other studies, our results demonstrate that the internal consistency for all dimensions of the SF-36 exceeds the 0.7 standard for group comparisons.12 Similarly, convergent and discriminant construct validity are supported by the direction, magnitude, and pattern of correlations with other health measures. Further evidence for construct validity has been provided by support for the clinical hypotheses tested. These data support the internal consistency reliability and validity of the SF-36 as a measure of health status in multiple sclerosis. Consequently it would seem reasonable to choose the SF-36 as an outcome measure in clinical trials evaluating the effectiveness of interventions in multiple sclerosis.

When the data are examined in more detail, however, some limitations of this measure become apparent. For instance the large floor and ceiling effects in four of the eight dimensions indicate that the range of health status measured is unlikely to represent the range experienced by this population, and demonstrate limitations in the ability of the SF-36 to discriminate between individual patients in these dimensions. It is notable that the floor and ceiling effects do not simply apply to patients at the extremes of the disease severity range; the moderate group also exhibit significant floor effects in three dimensions. The data also show a polarisation of responses in the role limitations dimensions. This is perhaps not surprising when the dichotomous format of the questionnaire items is considered. For an example, refer to fig1, which contains the emotional role limitations question.

These concerns as to the appropriateness of the SF-36 in multiple sclerosis are heightened when the population is subdivided into groups according to disease severity. This is very important as the selection criteria of most clinical trials will inevitably narrow the range of disease severity of the study sample, sometimes markedly (for example EDSS 1.0<3.536; EDSS 3.0– 6.52; EDSS<6.537). These results highlight the importance of examining the appropriateness of an instrument for the specific population under investigation. Even though an instrument may prove to be appropriate for one group of patients this may not necessarily be the case for a different group, even within the same medical condition.

No floor or ceiling effects occur in the SF-36 mental and physical summary scales, suggesting that these scales may be more appropriate than the individual dimensions for discriminating between individual patients at a single point in time. Additionally they have the advantage of reducing the number of statistical comparisons required in the analysis of results, thereby reducing the role of chance in testing experimental hypotheses.24 A disadvantage, however, is that it is impossible to interpret precisely where any changes have occurred; a common feature of all multidimensional instruments.

The responsiveness of an instrument is of key importance in outcome studies. If the instrument is unable to detect change in health, an intervention that improves health status may show no apparent difference between treated and untreated patients. Unfortunately, this property is often overlooked and information about the responsiveness of the SF-36 in multiple sclerosis trials is scarce. The negligible to small effect sizes demonstrated by the SF-36 show the responsiveness of the SF-36 to be poor in evaluating the effectiveness of inpatient rehabilitation in people with moderate to severe disability. Although some may suggest that this is because little or no change has occurred, the moderate effect size results of the FIM (measuring physical function), the GHQ (measuring emotional health), and the LHS (measuring handicap) show that change has indeed occurred, at least in these select areas. Furthermore statistically significant changes were demonstrated between change scores on each of these three measures, but in only two of the dimensions of the SF-36. The poor responsiveness of the SF-36 may, in part, be explained by the fact that it measures broad issues of both function and wellbeing, which, taken together may not give a clear effect. By contrast, the FIM, GHQ, and LHS measure more specific health constructs. We would also suggest that the clustering of scores at either end(s) of the scale, found in half of the SF-36 dimensions, means that the range of the scale is too limited to enable small but possibly clinically significant changes to be recorded; thereby limiting responsiveness. It is stressed, however, that the responsiveness data in this study is restricted to patients with moderate to severe disability undergoing rehabilitation and that the SF-36 has therefore not been assessed in a population representative of the patients included in most multiple sclerosis trials. This is a limitation of this study.

Different approaches to consider some of the limitations of generic measures have been used in recent years. For example, the development of disease specific measures for multiple sclerosis has been undertaken either by adapting current measures (for example, the functional assessment measure38 or the multiple sclerosis QoL-544); by gathering together a wide range of symptom specific measures (for example, the QoL inventory39); or by identifying key areas and then weighting them according to how important the patient thinks these areas are to their lifestyle (for example, the disability and impact profile40). All of these measures are in the early stages of evaluation.

Conclusions

Understanding the properties of an outcome measure is essential when choosing the most appropriate instrument for a study and interpreting the information it generates. The SF-36 is widely acknowledged as the gold standard generic measure of health status. It is being increasingly used as an outcome measure in a range of clinical trials to determine the effectiveness of interventions. The results of this study highlight some limitations of the SF-36 for this purpose. The marked floor and ceiling effects demonstrated in half of the dimensions, and across the range of disease severity, indicate a limited ability to discriminate between patients with multiple sclerosis at a single point in time. The poor responsiveness of the dimension scores suggest that it is limited in detecting change over time in people with moderate to severe disability. These results highlight the need for “generic” measures to be tested for specific populations and for specific purposes. We suggest that trials evaluating health status in multiple sclerosis should supplement the use of the SF-36 with other relevant and scientifically sound instruments to maximise the validity of health measurement.