PARTICIPANTS AND METHODS

Two groups of subjects were studied.

Individuals with MS in the community

People with clinically definite or probable MS (Poser criteria)¹⁹ were identified during the course of an epidemiological study in Counties Wexford and Donegal in the Republic of Ireland. A total of 366 cases were ascertained in both counties (126 in Wexford and 240 in Donegal), and of these 71 (56.3%) in Wexford and 101 (42.1%) in Donegal were interviewed and agreed to take part in the study. There were no demographic differences between study participants and non-participants.

Cross-sectional study

Kurtzke EDSS scores and MSFC scores were assessed by C McG. Subjects were also asked to complete the MSIS-29; the London Handicap scale (LHS), a generic measure of handicap²⁰; and Beck’s Depression Inventory (BDI-II), a validated, generic measure of depression.²¹

Test-retest reliability

The 71 participants enrolled in the study from County Wexford were asked to complete a second MSIS-29 form six months after the first assessment, by postal survey. They were also asked whether their physical condition had improved, remained stable, or deteriorated since the first assessment. This information was used in the determination of item test-retest reliability.

Individuals with MS attending outpatients

Cross-sectional assessment

Of patients attending the neurology outpatient department with clinically definite or probable MS (Poser criteria),¹⁹ 102 were seen by both authors and completed MSIS-29 questionnaires; the Kurtzke EDSS scores were also assessed.

Responsiveness

Of these patients, 55 were prospectively followed over six months; at each visit they were asked to complete an MSIS-29 questionnaire and the EDSS was assessed. In 41 cases an MSIS-29 questionnaire and a Kurtzke EDSS score were obtained at both time points. The EDSS was assessed by both authors without knowledge of the MSIS-29 score; the authors had trained in EDSS assessment together to improve inter-rater reliability. On the basis of the difference in EDSS scores between the two time points (and before the MSIS-29 was scored), the 41 patients were assigned to one of two categories: either “Changed” (if the two EDSS scores differed by 1.0 or more), or “Static” (if the difference between the two scores was not more than 0.5 points). The ethics committee of St. Vincent’s University Hospital, Dublin, granted approval for the study.

Statistical methods

The groups were assessed using standard psychometric techniques including:^3,22

• data quality: percentage missing data and percentage computable scores

• scaling assumptions: item mean scores and standard deviations and item to total correlations

• acceptability as determined by score range, mean scores, floor/ceiling effects, and skewness

• reliability: Cronbach’s alpha was estimated for all samples. The standard error of measurement was used to calculate the 95% confidence interval for individual scores. Item test-retest reliability was assessed in the Wexford population who completed a second MSIS-29 by postal survey, and also in the “Static” group of hospital patients whose Kurtzke EDSS scores had not changed over a six month period

• validity: the convergent validity of the MSIS-29 physical scale was tested against the Kurtzke EDSS scale, the MSFC, and the LHS in the community population, and against the Kurtzke EDSS scale only in the outpatient population. The divergent validity of the MSIS-29 physical scale was examined against the MSIS-29 psychological scale and the BDI-II. The convergent validity of the MSIS-29 psychological scale was assessed in the community participants only against the BDI-II, and the divergent validity was measured against the EDSS and the MSIS-29 physical scale. A Spearman’s rank correlation coefficient was used to test the strength of the correlations. Rasch analysis was used to examine correlations that were greater than expected²³

• responsiveness: the responsiveness of the MSIS-29 scale was evaluated by comparing the mean scores at the assessments at the two time points in the “Changed” subgroup of the 41 hospital outpatients. A Wilcoxin Signed Rank test was used to assess the significance of the mean differences between groups. Effect sizes (mean change score / standard deviation of Time 1 scores) were calculated for the “Changed” and “Static” groups.²⁴

RESULTS

Populations

The demographic details of the 172 participants from the epidemiological study and the 102 hospital attendees are given in table 1. The former group is more representative of the community MS population, with a wider range of disability and a more realistic proportion of primary progressive MS patients.

MSIS-29 psychometric properties

Data quality

The percentage missing data for the MSIS-29 was very low at 0–2.8%. Scores were computable for all participants (table 2).

Scaling assumptions

Items within each sample had similar means and standard deviations. Frequency results for each item were well distributed. Item-total correlations were acceptable but consistently less for the MSIS-29 psychological score in comparison with the physical score (table 2).

Acceptability

Floor/ceiling effects were very low for all samples. Score ranges spanned from 86.1% to 98.68% of the possible range. The mean scores were near the scale midpoints and were not notably skewed (table 2).

Reliability

All estimates of reliability using Cronbach’s alpha were in excess of the recommended 0.80. The 95% confidence intervals for the MSIS-29 physical scales were narrower in all groups compared with the MSIS-29 psychological scale, indicating the expected increase in precision due to the number of items (table 2).

Validity

The convergent validity of the MSIS-29 physical scale with other measures of physical impairment is indicated by the high correlations found with the Kurtzke EDSS (0.704), the LHS (0.843), and the MSFC (0.577). In a test of divergent validity, the correlation of the MSIS-29 physical scale and the BDI-II at 0.399 is higher than would have been expected. Rasch analysis, a statistical method using interval level measurement, was applied to the MSIS-29 results of patients scoring between 0 and19 on the BDI-II scale (minimally to mildly depressed), and between 20 and 60 (moderately to severely depressed). The analysis confirmed that the MSIS-29 score items were operating similarly in the two populations, and therefore were not unduly influenced by mood. The greater than expected correlation is likely to highlight the problems associated with applying parametric analyses to interval measure scores.

The MSIS-29 psychological scale scores correlated highly with the BDI-II, as predicted (0.799). The divergent validity of the MSIS-29 psychological impact scale was confirmed when assessed against the EDSS (0.095) and the MSFC (0.020). The correlation between the MSIS-29 physical and psychological scales was 0.438, showing, as expected, that they measure similar but distinct constructs.

Test-retest reliability

Community postal survey. In the community study, 58 of 71 (81%) patients who received a repeat MSIS-29 responded; 36 felt their condition had remained stable since the first assessment, 12 felt their condition had deteriorated, and four reported an improvement. In the 36 stable responders there were no statistical differences between the mean MSIS-29 scores at the two time points. Those who felt their condition had deteriorated reported a mean increase of 7.98 points on the MSIS-29 physical scale (SD 15.15, p  =  0.034) but no significant change in the MSIS-29 psychological scores. In the four improved patients, the MSIS-29 physical scores decreased by 13.4 (p  =  0.017) and their MSIS-29 psychological scores were unchanged.

“Static” hospital group. There were 24 patients with unchanged EDSS scores. Their mean MSIS-29 physical score at the first assessment was 27.1 compared with 29.1 at the second assessment. There was no significant change in their mean MSIS-29 psychological scores from first to second assessments.

Responsiveness

The responsiveness of the MSIS-29 was measured for the 17 hospital attendees who were assessed as having worsened or improved by one point or more in the EDSS between the two time points. The mean change in the EDSS was 1.35 (range 1–2.5). The mean difference in the MSIS-29 physical impact score from first to second examination was 18.16 (Z  =  −3.259, p  =  0.001); the mean difference in the MSIS-29 psychological impact scores was 12.25 (Z  = −1.967, p  =  0.049). Table 3 shows the mean MSIS-29 physical and psychological scores at both time points, together with the range in scores and mean differences for both the “Static” and the “Changed” groups according to change in the EDSS scores. The effect size for the change in the MSIS-29 physical score in the “Static” group was 0.43 compared with 1.25 for the “Changed” group, further supporting the responsiveness to change of measure. The moderate effect size noted in the “Static” group is likely to represent the lack of fine sensitivity of the EDSS scale in detecting minimal change in MS disability. The differences in the mean MSIS-29 physical score over the six month period when plotted against change in the EDSS (Figure 1) indicates that the relationship between these measures is highly correlated (MSIS-29 physical change v EDSS change, R²  =  0.9, p  =  0.014).

DISCUSSION

The objective of this study was to assess the psychometric properties of the MSIS-29 physical and psychological scales in two different populations with MS in Ireland, and to determine their effectiveness as an outcome measure in population based studies and clinical trials. The MSIS-29 is an easy instrument to administer, taking approximately five to 10 minutes to complete. The questionnaire was acceptable to patients, and no particular areas of concern were raised with regard to the content or the phrasing of the individual items.

The psychometric analysis revealed the consistency of the results observed for both community and outpatient based populations; the results are similar to the published results of the reference populations.^17,18 The scale is of use in the full range of impairments, disabilities, and handicaps seen in the MS population (other than dementia). The correlations of the MSIS-29 physical scale with MSFC and the EDSS, and of the psychological scale with the BDI-II, indicate that the measures identify the appropriate impact status. Initial assessment suggested a higher than expected correlation between the MSIS-29 physical scale and the BDI-II; further analysis by Rasch techniques showed that the physical impact scale operated similarly in depressed and in euthymic patients, allowing it to be used with confidence irrespective of the patient’s affect.

The results of the test-retest analysis and of the study of responsiveness to change suggest that the MSIS-29 is not only of use in cross-sectional studies but may also be used longitudinally to monitor disease progression. The MSIS-29 was responsive for both change perceived by participants in the community sample, and change as assessed by the EDSS in the outpatient sample. However, change perceived by participants and change rated by observers may not be concordant, and measuring both in the same cohort would help to establish further the sensitivity to change of the MSIS-29. The results are therefore consistent with a recent report of the relationship between another self-assessed measure of disability, the Guy’s Neurological Disability Scale, and with the Kurtzke EDSS scale, strengthening the case for the use of self-assessed measures of disease impact in future clinical trials.²⁵

Further validation by other MS centres is required, particularly in the longitudinal assessment of the MSIS-29 to confirm or deny its responsiveness in comparison with recognised instruments such as the EDSS and the MSFC. The scale offers a promising, scientifically developed, self-reported measure of both the physical and the psychological impacts of MS, for use in clinical trials, population based studies, and continuing evaluation of individual patients.