Article Text

PDF

The multiple sclerosis impact scale (MSIS-29) is a reliable and sensitive measure
  1. C McGuigan,
  2. M Hutchinson
  1. Department of Neurology, University College Dublin and St Vincent’s University Hospital, Dublin, Ireland
  1. Correspondence to:
 Dr C McGuigan
 St Vincent’s University Hospital, Elm Park, Dublin 4, Ireland; cmcguigandoctors.org.uk

Abstract

Objective: The aim of this study was to assess the psychometric properties of the Multiple Sclerosis Impact Scale (MSIS-29) for patients in the community and in a hospital setting.

Methods: During an epidemiological study, 172 people with multiple sclerosis (MS) were examined and completed the MSIS-29, the London Handicap Scale, and Beck’s Depression Inventory; disability was assessed by the Kurtzke Expanded Disability Status Score (EDSS) and the Multiple Sclerosis Functional Composite. At the hospital neurology clinic, 102 MS patients completed the MSIS-29 and EDSS assessments were performed; 41 of these patients had repeat evaluations six months later. The psychometric properties of the MSIS-29 were examined.

Results: In the 172 community and the 102 hospital patients the psychometric properties of the MSIS-29 were satisfactory, with high convergent and low divergent validity. It was significantly responsive to change in the contexts of self-reported change (p<0.034) and EDSS worsening (p<0.001). The MSIS-29 physical score did not change over time when the EDSS was stable, and increased significantly in proportion to EDSS deterioration (p  =  0.014).

Conclusions: The psychometric properties of the MSIS-29 are acceptable; it is a valuable outcome measure in intervention studies of patients with MS.

  • multiple sclerosis
  • Multiple Sclerosis Impact Scale (MSIS-29)
  • outcome measures
  • psychometric properties
  • MS, multiple sclerosis
  • MSIS, Multiple Sclerosis Impact Scale
  • EDSS, Expanded Disability Status Score
  • MSFC, Multiple Sclerosis Functional Composite
  • FAMS, Functional Assessment of Multiple Sclerosis
  • MSQLI, Multiple Sclerosis Quality of Life Inventory
  • UKNDS, United Kingdom Neurological Disability Scale
  • MSQOL, Multiple Sclerosis Quality of Life
  • HRQOL, Health Related Quality of Life
  • LHS, London Handicap Score
  • BDI, Beck’s Depression Inventory

Statistics from Altmetric.com

The assessment by patients of their own state of well-being and of the limitations imposed by a condition is increasingly recognised as a valid part of the assessment of therapeutic interventions in chronic diseases. Two instruments commonly used in outcome assessment and administered by physicians are the Kurtzke Expanded Disability Status Scale (EDSS)1 and the Multiple Sclerosis Functional Composite (MSFC).2 The EDSS, developed before the acceptance of psychometric methods of scale development,3 has a number of problems, including: an ordinal scale; rater variability; poor reliability; insensitivity to change at certain levels; and an emphasis on mobility status.4,6 The more recently introduced measure, the Multiple Sclerosis Functional Composite (MSFC), has been shown to be a sensitive and responsive instrument in the combined assessment of cognitive, upper limb, and gait disorder.2,7,8

Comprehensive self-reported measures of disease impact, both generic and specific to the disease, have been widely assessed in multiple sclerosis (MS). The Short Form Health Survey Scale has been frequently used in the MS population, but it does not evaluate many symptoms specific to MS, and the high frequency of floor and ceiling effects indicates a lack of responsiveness in this disease.9,11 We evaluated measures specific to the disease, including: the Functional Assessment of MS (FAMS)12; the Multiple Sclerosis Quality of Life Inventory (MSQLI)13; the UK Neurological Disability Scale (UKNDS)14; the MS Quality of Life Score (MSQOL-54)15; and the Health Related Quality of Life in MS Measure (HRQOL-MS).16 Although useful in varying degrees, none of these measures used scientifically based psychometric methods of scale construction; the first such validated measure of disease impact specific to the disease and rated by the patient, the Multiple Sclerosis Impact Scale (MSIS-29), was published in 2001.17 The MSIS-29 consists of 29 questions of which 20 address the physical impact component and nine assess the psychological impact; a combined score can be generated, or both components can be reported separately. The reliability and validity of the MSIS-29 were assessed by a postal survey of members of the United Kingdom MS society, and its responsiveness was evaluated in a patient population undergoing treatment.17 The validity of the measure in persons undergoing rehabilitation in MS has also been assessed.18

The MSIS-29 has not been independently validated in other MS populations. It is the aim of this paper to examine the reliability, validity, and responsiveness of the MSIS-29 in MS patients both in a community setting and attending the hospital neurology outpatient department.

PARTICIPANTS AND METHODS

Two groups of subjects were studied.

Individuals with MS in the community

People with clinically definite or probable MS (Poser criteria)19 were identified during the course of an epidemiological study in Counties Wexford and Donegal in the Republic of Ireland. A total of 366 cases were ascertained in both counties (126 in Wexford and 240 in Donegal), and of these 71 (56.3%) in Wexford and 101 (42.1%) in Donegal were interviewed and agreed to take part in the study. There were no demographic differences between study participants and non-participants.

Cross-sectional study

Kurtzke EDSS scores and MSFC scores were assessed by C McG. Subjects were also asked to complete the MSIS-29; the London Handicap scale (LHS), a generic measure of handicap20; and Beck’s Depression Inventory (BDI-II), a validated, generic measure of depression.21

Test-retest reliability

The 71 participants enrolled in the study from County Wexford were asked to complete a second MSIS-29 form six months after the first assessment, by postal survey. They were also asked whether their physical condition had improved, remained stable, or deteriorated since the first assessment. This information was used in the determination of item test-retest reliability.

Individuals with MS attending outpatients

Cross-sectional assessment

Of patients attending the neurology outpatient department with clinically definite or probable MS (Poser criteria),19 102 were seen by both authors and completed MSIS-29 questionnaires; the Kurtzke EDSS scores were also assessed.

Responsiveness

Of these patients, 55 were prospectively followed over six months; at each visit they were asked to complete an MSIS-29 questionnaire and the EDSS was assessed. In 41 cases an MSIS-29 questionnaire and a Kurtzke EDSS score were obtained at both time points. The EDSS was assessed by both authors without knowledge of the MSIS-29 score; the authors had trained in EDSS assessment together to improve inter-rater reliability. On the basis of the difference in EDSS scores between the two time points (and before the MSIS-29 was scored), the 41 patients were assigned to one of two categories: either “Changed” (if the two EDSS scores differed by 1.0 or more), or “Static” (if the difference between the two scores was not more than 0.5 points). The ethics committee of St. Vincent’s University Hospital, Dublin, granted approval for the study.

Statistical methods

The groups were assessed using standard psychometric techniques including:3,22

  • data quality: percentage missing data and percentage computable scores

  • scaling assumptions: item mean scores and standard deviations and item to total correlations

  • acceptability as determined by score range, mean scores, floor/ceiling effects, and skewness

  • reliability: Cronbach’s alpha was estimated for all samples. The standard error of measurement was used to calculate the 95% confidence interval for individual scores. Item test-retest reliability was assessed in the Wexford population who completed a second MSIS-29 by postal survey, and also in the “Static” group of hospital patients whose Kurtzke EDSS scores had not changed over a six month period

  • validity: the convergent validity of the MSIS-29 physical scale was tested against the Kurtzke EDSS scale, the MSFC, and the LHS in the community population, and against the Kurtzke EDSS scale only in the outpatient population. The divergent validity of the MSIS-29 physical scale was examined against the MSIS-29 psychological scale and the BDI-II. The convergent validity of the MSIS-29 psychological scale was assessed in the community participants only against the BDI-II, and the divergent validity was measured against the EDSS and the MSIS-29 physical scale. A Spearman’s rank correlation coefficient was used to test the strength of the correlations. Rasch analysis was used to examine correlations that were greater than expected23

  • responsiveness: the responsiveness of the MSIS-29 scale was evaluated by comparing the mean scores at the assessments at the two time points in the “Changed” subgroup of the 41 hospital outpatients. A Wilcoxin Signed Rank test was used to assess the significance of the mean differences between groups. Effect sizes (mean change score / standard deviation of Time 1 scores) were calculated for the “Changed” and “Static” groups.24

RESULTS

Populations

The demographic details of the 172 participants from the epidemiological study and the 102 hospital attendees are given in table 1. The former group is more representative of the community MS population, with a wider range of disability and a more realistic proportion of primary progressive MS patients.

Table 1

Participant characteristics for each population

MSIS-29 psychometric properties

Data quality

The percentage missing data for the MSIS-29 was very low at 0–2.8%. Scores were computable for all participants (table 2).

Table 2

Psychometric properties of the MSIS-29 physical and psychological scales in each population

Scaling assumptions

Items within each sample had similar means and standard deviations. Frequency results for each item were well distributed. Item-total correlations were acceptable but consistently less for the MSIS-29 psychological score in comparison with the physical score (table 2).

Acceptability

Floor/ceiling effects were very low for all samples. Score ranges spanned from 86.1% to 98.68% of the possible range. The mean scores were near the scale midpoints and were not notably skewed (table 2).

Reliability

All estimates of reliability using Cronbach’s alpha were in excess of the recommended 0.80. The 95% confidence intervals for the MSIS-29 physical scales were narrower in all groups compared with the MSIS-29 psychological scale, indicating the expected increase in precision due to the number of items (table 2).

Validity

The convergent validity of the MSIS-29 physical scale with other measures of physical impairment is indicated by the high correlations found with the Kurtzke EDSS (0.704), the LHS (0.843), and the MSFC (0.577). In a test of divergent validity, the correlation of the MSIS-29 physical scale and the BDI-II at 0.399 is higher than would have been expected. Rasch analysis, a statistical method using interval level measurement, was applied to the MSIS-29 results of patients scoring between 0 and19 on the BDI-II scale (minimally to mildly depressed), and between 20 and 60 (moderately to severely depressed). The analysis confirmed that the MSIS-29 score items were operating similarly in the two populations, and therefore were not unduly influenced by mood. The greater than expected correlation is likely to highlight the problems associated with applying parametric analyses to interval measure scores.

The MSIS-29 psychological scale scores correlated highly with the BDI-II, as predicted (0.799). The divergent validity of the MSIS-29 psychological impact scale was confirmed when assessed against the EDSS (0.095) and the MSFC (0.020). The correlation between the MSIS-29 physical and psychological scales was 0.438, showing, as expected, that they measure similar but distinct constructs.

Test-retest reliability

Community postal survey. In the community study, 58 of 71 (81%) patients who received a repeat MSIS-29 responded; 36 felt their condition had remained stable since the first assessment, 12 felt their condition had deteriorated, and four reported an improvement. In the 36 stable responders there were no statistical differences between the mean MSIS-29 scores at the two time points. Those who felt their condition had deteriorated reported a mean increase of 7.98 points on the MSIS-29 physical scale (SD 15.15, p  =  0.034) but no significant change in the MSIS-29 psychological scores. In the four improved patients, the MSIS-29 physical scores decreased by 13.4 (p  =  0.017) and their MSIS-29 psychological scores were unchanged.

“Static” hospital group. There were 24 patients with unchanged EDSS scores. Their mean MSIS-29 physical score at the first assessment was 27.1 compared with 29.1 at the second assessment. There was no significant change in their mean MSIS-29 psychological scores from first to second assessments.

Responsiveness

The responsiveness of the MSIS-29 was measured for the 17 hospital attendees who were assessed as having worsened or improved by one point or more in the EDSS between the two time points. The mean change in the EDSS was 1.35 (range 1–2.5). The mean difference in the MSIS-29 physical impact score from first to second examination was 18.16 (Z  =  −3.259, p  =  0.001); the mean difference in the MSIS-29 psychological impact scores was 12.25 (Z  = −1.967, p  =  0.049). Table 3 shows the mean MSIS-29 physical and psychological scores at both time points, together with the range in scores and mean differences for both the “Static” and the “Changed” groups according to change in the EDSS scores. The effect size for the change in the MSIS-29 physical score in the “Static” group was 0.43 compared with 1.25 for the “Changed” group, further supporting the responsiveness to change of measure. The moderate effect size noted in the “Static” group is likely to represent the lack of fine sensitivity of the EDSS scale in detecting minimal change in MS disability. The differences in the mean MSIS-29 physical score over the six month period when plotted against change in the EDSS (Figure 1) indicates that the relationship between these measures is highly correlated (MSIS-29 physical change v EDSS change, R2  =  0.9, p  =  0.014).

Table 3

Mean scores, difference in mean scores and median scores with interquartile range for each scale at both time points displayed according to change in the Kurtzke EDSS score

Figure 1

The difference in the means of the MSIS-29 physical scores for the 41 patients enrolled in the responsiveness study plotted against the observed change in the Kurtzke EDSS score.

DISCUSSION

The objective of this study was to assess the psychometric properties of the MSIS-29 physical and psychological scales in two different populations with MS in Ireland, and to determine their effectiveness as an outcome measure in population based studies and clinical trials. The MSIS-29 is an easy instrument to administer, taking approximately five to 10 minutes to complete. The questionnaire was acceptable to patients, and no particular areas of concern were raised with regard to the content or the phrasing of the individual items.

The psychometric analysis revealed the consistency of the results observed for both community and outpatient based populations; the results are similar to the published results of the reference populations.17,18 The scale is of use in the full range of impairments, disabilities, and handicaps seen in the MS population (other than dementia). The correlations of the MSIS-29 physical scale with MSFC and the EDSS, and of the psychological scale with the BDI-II, indicate that the measures identify the appropriate impact status. Initial assessment suggested a higher than expected correlation between the MSIS-29 physical scale and the BDI-II; further analysis by Rasch techniques showed that the physical impact scale operated similarly in depressed and in euthymic patients, allowing it to be used with confidence irrespective of the patient’s affect.

The results of the test-retest analysis and of the study of responsiveness to change suggest that the MSIS-29 is not only of use in cross-sectional studies but may also be used longitudinally to monitor disease progression. The MSIS-29 was responsive for both change perceived by participants in the community sample, and change as assessed by the EDSS in the outpatient sample. However, change perceived by participants and change rated by observers may not be concordant, and measuring both in the same cohort would help to establish further the sensitivity to change of the MSIS-29. The results are therefore consistent with a recent report of the relationship between another self-assessed measure of disability, the Guy’s Neurological Disability Scale, and with the Kurtzke EDSS scale, strengthening the case for the use of self-assessed measures of disease impact in future clinical trials.25

Further validation by other MS centres is required, particularly in the longitudinal assessment of the MSIS-29 to confirm or deny its responsiveness in comparison with recognised instruments such as the EDSS and the MSFC. The scale offers a promising, scientifically developed, self-reported measure of both the physical and the psychological impacts of MS, for use in clinical trials, population based studies, and continuing evaluation of individual patients.

REFERENCES

View Abstract

Footnotes

  • Competing interests: none declared

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.