Background and aim: The psychometric properties of rating scales are sample dependent and need evaluations in different samples. The Multiple Sclerosis Impact Scale (MSIS-29), a new patient based rating scale for multiple sclerosis (MS) was predominantly developed from a community based sample derived from the MS Society. A number of important patient characteristics of this sample remain unknown. The aim of the study was to evaluate five psychometric properties of the MSIS-29 in three hospital based samples: people admitted for rehabilitation, people admitted for intravenous corticosteroid treatment for MS relapses, and people with primary progressive MS.
Methods: People with MS were recruited from the three clinical settings. They completed several health measures. MSIS-29 data were evaluated for data quality, scaling assumptions, acceptability, reliability and validity, and compared with those from a previously reported community based study.
Results: A total of 233 people (rehabilitation =53; corticosteroids =76; primary progressive =104) completed questionnaires. In all samples, missing data were low (≤2.2%), scaling assumptions were satisfied, and reliability was high (≥0.91). Correlations between the MSIS-29 and other scales were consistent with a priori hypotheses. Findings were consistent with those from the community samples.
Conclusions: The psychometric properties of the MSIS-29 are consistent across three hospital based samples, and similar to those in the community samples. These findings further support its use as an outcome measure in different clinical settings.
- multiple sclerosis
- Multiple Sclerosis Impact Scale (MSIS-29)
- health outcomes measurement
- psychometric methods
- MS, multiple sclerosis
- MSIS-29, Multiple Sclerosis Impact Scale
Statistics from Altmetric.com
- multiple sclerosis
- Multiple Sclerosis Impact Scale (MSIS-29)
- health outcomes measurement
- psychometric methods
The Multiple Sclerosis Impact Scale (MSIS-29) is a new measure of the physical and psychological impact of MS from the patient’s perspective (a copy of the MSIS-29 is on the journal web site; www.jnnp.com).1 It was developed using the standard psychometric approach of reducing an item pool generated de novo from people with multiple sclerosis (MS). Psychometric evaluation of the scale was conducted in two large independent postal surveys of randomly selected, geographically stratified members of the Multiple Sclerosis Society.
In those samples, the MSIS-29 satisfied all recommended psychometric criteria for rigorous measurement.1 However, the psychometric properties of health measurement instruments are sample dependent and cannot be established in a single study.2 Morever, the psychometric properties of the MSIS-29 were tested in randomly selected members of the MS Society. Although this sampling frame has the advantage of being truly representative of MS Society members, the disadvantage is that the percentage of people with a neurologist confirmed diagnosis of clinically definite MS cannot be confirmed. Furthermore, the disease type of those with MS and the representativeness of people who join charitable groups is unknown. This study aimed to examine the psychometric properties of the MSIS-29 in three hospital based samples of people with a confirmed diagnosis of MS in comparison with a community sample of people with MS.
The sample was derived from adults with clinically definite MS who were consecutively admitted to the National Hospital for Neurology and Neurosurgery (NHNN) between February 2000 and July 2001 for rehabilitation and intravenous corticosteroid treatment of relapses, and a postal survey of people from the Primary Progressive MS (PPMS) database of the NHNN. Patients in the rehabilitation and corticosteroid samples completed the MSIS-29 on admission along with other health measures and demographic questions.
People admitted for rehabilitation and corticosteroids were excluded from the study if they appeared to have severe cognitive impairment substantiated by neuropsychological testing, other comorbid disabling disorders, or were not English speaking.
All patients completed four self report measures: the MSIS-29 described above, the 59 item Functional Assessment of Multiple Sclerosis (FAMS3), the Medical Outcomes Study 36 item Short-Form Health Survey (SF-364), and the 12 item General Health Questionnaire (GHQ-125). In addition, rehabilitation and corticosteroid patients were rated by a neurologist on the Expanded Disability Status Scale (EDSS6).
The psychometric properties of the MSIS-29 were evaluated using standard methods. These methods are fully described in previous publications.1,7,8 In short, we examined five psychometric properties: data quality, scaling assumptions, acceptability, reliability, and validity.
Fifty eight people were consecutively admitted to the rehabilitation unit. Five people were excluded because of cognitive impairment. A total of 53 people (91%) admitted for rehabilitation completed the questionnaires on admission.
Seventy eight people were consecutively admitted for intravenous corticosteroid treatment for relapses. Two people were excluded for cognitive impairment. A total of 76 (97%) people completed the questionnaires on admission.
Questionnaires were sent to 119 people, 104 completed questionnaires were returned (87% response rate).
Table 1 shows the characteristics of the patients in the three samples. There were some differences in characteristics between the samples. The majority of patients in the rehabilitation and corticosteroid samples were women, whereas the majority of the PPMS sample was men. PPMS patients were older than the rehabilitation or corticosteroid samples, as might be anticipated.9 Other differences between the samples were consistent with expectations. The majority of patients in the rehabilitation sample used an aid or a wheelchair and the majority of patients in the corticosteroid sample walked unaided or used an aid. Many PPMS patients used an aid or a wheelchair, although some walked unaided. More than half of the corticosteroid sample was employed full time. More than half of the rehabilitation sample and an even greater proportion of the PPMS sample were retired because of their MS.
Data quality (table 2)
For all samples, missing data for items were low for both scales, and total scores could be computed for at least 99% of patients.
Scaling assumptions (table 2)
For all samples, frequency distributions for item response scales were quite symmetrical; items within each scale had similar mean scores and standard deviations. Item-total correlations were satisfactory, although the lower range of item-total correlations of the physical scale in the rehabilitation and PPMS sample was <0.40. The physical scale of the rehabilitation sample had the lowest item-total correlations.
Acceptability (table 2)
For all samples, psychological scale scores spanned the entire scale range and scale scores were not notably skewed. Floor and ceiling effects were small and were considerably less than the recommended maximum of 15%.10
There were some differences between the samples for the physical scale scores. The lowest score was 3.8 (PPMS sample), 7.5 (corticosteroid sample), and 22.5 (rehabilitation sample). In this rehabilitation sample, only 75% of the MSIS-29 physical score range was used.
Although the mean scores were near the scale midpoint for all samples, the mean physical scale score of the rehabilitation sample was higher than the other two groups indicating greater physical impact. The standard deviation of the physical scale in the rehabilitation group was also lower than the other two groups. The mean psychological scale score of the PPMS sample was lower than the other two groups.
Reliability (table 2)
All estimates for Cronbach’s α exceeded the standard criteria of 0.80.11 The standard error of measurement is an estimate of error that can be used to calculate the 95% confidence interval (CI) for individual patient scores. For all samples, the 95% CIs for the MSIS physical scales were narrower than MSIS psychological scales, indicating that greater measurement precision is obtained when more items are used.
Validity (table 3)
Correlations between the physical and psychological scales of the MSIS-29 were 0.47 in the rehabilitation sample, 0.66 in the corticosteroid sample, and 0.65 in the PPMS sample, indicating that the two scales measure related but distinct constructs.
Correlations with other measures and variables provide evidence for the convergent and discriminant validity of MSIS-29 as a measure of the physical and psychological impact of MS. The direction, magnitude and pattern of correlations were generally consistent with predictions. For example, in the corticosteroid and PPMS samples, the MSIS-29 physical scale correlated most highly with the FAMS mobility and SF-36 PF scales, and lowest with the SF-36 MH scale. In the corticosteroid sample, the MSIS-29 physical scale also correlated highly with the EDSS. In all samples, the MSIS-29 psychological scale correlated most highly with SF-36 mental health, FAMS emotional wellbeing and GHQ scales.
There were some notable differences in the pattern of correlations across the three samples. Correlations of the MSIS-29 physical scale with SF-36 PF, FAMS mobility, and EDSS were lower in the rehabilitation sample (0.27 to 0.63) than the other samples (0.69 to 0.88). Correlations of the MSIS-29 psychological scale with FAMS mobility and EDSS were low as predicted in the rehabilitation and primary progressive samples (0.14 to 0.50), but higher in the corticosteroid sample (0.48 to 0.61).
The aim of this study was to evaluate the psychometric properties of the MSIS-29 in three hospital based samples, and compare these with the data from the community based sample. The psychometric properties of the MSIS-29 scales were similar in all four samples and demonstrate that the MSIS-29 satisfies standard criteria for reliable and valid measurement. The results demonstrate that the measurement properties of MSIS-29 in a range of hospital based samples are very similar to those obtained in a community context1 and support the appropriateness of the instrument in a range of clinical settings.
The differences in mean scores among the three samples were as expected. Patients in the rehabilitation sample scored highest (worst) on the MSIS physical scale. Patients in the PPMS sample scored lowest (best) on the MSIS psychological scale, supporting a previous finding that PPMS patients show better psychological functioning than patients with secondary progressive MS.12 Vleugels et al suggested that this may be because PPMS patients feel they are more likely to know the rate of their physical deterioration, and that relapses are unlikely.
There were some other notable differences in the psychometric characteristic of the MSIS-29 among the samples. In the rehabilitation sample, the correlation between MSIS physical scale and the SF-36 PF scale and the EDSS was lower than expected. Closer inspection of findings in this sample revealed a substantial floor effect for the SF-36 PF (31%) and limited variability in EDSS score (range 5.5 to 8.5). In contrast, MSIS-29 physical scores spanned nearly the full scale range and showed minimal floor and ceiling effects. Even when cases with floor and ceiling effects were removed from the analysis, the correlation between the MSIS-29 physical and SF-36 PF was virtually unchanged at r=−0.55 . This suggests that the floor and ceiling effects did not attenuate the correlations between scales. However, the limited score distribution of the SF-36 (range 0 to 65) may have attenuated the correlations between scales, as correlations are proportionally sensitive to scale ranges.
Although this study provides further evidence for the use of MSIS-29 as a rigorous measure of the physical and psychological impact of MS, ongoing evaluations are still required to further define its role in clinical practice and research. As traditional psychometric methods were used to develop and evaluate the MSIS-29, and it is difficult to interpret small differences in performance across samples, it is important that newer psychometric methods such as Rasch item analysis13 and Item Response Theory models14 are applied to the MSIS-29 data. These will generate item calibrations in the different samples permitting a more extensive comparison of scale performance.
We wish to thank the people with MS who freely participated in the studies that provided the data used in this study, the Multiple Sclerosis Society of Great Britain and Northern Ireland for their collaboration, and Gordon Ingle for help with the PPMS database. This study was funded by a grant from the NHS Health Technology Assessment Programme (but the views and opinions expressed do not necessarily reflect those of the NHS Executive) and the MS Society of Great Britain and Northern Ireland.
Review history and Supplementary material
Multiple Sclerosis Impact Scale (MSIS-29): reliability and validity in hospital based samples
A Riazi, J C Hobart, D L Lamping, R Fitzpatrick, and A J Thompson
MS Impact Scale (MSIS-29)
The Appendix is available as a downloadable PDF (printer friendly file).
If you do not have Adobe Reader installed on your computer,
you can download this free-of-charge, please Click here
Competing interests: none declared.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.