Article Text
PDF
Abstract
Objective To develop and validate a reliable patient-reported scale that grades the severity of disability in Charcot-Marie-tooth disease (CMT), from an in-depth analysis of patient and healthcare provider perspectives on what mobility changes constitutes mild, moderate and severe disability.
Design In this prospective, cross-sectional study, a 19-item Disability Questionnaire was developed following literature and expert review. Between 2011 and 2012, the Disability Questionnaire was provided to healthcare providers experienced in CMT attending national scientific meetings, and to patients self-registered with the Inherited Neuropathy Consortium—Rare Diseases Clinical Research Consortium on-line contact registry. Provider and patient responses were compared utilising a two-sided unpaired t test with Bonferroni correction. The questionnaire was then assessed for validity, reliability and unidimensionality.
Results We analysed 259 Disability Questionnaires (167 patients, 92 providers); these showed perfect agreement between patient and provider responses on qualitative descriptions of disability, but significant differences in quantitative responses on items corresponding to minimal or severe disability (p<0.001). Validity and test–retest reliability of the questionnaire was excellent (Cronbach's α=0.96; intraclass correlation coefficients (ICC)=0.977 (0.951 to 0.993). Exploratory factor analysis and the Mokken Scaling Procedure supported the unidimensionality of the mobility-Disability Severity Index.
Conclusions The mobility-Disability Severity Index is a unique instrument, categorising disability from the patient's perspective, and will undergo further cross-validation studies in CMT.
- Hmsn (Charcot-Marie-Tooth)
- Neuromuscular
- Neuropathy
Statistics from Altmetric.com
Introduction
There are currently no approved curative therapies for the 31 million people in the USA alone who suffer from chronic peripheral neuropathy.1 Specifically, there is a critical need for therapies to reduce the disability associated with neuropathy; however, definitions of disability can differ between patients and providers. Inherited neuropathies (Charcot-Marie-tooth disease or CMT), with their known genetic causes, offer the opportunity to develop targeted disability treatments with potential for broader clinical applications. Our NINDS-funded Inherited Neuropathy Consortium (INC) (1U54-NS065712) is leading efforts to assess disability progression in CMT through the development of validated outcome measures. There are validated measures that delineate the profound social, physical and mental impact of disability on patients: clinician-developed measures ascribing to the WHO ICF framework can quantify the degree of disability depending on participation in life activities2 and resulting functional limitations3–6; however, there is no measure that provides the patient's direct input on the severity of their disability and its impact on their mobility, as measured by the utilisation of increasingly more visible and complicated mobility aids to ambulate, and how they perceive this dependence. The objective of this study was to develop and validate a practical index to grade the severity of disability in CMT, by critically analysing patient and healthcare provider perspectives on what changes in mobility constitutes mild, moderate and severe disability.
Design
Development of Disability Questionnaire
This was a prospective, cross-sectional instrument development and validation study. A systematic literature review helped identify items pertinent for disability assessment in neuropathy.2–6 The identified items were reviewed by genetic counsellors and neurologists with experience in CMT to develop a standardised 19-item Disability Questionnaire (see online supplementary eTable 1). Each item pertained to how the responder would classify the disability severity in a generic patient, based on prespecified functional limitations and utilisation of mobility aids. Questionnaire responders were asked to answer each question qualitatively (choosing from ‘none,’ ‘mild,’ ‘moderate,’ or ‘severe’ disability) and quantitatively (0–10 scale, with 10 being the most disabled). Standardised demographic data (age, gender, race and ethnicity) and self-reported clinical data (CMT genotype) were also collected.
Participants
The eligibility criterion specified was that the responder either be a patient with CMT, or a healthcare provider who takes care of patients with CMT. The Disability Questionnaire was distributed by paper to healthcare professionals (attending the Peripheral Nerve Society annual meeting in Potomac, Maryland, USA in June 2011, 4th International CMT Consortium meeting in Potomac, Maryland, USA in July 2011, or the Muscular Dystrophy Association national Clinical Conference in March 2012; providers were requested to only participate once). The Disability Questionnaire was also sent via an online link to the 450 patients worldwide, who self-registered with the INC—Rare Diseases Clinical Research Consortium (RDCRC) contact registry: (https://rarediseasesnetwork.epi.usf.edu/registry/index.htm); the online link was open from August 2011 through September 2011. Additionally, 25 participants from the patients who completed the Disability Questionnaire were randomly selected to receive the questionnaire again in an 8–12-week period to assess test–retest validity. Both the pen-and-paper questionnaire and the online questionnaire were self-completed by the participant, to minimise mode effect bias.
Standard protocol approvals, registrations and patient consents
The protocol was approved and monitored by the institutional ethics review board at Stanford Hospital and Clinics. All participants received a summary sheet (paper or online) explaining the study; this sheet notified them that completion of the questionnaire constituted informed consent to participate in the study.
Statistical analysis
Descriptive statistics to characterise the study sample were calculated using Stata-IC V.12.1 (College Station, Texas, USA) and SPSS (V.20.0). An a priori decision was made to exclude samples with less than 50% data imputed, as per standard scale development analyses. An unpaired t test with Bonferroni correction for multiple testing was used to compare mean responses between patients and healthcare providers. The internal consistency of the patient responses to the Disability Questionnaire, a measure of validity, was calculated using the Cronbach's α coefficient. Test–retest reproducibility of the Disability Questionnaire, a measure of reliability, was calculated through intraclass correlation coefficients (ICC) using a two-factor mixed effects model and type consistency.7 ,8 The structural validity of the Disability Questionnaire was assessed using Stata-IC V.12.1 through exploratory factor analysis using Principal-Component Analysis and oblique rotation for correlated factors,9 followed by the Mokken Scaling Procedure,10 ,11 a non-parametric item response theory model analogous to Rasch analysis, to test unidimensionality of the final scale.
Results
Two hundred and ninety questionnaires, plus 16 retest patient questionnaires, were collected during the study period. Of the 290 questionnaires, 184 were from online participants and 97 from healthcare providers at scientific meetings, figure 1 (participant flow chart) shows the breakdown of questionnaires received and ultimately analysed. Thirty-one questionnaires were ruled ineligible for the following reasons: 12 questionnaires did not meet the eligibility criterion (these 12 responders were neither a patient with CMT, nor a healthcare provider taking care of patients with CMT) and 19 questionnaires had less than 50% of items completed in the questionnaire and were excluded from the final analysis, as specified by our a priori methodology. Item 6 also had a large number of missing responses (only 67 responses collected), and was excluded from further analyses. The demographic and clinical characteristics of the 167 patients and 92 healthcare providers who completed the 19-item Disability Questionnaire are summarised in table 1.
Demographics
Participant flow chart (CMT, Charcot-Marie-tooth disease).
The distribution of CMT types of the patients responding through the INC RDCRC is presented in table 2. Historical distributions from large academic centres are presented alongside for comparison.12 ,13 The genotype distribution of our patient population was similar to what has been reported in larger academic centres in the USA and the UK, validating that we reached our intended population.
CMT genotype distributions
Patient and healthcare provider responses to the Disability Questionnaire are provided in table 3, with summary statistics for quantitative and qualitative responses. A two-sided unpaired t test with Bonferroni correction for multiple testing was used to determine significant differences in the mean quantitative scores per question between the patient group and the healthcare providers’ group. In addition, mean scores from data points excluded from the final analysis are provided alongside to allow the reader to draw their own conclusions. On average, there was perfect agreement on qualitative disability classifications between healthcare providers and patients. Quantitative responses differed significantly between the groups on five items: 3, 4, 5, 7 and 9, which corresponded to either extreme of the disability severity spectrum.
Disability Questionnaire responses
The internal consistency of our 19-item Disability Questionnaire, a measure of validity, was excellent (Cronbach's α coefficient of 0.96). The questionnaire also exhibited excellent test–retest reliability (ICC=0.977) with narrow 95% CIs (0.951 to 0.993). The 167 patient responses to the 19-item Disability Questionnaire underwent exploratory factor analysis, using Principal-Components factor Analysis. The Kaiser-Meyer-Olkin value14 was excellent at 0.91, supporting factorability of the correlation matrix. The factor analysis revealed two components with eigenvalues >1, explaining 75% of the total variance. Oblique rotation, to account for correlated factors, revealed a simple structure with all but two items (items 4 and 7) loading substantially on one factor. Next, the Mokken Scaling Procedure assessed the 19-item Disability Questionnaire's structural validity by calculating Loevinger's15 scalability coefficient Hi. The results stratified to a 17-item scale where none of the Hi coefficients fell below the threshold of 0.4. Items 4 and 7 (‘Uses a shoe insert in one foot’ and ‘Uses a shoe insert on both feet’) formed the second scale, with Hi coefficients greater than 0.8 (see online supplementary eTable 2).
We noted that the items on scale 2 corresponded to minimal disability. Based on these results, and the patients’ qualitative responses, we recategorised the 19 items of the Disability Questionnaire into a four-scaled mobility-Disability Severity Index (mDSI): score 0=no disability (no qualifying items); score 1=minimal disability (items 4 and 7); score 2=mild disability (items 3 and 10); score 3=moderate disability (items 2, 8, 9, 11, 12, 13, 14, 15, 16, 17 and 19); and score 4=severe disability (items 1, 18 and 5). The internal consistency of the mDSI was good (Cronbach's α=0.72). Assessment of the structural validity of the mDSI using the Mokken Scaling Procedure yielded Hi coefficients between 0.4 and 0.8 for items within the minimal, mild, moderate and severe scales. To facilitate scoring, the individual items of the mDSI and its scores are presented in table 4.
The mobility Disability Severity Index (mDSI)
Discussion
Recent studies have shown that chronic disability accounts for approximately 50% of the US health burden16; this parallels the global shift in disease burden from deaths to disability.17 In order to develop meaningful interventions to reduce the prevalence and impact of disability, it is necessary to understand how disability is defined by not only the physician, but also the patient. In this study, we characterised the responses from patients with CMT to a 19-item Disability Questionnaire to develop a valid and reliable mDSI. The mDSI is a patient-reported instrument: based on what patients state that they use to ambulate, we can classify them as minimally, mildly, moderately or severely disabled.
The NINDS-funded INC has validated two measures to quantify disability in CMT: The CMT Neuropathy Score (CMTNS)4 ,5 is a composite score derived from historical, physical examination and nerve conduction data, which can distinguish between mild, moderate and severe disability in CMT. The CMT Pediatric Examination Score (CMTPedS),6 is an age-adjusted, Rasch-built functional score, and can assess disability in children with CMT. While both measures allow the clinician to determine the degree of disability in their patients with CMT, neither measure offers the patients’ perspective on their disability status. For example, in the CMTNS, it has been inferred that the wearing of ankle-foot orthotics (AFOs), the use of additional walking aids such as canes or ‘sticks’ or the use of wheelchairs, are appropriate markers for ‘mild,’ ‘moderate’ or ‘severe’ disability, respectively. Similarly, the CMTPedS relies on stringent physician-determined functional assessments for its scoring algorithms. The mDSI was developed in order to include the patients’ voices among our clinical outcome measures.
One of the more interesting findings in this study was the concordance and discrepancy between provider and patient responses. We found that on average, patients and healthcare providers show perfect agreement on qualitative descriptions of disability; however, there is significant disagreement on quantifying the extent of disability, especially on the extreme ends of ability. For example, healthcare providers scored using a shoe insert on one foot (item 4) or both feet (item 7) at 1.9 and 2.3, respectively; patients, however, scored these, respectively, at 3.1 and 3.4 (p<0.0001). It emphasises that utilisation of any ambulatory aid, no matter how minimal they may seem to a physician, has a significant impact on the patient, and represents a significant change from the norm, from the patient's perspective. Consequently, even though the exploratory factor analysis and the Mokken Scaling Procedure showed that the remaining 17 items of the 19-item Disability Questionnaire had better agreement for the concept of ‘disability’ than those two items; we strongly believed these two items must be included in the final mDSI scale as it was clearly meaningful to the patient with CMT. Conversely, ‘bed bound’ status was scored at 9.5 by providers and 8.1 by patients (p=0.0004), perhaps indicating the disability paradox18: despite seemingly severe physical limitations imposed by the descriptor ‘bed bound’, patients are unwilling to characterise that numerically as the worst disability. This is important to bear in mind while scoring this instrument in future studies: as clinicians, we may state that a patient would strongly benefit from an orthotic insert, an AFO or cane, but the patient may refuse to use these aids, since they perceive the use as a visible marker of significant disability. To maintain the ‘patient-reported’ aspect of this instrument, we recommend that scoring of the mDSI reflect what the patients use, rather than what the clinician thinks they should be using.
Comparing table 4 to the CMTNS,5 we note that there is excellent concordance between that clinician-determined neuropathy severity scale (scores 1, 2, 3 and 4) and the patient-determined mDSI (minimal, mild, moderate and severe). One notable exception is that in the CMTNS, using a cane is assigned severity grade 3; based on our results, using a single 1-point cane is classified by most patients (and physicians) as mild disability; using a 4-point cane or two canes changes the perception to moderate disability. We plan to cross-validate the mDSI against our validated clinical measures of disease severity in CMT, the CMTNS and the CMTPedS. This will help identify any floor or ceiling effects of the mDSI in the CMT population. If there are consistent differences between the mDSI and other outcome measures, we will alter the severity classifications of the other instruments based on the patient's interpretations from the mDSI.
Our study has several limitations. For the items in our questionnaire, we used the adaptive mobility aids seen in our clinics and referenced by the Americans with Disability Act; however, focus group sessions could have identified de novo mobility aid items that are meaningful to patients with CMT. We focused on ambulation since we aimed to create a scale that was as universally applicable as possible; we may have sacrificed specific components that could be quite meaningful to a patient's perception of disability, such as hand function. The participant's self-completion of the paper questionnaire or online questionnaire should minimise mode effect bias, but these may still exist and complicate the duplication of our results. The lack of responses to item 6 (‘walks noticeably differently than other people, but does not require any aid to walk’), may indicate that patients internalised the questions despite the instructions, and only responded to questions perceived as pertinent to them. The online contact registry we used to target patients is open to patients worldwide, and the genotype distribution of our patient population was similar to what has been reported in larger academic centres in the USA and the UK,12 ,13 thus validating that we reached our intended population, and supporting the generalisability of our findings to all patients with CMT. However, we may have missed potential bias from language/cultural interpretation of our disability items, based on our aggregate analysis. Our planned validation studies in the consortium clinics will be able to address this issue, by comparing item means within specific regions. One striking difference was the large number of self-identified ‘CMT type 2’. Since type 2 presents phenotypically as an axonal neuropathy, there is a possibility that some of those patients screened in the study were axonal neuropathies of another aetiology, and not CMT. To demonstrate that these results are truly applicable to neuropathies of other aetiologies, cohort-specific studies are required.
Conclusion
The mDSI, developed from the perspective of patients with CMT, is a reliable and valid patient-reported measure of disability. Results of on-going cross-validation studies will determine the utility of the mDSI as an anchor to characterise disease severity in patients with CMT. Longitudinal correlations between the mDSI and other outcome instruments will enable us to determine whether particular components of some clinical outcome measures correlate better than others with patient-reported data, and whether these should be emphasised in future trials. The mDSI may have broad applications to other neuromuscular diseases as well, and would thus benefit from future validation studies in the appropriate cohorts. The mDSI will allow us to incorporate the neuromuscular patients’ voices into our outcome measures, to fully capture both benefit and harm to patients in future clinical trials.
Acknowledgments
The authors are grateful for the assistance of the Rare Diseases Clinical Research Network, as well all healthcare providers who participated anonymously in the study. The authors would also wish to thank the patients for their participation in the study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors SR had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. SR, MS and CS contributed to the study concept and design. MS and CS contributed to acquisition of the data. SR and MS contributed to analysis and interpretation of the data. SR Drafted the manuscript. SR, MS, EF, RC and CS critically revised the manuscript for important intellectual content. SR performed the statistical analysis. MS and CS supervised the study.
-
Funding This study was supported by the NIH: K23-NS072279—SR; NINDS/ORD U54-NS065712, the Charcot Marie Tooth Association, and the Muscular Dystrophy Association—MS; and The A. Alfred Taubman Medical Research Institute—EF.
-
Competing interests None.
-
Ethics approval Institutional ethics review board at Stanford Hospital and Clinics.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement Technical statistical code for dataset is available from the corresponding author. All data from this study is virtually housed in the highly secure Database Management and Coordinating centre (DMCC) at the University of South Florida, and will be made available as deemed appropriate on request to the Internal Advisory Board for the Inherited Neuropathy Consortium.