Statistics from Altmetric.com
Amyotrophic lateral sclerosis (ALS) is a fatal neurodegenerative disease leading to the majority of patients dying within a limited number of years.1 ,2 The disability score ALS Functional Rating Scale-Revised (ALSFRS-R)3 has been used to measure the course of the disease and to assess the efficacy of candidate treatments in clinical trials.4–8 However, its ability to be a reliable outcome in disease-modifying clinical trials has been argued due to the following limitations: (1) non-linear, thus prone to biases;9 (2) multidimensional, thus unfit as single score and unable to satisfy rigorous measurement standards;9 (3) floor-effect, thus unable to capture late-stage clinical changes.1 ,10 ,11
Recently, the ALS Milano-Torino staging (ALS-MITOS) system was proposed as a novel tool to measure the progression of ALS.1 The ALS-MITOS system, which is based on the assessment of four functional domains assayed by the ALSFRS-R, was found to reliably identify relevant stages of disease in patients according to the number of functions lost, to be consistent with sequential disease progression, to overcome non-linearity and multidimensionality features of ALSFRS-R, and to correlate well with patients’ quality of life and health service costs.1 This study aimed to confirm the robustness of the ALS-MITOS system and validate its ability to be a proxy of long-term survival.
Patients and measures
Data were derived from a randomised, double-blind, placebo-controlled clinical trial (RCT) on the efficacy of recombinant human erythropoietin in ALS that failed to demonstrate differences between active treatment and placebo, in primary and in secondary outcomes.12 Briefly, patients aged 18–75 years and diagnosed with probable laboratory-supported, probable or definite ALS according to El Escorial revised criteria were enrolled in 25 Italian ALS centres. Inclusion criteria were onset ≤18 months and slow vital capacity ≥70% of predicted value at the screening visit. The primary outcome was time to death, tracheotomy or >23 h non-invasive ventilation (NIV) daily for 14 consecutive days at 12 months. Development and evaluation of the ALS-MITOS system have been previously described.1 Briefly, four key domains included in the ALSFRS-R scale (walking/self-care, swallowing, communicating and breathing) were selected to obtain a tool capable of capturing the lost functions. Each domain has a threshold reflecting the loss of function in the specific ALSFRS-R subscores. Values of 0 (below threshold) or 1 (above threshold) were assigned, and the stages were determined as the sum of those values across the four domains. Five stages were defined: stage 0 indicates functional involvement and no loss of function in any domain; stages 1–4 represent the number of domains in which function was lost; and stage five is death (see online supplementary table).1
Baseline patients’ features were reported as percentages for dichotomous data, means with SD and medians with value range for continuous data. The ALS-MITOS system was reported using descriptive statistics for each participant at baseline, and at 6-month and 12-month follow-up. Sensitivity and specificity with the corresponding 95% CI, and the area under the curve of the receiver operating characteristic (ROC) curves of the ALS-MITOS system stages as well as ALSFRS-R decline at 6 months were calculated, and compared to the primary outcome (survival, tracheotomy or >23 h NIV) at 12 and 18 months, where an event was defined by the occurrence of at least one among the three. Predicted probability with the corresponding 95% CI of the ALS-MITO system at 6 months for any event at 12 and 18 months was computed through logistic regression models.
Among the 200 patients randomised, nine withdrew consent before 6-month follow-up and five between 6-month and 12-month follow-up (figure 1). Table 1 reports patients’ demographic and clinical features at the baseline visit. In the analysis of progression based on the ALS-MITOS system, 153 of 200 patients (76.5%) at baseline were in stage 0 and 44 patients (22%) were in stage 1, while only three patients (1.5%) were in stage 2 and none in stage 4. Among the 44 patients in stage 1, 38 (86.4%) had lost function in walking/self-care, 3 (6.8%) in breathing, 2 (4.6%) in swallowing and 1 (2.3%) in communicating. Among the 191 patients who performed the 6-month follow-up visit or died, 91 (47.6%; 49.7% of 141 spinal and 42% of 50 bulbar onset patients, p=0.35) progressed to advanced stages of disease (figure 2A). Among the 186 patients who performed the 12-month follow-up visit or died, 136 (73.1%; 74.5% of 137 spinal and 69.4% of 49 bulbar onset patients, p=0.49) progressed to advanced stages of disease and 50 (26.9%) did not. Overall, 42 patients (22.6%) were in stage 0, 50 patients (26.9%) were in stage 1, 34 patients (18.3%) were in stage 2, 29 patients (15.6%) were in stage 3, nine patients (4.8%) were in stage 4 and 22 patients (11.8%) were in stage 5 (death). At 12 months, among the 122 subjects with at least one function lost, 109 (89.3%) had loss of autonomy in walking/self-care, 56 (45.9%) in breathing, 44 (36.1%) in swallowing and 32 (26.2%) in communicating. Patients at stage 1 at baseline had a higher, although non-statistically significant, probability of transition to advanced stages (75%) or of dying (20%) than those at stage 0 (70.6% and 9.8%, respectively, p=0.07) (figure 2B). No transition to a lower stage of disease was observed between baseline and 12 months (two patients moved from stage 1 at baseline to stage 0 at 6 months and returned to stage 1 at 12 months, due to fluctuations in walking/self-care domain).
The analysis for any stage of disease progression from baseline to 6-month follow-up showed sensitivity of 82% (95% CI 71% to 93%, n=37/45), specificity of 63% (95% CI 55% to 71%, n=92/146), a positive predictive value of 41% and a negative predictive value of 92% of the ALS-MITOS system for the primary outcome (death, tracheotomy or >23 h NIV) at 12 months. In other words, based on the disease worsening at 6 months from the first observation at baseline, irrespective of the number of functions lost at 6 months, the ALS-MITOS system was able to correctly identify one patient who would have an event included in the primary outcome at 12 months with a probability of 82%. At the same time, the system was able to correctly identify with a probability of 63% one patient who would not have an event included in the primary outcome. When we analysed the number of functions lost at 6 months compared with baseline, the ALS-MITOS system was found to have a probability to predict the primary outcome at 12 months with sensitivity of 62% and specificity of 67% in patients who advanced by 1 stage (eg, from stage 0 to 1, or from 1 to 2, etc), and of 62% and 94%, respectively, in those who advanced by two stages (eg, from stage 0 to 2, or from 2 to 4, etc). The corresponding positive predictive values were 22% for patients who advanced by one stage and 68% for those who advanced by two stages, showing a higher probability to have an event at 12 months for patients who advanced by two stages compared to a progression of one stage.
The corresponding analysis at 18 months for any stage of progression from baseline to 6 months showed sensitivity of 71% (95% CI 61% to 82%, n=50/70), specificity of 68% (95% CI 60% to 77%, n=76/111), a positive predictive value of 59% and a negative predictive value of 79% of the ALS-MITOS system for the primary outcome (death, tracheotomy or >23 h NIV). When we analysed the number of functions lost at 6 months compared with baseline, the ALS-MITOS system was found to have a probability to predict the primary outcome at 18 months with sensitivity of 52% and specificity of 71% in patients who advanced by one stage (eg, from stage 0 to 1, or from 1 to 2, etc), and of 44% and 96%, respectively, in those who advanced by two stages (eg, from stage 0 to 2, or from 2 to 4, etc).
The comparison between ALS-MITOS system progression and ALSFRS-R decline over the first 6 months from baseline showed that the area under the ROC curves of the ALSFRS-R decline was slightly higher than that of the ALS-MITOS system progression at both 12 months (0.87 and 0.81, respectively) and 18 months (0.84 and 0.75, respectively) (figure 3). The corresponding values adjusted by the baseline ALSFRS-R score did not significantly change (0.88 and 0.83 for the ALSFRS-R and ALS-MITOS system, respectively, at 12 months; 0.85 and 0.79 for the ALSFRS-R and ALS-MITOS system, respectively, at 18 months). Based on these curves, the best cut-off of the ALS-MITOS system to predict at 6 months the primary outcome at 12 and 18 months was one (ie, loss of at least one function from baseline to 6 months), whereas that of the ALSRFS-R ranged between six to nine points of decline (table 2).
As expected, we observed that the higher the stage of disease based on the ALS-MITOS system at 6 months, the higher the probability to reach the primary outcome within 12 and 18 months (figure 4). Irrespective of the stage of the disease at 6 months, the predicted probability to have an event included in the primary outcome at 18 months was always higher than at 12 months. For example, patients who had lost two functions at 6 months had a predicted probability of about 40% to have an event of the primary outcome within 12 months and of about 65% within 18 months.
The present study confirmed the validity and robustness of the ALS-MITOS system to be consistent with sequential disease progression and revealed its ability to be a proxy of long-term outcome. Moreover, it has demonstrated reliability and ease of use across a large number of ALS centres. In particular, we found that, according to the ALS-MITOS system, in our population of 200 patients with ALS with an onset ≤18 months, almost 50% at 6 months and more than 70% at 12-month follow-up progressed to advanced stages of disease. According to previous findings,1 the probability of transition from a given stage was highest for the next higher stage, with the majority of patients reporting a loss of function in walking/self-care (89%), almost 50% in breathing and more than 30% in swallowing, at 12 months.
The progression of disease from baseline to 6 months as defined by the ALS-MITOS system appeared to be a good proxy of death, tracheotomy or >23 h NIV at 12-month and 18-month follow-up in terms of sensitivity and predicted probability. Furthermore, the ALS-MITOS system correlated well with the 6-month ALSFRS-R decline, showing similar accuracy. Accordingly, the analysis of the ROC curves demonstrated that, compared to the best cut-off values of the ALSFRS-R decline over 6 months (ie, values within the range from six to nine, where six would be the best cut-off to increase sensitivity), the best cut-off values of the ALS-MITOS system showed satisfactory sensitivity and specificity values. Moreover, it should be considered that there is no validated cut-off for the ALSFRS-R and, mostly, that the clinical meaningfulness of a difference in one or two points can be influenced by the baseline score of the study population and, overall, may be hardly considered clinically meaningful.
During the past 7 years, 18 RCTs4–6 ,12–26 on 14 different drugs were performed and failed to show positive results. Although progressively better organised, all these RCTs showed similar issues, which could have influenced the negative findings, including the choice of the outcomes.27 ,28 The ultimate end point in ALS is the time to death; however, it is not a feasible outcome unless very long trials are designed. Indeed, more than 95% of patients enrolled in RCTs were alive at 6 months and 75–90% at 12 months.4–6 ,12–14 ,17 ,18 ,21 ,23 As emphasised in the TCH346 trial,24 different clinical practice can influence survival, beyond the need of more than 1000 patients followed up for more than 3 years, to have an adequate power to use it as primary outcome. To overcome this issue, that and many other trials18 ,21 ,22 ,24 ,25 chose the mean ALSFRS-R decline as primary outcome. However, ALSFRS-R also showed major limitations, beyond the already mentioned non-linearity, multidimensionality and inability to detect late-stage clinical changes.1 ,9–11 Indeed, a mean decline of one point/month up to 12 months from randomisation was reported in all the longitudinal studies,4–6 ,12–26 and a plateau between 12 and 18 months in the only two studies including both those follow-up periods,6 ,14 ,16 ,17 suggesting that the mean ALSFRS-R decline cannot serve as a reliable outcome either at 6 months, since only differences of one to three points between treatment groups can be observed, or at 18 months, being unable to capture slower changes in later stages. Consistently, the authors of the glatiramer acetate trial conducted on 366 patients with ALS,21 speculated that the choice of the ALSFRS-R decline as primary outcome might be one of the reasons underlying the negative results of the study, likely being either insensitive or inappropriate.
A further important problem of the ALSFRS-R is how to consider deaths. Accordingly, in the CoQ10 trial,22 the significantly lower decline of the mean ALSFRS-R observed in the CoQ10-treated arm was almost entirely driven by outlying values from five deceased placebo patients, while the median decline that is insensitive to outliers, was lower in the placebo arm. Consistently, in the dexpramipexole trial,5 the authors used as primary outcome a combined assessment of survival and change from baseline in the ALSFRS-R score, emphasising that the ALSFRS-R decline alone could not be proposed as primary outcome due to the well known non-linearity over time, and because it should be assumed that discontinuations, including deaths, were random and non-informative.
Combined outcome measures including survival, tracheotomy, NIV and/or selected domains of the ALSFRS-R scale showed better performances compared to survival or mean ALSFRS-R decline alone in several studies.4 ,5 ,12 ,17 ,19 ,20 For example, in one of the lithium trials,19 the composite outcome of severe loss of autonomy, defined as the time from inclusion to death/tracheotomy or at least two of three selected ALSFRS-R scores (ie, ≤1 for swallowing, ≤1 for walking, or ≤2 for respiratory insufficiency), was able to capture a percentage of events around 15% at 6 months and above 30% at 12 months. Similarly, in the acetyl-l-carnitine trial,4 the composite primary outcome of loss of self-sufficiency, defined as inability to swallow, cut food/handle utensils and walk, was observed in more than 80% of patients at 12 months.
The ALS-MITOS system1 was developed to be a reliable composite outcome measure. The present study, performed to validate its use in RCTs, demonstrated its good correlation with death, tracheotomy or >23 h NIV, as well as with the mean ALSFRS-R decline over 6 months. Moreover, the ALS-MITOS system can be a more clinically meaningful outcome in short-term and medium-term trials compared with ALSFRS-R. Indeed, it is based on the unidirectional progression of function impairment in four key domains (walking/self-care, swallowing, communicating and breathing), the achievement of which means their loss without any possibility to recover.
What an ALS trial is expected to demonstrate in a time frame useful to patients is that the candidate treatment can slow (or hopefully halt) the progression of the disease. While it is arguable to consider a drug that only prolongs survival by a few months as efficacious, a difference of two-point decline over 6 months at the ALSFRS-R can hardly be considered clinically meaningful. Moreover, the ALSFRS-R is not unequivocally unidirectional, typically showing fluctuations over time that can be due to treatments ameliorating certain symptoms (eg, drooling). Conversely, a difference in terms of function lost or retained can more reliably reflect the real effect of a treatment in ALS. Our findings demonstrated that the ALS-MITOS system can reliably correlate loss of functions at 6 months to survival at 12 and 18 months. Meeting the requirements of modern approaches to ALS trials,28 it can be considered a novel and valid outcome measure.
The authors thank all the patients and their families, and the Fondazione Italiana di Ricerca per la SLA - Sclerosi Laterale Amiotrofica (AriSLA). The authors thank Domenico Arenella and Laura Ferradini for administrative and coordination assistance, and the entire study group for the participation.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.