Results

TOTAL COHORT ANALYSES WITH COMBINED DATA

EDSS

”2 year EDSS difference” demonstrated significant benefits favouring the 22 μg dose (p=0.026) and a tendency in favour of 44 μg (p=0.052) over placebo. AUC^SUM calculations similarly showed that 22 μg IFNβ-1a was beneficial compared with placebo (p=0.046). Although the size of the treatment effect was similar, significance was not reached with the 44 μg dose (p=0.064).

The median AUC^CHANGE was +0.06, +0.05, and +0.48 EDSS-year for 44 μg, 22 μg IFNβ-1a, and placebo, respectively (table 1 A). Both doses conferred significant advantages over placebo (p=0.013 and 0.008 for 44 μg and 22 μg IFNβ-1a, respectively), with an estimated treatment effect of −0.5 EDSS-year for each dose (95% CI −0.9 to−0.1 and −0.8 to −0.1, respectively) (fig 1A-B).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

AUC^CHANGE data obtained on total visits with intention to treat analysis for the entire cohort (n=560). AUC^CHANGE EDSS: (A) basic statistics and (B) treatment comparisons in EDSS-years. AUC^CHANGE SNRS: (C) basic statistics and (D) treatment comparisons in SNRS-years. AUC^CHANGE AI: (E) basic statistics and (F) treatment comparisons in AI-years. (Basic statistics: histogram=group mean, horizontal bar=median, vertical line=interquartile range. Treatment comparisons: histogram=estimated mean, vertical line=95% CI, p value from ANOVA on ranks).

SNRS

There were no significant group differences in efficacy based on either “2 year SNRS difference”, or AUC^SUManalysis. With AUC^CHANGE, the 44 μg treatment arm improved by a median of +0.17 SNRS-year compared to deteriorations of −0.25 and −1.68 SNRS-year for the 22 μg dose and placebo cohorts, respectively (table 1 B). There were significant effects in favour of 44 μg IFNβ-1a over placebo (p= 0.038), with an estimated mean benefit of +2.5 SNRS-year (95% CI +0.1 to +4.9) (fig 1 C-D).

AI

There were no significant group differences for either the “2 year AI difference”, or AI AUC^SUM. On AUC^CHANGE, the 44 μg dose (median of 0 AI-year) was significantly superior to placebo (median worsening of +0.11 AI-year) (p=0.039) with a mean benefit of −0.4 AI-year (95% CI -0.7 to -0.1) (table 1 C, fig 1 F).

TOTAL COHORT ANALYSES WITH SCHEDULED VISIT DATA

Utilising data from scheduled visits only, AUC^SUManalysis on EDSS scores disclosed a trend favouring 22 μg IFNβ-1a over placebo (p=0.058). AUC^CHANGE calculations showed that on EDSS scores, there were significant benefits of both doses over placebo (p=0.024 and 0.014 for 44 μg and 22 μg, respectively). For SNRS and AI data, treatment with IFNβ-1a 44 μg was also superior to placebo.

SUBGROUP ANALYSES

EDSS

For patients with entry EDSS⩽3.5, treatment comparisons showed 22 μg to be better than placebo, for both “2 year EDSS difference” (p=0.016) and AUC^SUM analyses (p=0.043). Using AUC^CHANGE, there were significant effects in favour of both treatment doses compared with placebo (p=0.036 and 0.016 for 44 μg and 22 μg IFNβ-1a), with estimated benefits of −0.4 and −0.5 EDSS-year, respectively (95% CI −0.8 to 0; −0.8 to −0.1) (table 2 A, fig 2 A-B).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

AUC^CHANGE data obtained on total visits with intention to treat analysis for the subgroups of low (EDSS(⩽3.5, n=466) and high (EDSS>3.5, n=94) entry disability. AUC^CHANGE EDSS (A) basic statistics and (B) treatment comparisons in EDSS-years. AUC^CHANGE SNRS (C) basic statistics and (D) treatment comparisons in SNRS-years. AUC^CHANGE AI (E) basic statistics and (F) treatment comparisons in AI-years. (Basic statistics: histogram=group mean, horizontal bar=median, vertical line=interquartile range. Treatment comparisons: histogram=estimated mean, vertical line=95% CI, p value from ANOVA on ranks).  

For patients with baseline EDSS>3.5, neither conventional “2 year EDSS difference”, nor AUC analyses, showed significant differences between treatment arms, although AUC^CHANGE showed an estimated mean benefit for the 44 μg dose over placebo of −1.0 EDSS-year (95% CI −1.9 to 0) (fig 2 A-B).

SNRS

For patients with entry EDSS⩽3.5, neither “2 year SNRS difference”, AUC^SUM, nor AUC^CHANGE showed any significant distinctions between treatment arms (table 2 B, fig 2C-D).

For patients with a baseline EDSS >3.5, there was a significant effect in favour of the 44 μg dose over placebo using conventional “2 year SNRS difference” (p=0.016). AUC^SUM did not show differences between treatment arms. AUC^CHANGE confirmed significant effects of 44 μg IFNβ-1a versus placebo (p=0.028) with a mean benefit of +9.0 SNRS-years (95% CI +2.9 to +15.1) (fig 2 C-D).

AI

Neither “2 year AI difference”, AUC^SUM, nor AUC^CHANGE showed any significant treatment effects over placebo for either subgroup (table 2 C, fig 2 E-F). However, for AUC^CHANGE, the 44 μg dose conferred an estimated mean saving of −0.9 AI-year over placebo (95% CI −1.6 to -0.1).

Subgroup scheduled visit data analyses

Treatment comparisons between subgroups using data restricted to scheduled visits showed significant benefits of both treatment doses for patients with entry EDSS⩽3.5 on EDSS assessments (p=0.021 to 0.043), and favourable effects on SNRS scores with high dose 44 μg IFNβ-1a over placebo in patients with EDSS>3.5 (p=0.034).

Discussion

Whereas it is generally agreed that the primary outcome in phase III immunomodulatory trials in relapsing-remitting multiple sclerosis must be clinical,17 commonly employed end points associated with both disability and relapse assessment can be difficult to define and interpret.8 16 Meaningful clinical milestones, such as the conversion to secondary progression,18 19 are not realistic end points for relapsing-remitting multiple sclerosis cohorts within relatively short study periods of 2 to 3 years. Currently available clinical rating scales are variously flawed,20-22 with the EDSS being the most criticised.13 14 20 23 24 Other scales (including SNRS and AI) have been less thoroughly evaluated.25 We have previously proposed that the AUC summary measure statistic may solve some of these problems in a clinically meaningful way by indexing and accounting for the total morbidity change experienced by patients during the course of a trial.8

In this paper, we have analysed data from the PRISMS trial using two AUC methods, AUC^SUM and AUC^CHANGE. AUC^SUM summates the total disability data serially over the study period and data independence is assured (the independence is lost in analyses involving any change in disability scores as each data point is influenced by its preceding values). However, it would be expected to be insensitive to small changes in short trials if there were a large disability range in the cohort at baseline. On the other hand, AUC^CHANGE obtained by normalising the AUC^SUM value to the baseline rating is sensitive to in trial changes and has been previously utilised in a neurorehabilitation pilot study.26 However, neurological improvements and deteriorations may cancel and the technique is susceptible to unstable scores at trial entry, a problem partly resolved by ensuring a stable run in period.

In the present study, comparison of the two AUC techniques showed that AUC^SUM essentially confirmed the “2 year disability difference” analysis (treatment benefit over placebo for EDSS, but not for SNRS or AI data) without improved responsiveness. This can be attributed to the small in study changes relative to the wide range of disability (EDSS 0–5) at baseline. By contrast, the increased sensitivity of AUC^CHANGE for detecting positive therapeutic effects not only showed savings in terms of EDSS-years for both IFNβ-1a doses compared with placebo, but also significant beneficial effects in favour of the high dose (44 μg three times weekly), using SNRS and AI data.

AUC analysis has several advantages over conventional techniques for this type of study. The method can be applied to any clinical rating scale employed. Because it incorporates all available serial data, it should be statistically more powerful than end points of conventional mean disability change. By accounting for both the magnitude and duration of relapses as well as improvements and progressive deteriorations, it is more clinically meaningful, as it provides a measure of the patient’s total disability experience. Furthermore, cost effectiveness can be assessed in terms of disability-years. These factors allow a more complete intention to treat evaluation.

We examined the neurological scores from all visits (combined data), as well as those assessed only from scheduled appointments (scheduled visit data). The second was performed to reduce any sampling bias due to the reduced mean relapse rate in the treatment arms compared with placebo, and to separate out, at least in part, the effects of transient disability changes. The results in favour of therapy were similar with or without the inclusion of data from unscheduled visits.

We analysed our higher disability cohort separately in this study because there is accumulating evidence that patients with active relapsing-remitting multiple sclerosis and an EDSS >3.5 (when gait dysfunction ensues) are at a higher risk for disability progression.27 These patients have a tendency to deteriorate more rapidly13 and to be unresponsive to steroids and standard doses of IFNβ.28 The AUC^CHANGE analysis of this subgroup disclosed that patients on placebo experienced strikingly more disability on EDSS, SNRS, and AI than their treated counterparts. AUC^CHANGE confirmed the conventional end point of 2 year SNRS difference in showing significant treatment effects only for the 44 μg dose. Furthermore, from the estimated means and confidence intervals, trends were apparent for similar dose effects with the EDSS and AI data, although statistical significance was not reached due to the relatively few subjects. Overall, these results support the notion that subjects with significant disability require higher doses than patients at the lower end of the EDSS (with mainly impairment), to obtain similar benefits.27

The differential outcomes of the two treatment doses on disability as assessed by the different rating scales are intriguing and difficult to explain. They may be confounded by the modest correlations within individual patients across changes in rating scores,29 and the relatively lack of comparative data on SNRS and AI.25 Moreover, in the PRISMS Study cohort, the SNRS scoring system generates a more continuous dataset than the EDSS or AI and thus, for the relatively small subgroup of subjects with baseline EDSS>3.5 (n=94), may be more sensitive to the treatment effects of the 44 μg dose.

AUC analyses might be regarded as having certain disadvantages. For example, the method summarises all the disability experienced by a patient during a trial, but does not distinguish between permanent neurological dysfunction and the transient disability associated with relapses. However, we regard this as being useful for assessing the effects of potential therapies in patients with relapsing-remitting multiple sclerosis in whom significant transient morbidity may occur over short periods despite the slow accumulation of irreversible disability.19 30 AUC analyses also provide no information on disability trends over time: a subject with a period of improvement followed by deterioration may have the same AUC score as one with the opposite temporal sequence, or another in which the morbidity remains unaltered throughout. This issue, as well as the problem of erroneous treatment failures wrongly assigned by confirmed progression and Kaplan-Meier analyses, can only be solved by evaluating individual disability/time plots. The question of an optimal sampling rate, including concerns of the signal to noise of rating scales employed at high assessment frequency,17 also needs to be considered. Although considered appropriate for analysing graded data,12 AUC analyses do not eliminate problems peculiar to the clinical rating tools utilised, as discussed previously. However, these difficulties likewise apply to the disability end points presently utilised in clinical trials. It has been suggested that improved clinical rating tools, preferably with continuous scales, are required (for example, maximum ambulatory distance, timed 8 metre walk, nine hole peg test, paced auditory serial addition task, questionnaire based assessments).31-35 The numerical data derived from such scales is likely to be even more informative when subjected to AUC analyses.