Article Text

Original research
Effectiveness of multiple disease-modifying therapies in relapsing-remitting multiple sclerosis: causal inference to emulate a multiarm randomised trial
Free
  1. Ibrahima Diouf1,2,
  2. Charles B Malpas1,3,
  3. Sifat Sharmin1,
  4. Izanne Roos1,
  5. Dana Horakova4,5,
  6. Eva Kubala Havrdova4,5,
  7. Francesco Patti6,
  8. Vahid Shaygannejad7,
  9. Serkan Ozakbas8,
  10. Sara Eichau9,
  11. Marco Onofrj10,
  12. Alessandra Lugaresi11,12,
  13. Raed Alroughani13,
  14. Alexandre Prat14,15,
  15. Pierre Duquette14,15,
  16. Murat Terzi14,15,
  17. Cavit Boz16,17,
  18. Francois Grand'Maison18,
  19. Patrizia Sola19,
  20. Diana Ferraro19,
  21. Pierre Grammond20,
  22. Bassem Yamout21,22,
  23. Ayse Altintas23,24,
  24. Oliver Gerlach25,26,
  25. Jeannette Lechner-Scott27,28,
  26. Roberto Bergamaschi29,
  27. Rana Karabudak30,
  28. Gerardo Iuliano31,
  29. Christopher McGuigan32,
  30. Elisabetta Cartechini33,
  31. Stella Hughes34,
  32. Maria Jose Sa35,
  33. Claudio Solaro36,37,
  34. Ludwig Kappos38,39,
  35. Suzanne Hodgkinson40,
  36. Mark Slee41,
  37. Franco Granella42,
  38. Koen de Gans43,
  39. Pamela A McCombe44,
  40. Radek Ampapa45,
  41. Anneke van der Walt46,47,
  42. Helmut Butzkueven46,47,
  43. José Luis Sánchez-Menoyo48,
  44. Steve Vucic49,
  45. Guy Laureys50,
  46. Youssef Sidhom51,52,
  47. Riadh Gouider52,
  48. Tamara Castillo-Trivino53,
  49. Orla Gray54,
  50. Eduardo Aguera-Morales55,
  51. Abdullah Al-Asmi56,57,
  52. Cameron Shaw58,
  53. Talal M Al-Harbi59,
  54. Tunde Csepany60,
  55. Angel P Sempere61,
  56. Irene Treviño Frenk62,
  57. Elizabeth A Stuart63,
  58. Tomas Kalincik1,3
  1. 1 CORe, Department of Medicine, The University of Melbourne, Melbourne, Victoria, Australia
  2. 2 Health and Biosecurity Unit, Commonwealth Scientific and Industrial Research Organisation, Melbourne, Victoria, Australia
  3. 3 Neuroimmunology Centre, Department of Neurology, The Royal Melbourne Hospital City Campus, Parkville, Victoria, Australia
  4. 4 Department of Neurology, Center of Clinical Neuroscience, Charles University, Praha, Czech Republic
  5. 5 General University Hospital in Prague, Praha, Czech Republic
  6. 6 Department of Medical and Surgical Sciences and Advanced Technologies, University of Catania 'G.F. Ingrassia', Catania, Italy
  7. 7 Department of Neurology, Isfahan University of Medical Sciences, Isfahan, Iran (the Islamic Republic of)
  8. 8 Dokuz Eylul University, İzmir, Turkey
  9. 9 Neurology, Hospital Universitario Virgen Macarena, Sevilla, Spain
  10. 10 Deptartment of Neuroscience, Imaging, and Clinical Sciences, Gabriele d'Annunzio University of Chieti and Pescara, Chieti, Italy
  11. 11 UOSI Riabilitazione Sclerosi Multipla, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bologna, Italy
  12. 12 Dipartimento di Scienze Biomediche e Neuromotorie, Università di Bologna, Bologna, Italy
  13. 13 Division of Neurology, Department of Medicine, Amiri Hospital, Kuwait City, Kuwait
  14. 14 CHUM MS Center, Montreal, Quebec, Canada
  15. 15 Universite de Montreal, Montreal, Quebec, Canada
  16. 16 School of Medicine, Ondokuz Mayis Universitesi, Samsun, Turkey
  17. 17 KTU Medical Faculty Farabi Hospital, Trabzon, Turkey
  18. 18 Neuro Rive-Sud, Greenfield Park, Quebec, Canada
  19. 19 Department of Neuroscience, Azienda Ospedaliero-Universitaria di Modena, Modena, Italy
  20. 20 CISSS de Chaudiere-Appalaches, Levis, Quebec, Canada
  21. 21 Nehme and Therese Tohme Multiple Sclerosis Center, American University of Beirut Medical Center, Beirut, Lebanon
  22. 22 Department of Neurology, American University of Beirut, Beirut, Lebanon
  23. 23 Department of Neurology, Koc Universitesi, Istanbul, Turkey
  24. 24 Koc University Research Center for Translational Medicine, Istanbul, Turkey
  25. 25 Department of Neurology, Zuyderland Medical Centre Sittard-Geleen, Sittard-Geleen, The Netherlands
  26. 26 School for Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands
  27. 27 University of Newcastle Hunter Medical Research Institute, Newcastle, New South Wales, Australia
  28. 28 Department of Neurology, John Hunter Hospital, Newcastle, New South Wales, Australia
  29. 29 Foundation National Neurological Institute C Mondino Institute for Hospitalization and Care Scientific, Pavia, Italy
  30. 30 Department of Neurology, Faculty of Medicine, Hacettepe University, Ankara, Turkey
  31. 31 Ospedali Riuniti di Salerno, Salerno, Italy
  32. 32 Department of Neurology, St Vincent's University Hospital, Dublin, Ireland
  33. 33 UOC Neurologia, Azienda Sanitaria Unica Regionale Marche - AV3, Macerata, Italy
  34. 34 Royal Victoria Hospital, Belfast, UK
  35. 35 Department of Neurology, Centro Hospitalar de São João, Porto, Portugal
  36. 36 Department of Neurology, ASL3 Genovese, Genova, Italy
  37. 37 Department of Rehabilitaiton, Casa di Cura Centro di Recupero e Rieducazione Funzionale Mons Luigi Novarese, Moncrivello, Italy
  38. 38 Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB) and MS Center, Neurologic Clinic and Policlinic, Departments of Head, Spine and Neuromedicine and Clinical Research, University Hospital Basel, Basel, Switzerland
  39. 39 University of Basel, Basel, Switzerland
  40. 40 Liverpool Hospital, Sydney, New South Wales, Australia
  41. 41 Flinders University, Adelaide, South Australia, Australia
  42. 42 Department of Medicine and Surgery, University of Parma, Parma, Italy
  43. 43 Groene Hart Ziekenhuis, Gouda, The Netherlands
  44. 44 UQCCR, The University of Queensland, Brisbane, Queensland, Australia
  45. 45 Nemocnice Jihlava, Jihlava, Czech Republic
  46. 46 Department of Neurology, Alfred Hospital, Melbourne, Victoria, Australia
  47. 47 Department of Neuroscience, Central Clinical School, Monash University, Melbourne, Victoria, Australia
  48. 48 Department of Neurology, Galdakao-Usansolo Hospital, Galdakao, Spain
  49. 49 Westmead Hospital, Westmead, New South Wales, Australia
  50. 50 Universitair Ziekenhuis Gent, Gent, Belgium
  51. 51 Department of Neurology, Razi University Hospital, Tunis, Tunisia
  52. 52 Department of Neurology, University of Tunis El Manar, Tunis, Tunisia
  53. 53 Instituto de Investigacion Sanitaria Biodonostia, Hospital Universitario de Donostia, San Sebastian, Spain
  54. 54 South and East Belfast Health and Social Services Trust, Belfast, UK
  55. 55 Neurology, Hospital Universitario Reina Sofia, Cordoba, Spain
  56. 56 Department of Medicine, Sultan Qaboos University Hospital, Seeb, Oman
  57. 57 Sultan Qaboos University, Muscat, Oman
  58. 58 University Hospital Geelong, Geelong, Victoria, Australia
  59. 59 Department of Neurology, King Fahad Specialist Hospital-Dammam, Khobar, Saudi Arabia
  60. 60 Department of Neurology, University of Debrecen, Debrecen, Hungary
  61. 61 Department of Neurology, Hospital General de Alicante, Alicante, Spain
  62. 62 Department of Neurology, Instituto Nacional de Ciencias Medicas y Nutricion Salvador Zubiran, Mexico City, Mexico
  63. 63 Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland, USA
  1. Correspondence to Professor Tomas Kalincik, Neuroimmunology Centre, Department of Neurology, The Royal Melbourne Hospital City Campus, Parkville, Victoria 3050, Australia; tomas.kalincik{at}unimelb.edu.au

Abstract

Background Simultaneous comparisons of multiple disease-modifying therapies for relapsing-remitting multiple sclerosis (RRMS) over an extended follow-up are lacking. Here we emulate a randomised trial simultaneously comparing the effectiveness of six commonly used therapies over 5 years.

Methods Data from 74 centres in 35 countries were sourced from MSBase. For each patient, the first eligible intervention was analysed, censoring at change/discontinuation of treatment. The compared interventions included natalizumab, fingolimod, dimethyl fumarate, teriflunomide, interferon beta, glatiramer acetate and no treatment. Marginal structural Cox models (MSMs) were used to estimate the average treatment effects (ATEs) and the average treatment effects among the treated (ATT), rebalancing the compared groups at 6-monthly intervals on age, sex, birth-year, pregnancy status, treatment, relapses, disease duration, disability and disease course. The outcomes analysed were incidence of relapses, 12-month confirmed disability worsening and improvement.

Results 23 236 eligible patients were diagnosed with RRMS or clinically isolated syndrome. Compared with glatiramer acetate (reference), several therapies showed a superior ATE in reducing relapses: natalizumab (HR=0.44, 95% CI=0.40 to 0.50), fingolimod (HR=0.60, 95% CI=0.54 to 0.66) and dimethyl fumarate (HR=0.78, 95% CI=0.66 to 0.92). Further, natalizumab (HR=0.43, 95% CI=0.32 to 0.56) showed a superior ATE in reducing disability worsening and in disability improvement (HR=1.32, 95% CI=1.08 to 1.60). The pairwise ATT comparisons also showed superior effects of natalizumab followed by fingolimod on relapses and disability.

Conclusions The effectiveness of natalizumab and fingolimod in active RRMS is superior to dimethyl fumarate, teriflunomide, glatiramer acetate and interferon beta. This study demonstrates the utility of MSM in emulating trials to compare clinical effectiveness among multiple interventions simultaneously.

  • MULTIPLE SCLEROSIS
  • STATISTICS

Data availability statement

Data are available upon reasonable request. MSBase is a data processor, and warehouses data from individual principal investigators who agree to share their datasets on a project-by-project basis. Data access to external parties can be granted at the sole discretion of each MSBase Principal Investigator (the data controllers), who will need to be approached individually for permission.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

  • Direct simultaneous comparisons of several multiple sclerosis (MS) therapies, outside of analyses of secondary data—such as network meta-analyses, are presently lacking.

WHAT THIS STUDY ADDS

  • We applied methods of causal inference to emulate a multiarm randomised trial comparing simultaneously the effectiveness of six commonly used disease-modifying therapies and no treatment among patients with relapsing-remitting MS.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

  • This study fills the gap in the lack of methodology for robust simultaneous comparison of multiple disease-modifying therapies in MS, especially where randomised trials for such comparisons are not feasible.

Introduction

Disease-modifying therapies (DMTs) play a crucial role in the management of multiple sclerosis (MS). DMTs reduce risks of relapses, disability worsening, secondary progression and delay mortality.1–4 Multiple therapies have been approved for the treatment of MS,5 and neurologists and patients can now choose from among different mechanisms of action and methods of administration. Therefore, evidence comparing effectiveness to guide the choice among DMTs is needed.

Magnitudes of the effects of different DMTs when compared with placebo, as reported by meta-analyses of randomised controlled trials, suggest that there are differences in the efficacy of these DMTs.6 Randomised clinical trials involving multiple DMTs are restricted by significant financial and practical limitations. Because of methodological challenges, comparisons of treatments in MS cohorts and registries have been limited to pairwise and triple comparisons. The difficulty in matching more than three groups leads to a dramatic loss of power and hence limited generalisability. Simultaneous comparisons of multiple DMTs in a single trial design are still lacking.

To overcome these limitations, we aimed to implement an extension of marginal structural models (MSMs) to simultaneously compare the effectiveness of multiple MS treatments across a broad spectrum of patients with MS. To this end, we have extended a methodology developed in cross-sectional data to longitudinal data with time-varying covariates.7

Materials and methods

Ethics statement

Data were obtained from the international MSBase registry (registered with WHO ACTRN12605000455662).8 Participants were required to provide informed consent as per the local regulations.

Study design

This study emulates a randomised trial comparing relapse frequency, disability worsening and disability improvement among six different DMTs, using data from MSBase (online supplemental table 1). Patients were followed from their first recorded MS clinic visit with disability data (study baseline). For each patient, the first recorded treatment epoch was analysed. Patients were censored at the time of switching/discontinuing DMT or the last recorded follow-up (online supplemental figure 1). If a patient was untreated during the first recorded epoch, this epoch was included in the analysis as ‘untreated time’. The primary analysis estimated the average treatment effect (ATE)—a hypothetical effect of treatment if the entire study population had been treated with a DMT of interest versus glatiramer acetate.9 10 Because the reference pseudo-cohort in ATE is constant, it enabled us to combine multiple pairwise comparisons of the index and reference DMTs to compare the effectiveness across multiple index DMTs simultaneously. Its estimation of the treatment effect across the entire study sample offers maximal generalisability of the result. The secondary analysis estimated the average treatment effect among the treated (ATT)—a comparison between a factual (observed) effect of a DMT among patients who were treated with that DMT and a counterfactual (not observed) effect of another DMT in the same study subpopulation.7 Therefore, the results of ATT provide information about the pairwise comparative effectiveness of different DMTs in clinical contexts in which they are typically used.

Supplemental material

Study population

In March 2019, we extracted data from 23 236 patients from 74 centres across 35 countries. The follow-up was recorded prospectively since 1 January 2006 as part of clinical practice following MSBase Study Protocol (available from www.msbase.org). We excluded centres in the lowest quintiles of data quality or generalisability.11 Included patients who were diagnosed with clinically isolated syndrome or relapsing-remitting MS12 13 and were censored at the conversion to secondary progressive MS as per their neurologists.14 The minimum required data consisted of recorded follow-up ≥1 year, ≥3 Expanded Disability Status Scale (EDSS) scores with ≥1 score recorded per year, sex, date of birth, MS onset date and MS course. We conducted a sensitivity analysis restricted to patients with available MRI information.

Variables of interest

Follow-up time was divided into intervals (bins) of 6 months. Bins with DMT recorded for ≥15 days were considered as treated. Potential confounders of treatment effect were captured in each follow-up bin (see the Statistical methods section).

Disability was quantified with the EDSS, excluding scores obtained <30 days after a relapse. Neurostatus EDSS certification was required at the participating centres.15 Where no EDSS scores were recorded within a bin, the preceding EDSS score was carried over (only 27% of bins). In our previous work in this cohort, EDSS scores in consecutive bins were highly correlated (r=0.95).16

The presence/absence of relapses, confirmed disability worsening and disability improvement during each bin, were binary variables. Relapses were recorded by treating neurologists as new symptoms or exacerbation of existing symptoms persisting for ≥24 hours, in the absence of concurrent illness/fever, and occurring ≥30 days after a previous relapse. Disability worsening was defined as an increase in EDSS by 1 step (1.5 steps if baseline EDSS=0 and 0.5 steps if EDSS >5.5) confirmed by subsequent EDSS scores over ≥12 months, as over 80% of such events correspond to long-term worsening of disability.17 Disability improvement was defined as a decrease in EDSS by 1 step (≤1.5 steps if baseline EDSS ≤1.5 and 0.5 steps if EDSS >6) confirmed over ≥12 months. No carried-over EDSS scores were used for evaluation of disability worsening or improvement. The presence/absence of new or enlarging T2 hyperintense lesions or contrast-enhancing lesions on cerebral MRI was reported by treating neurologists in some patients.

Data availability

Data Dictionary and MSBase Study Protocol are available from www.msbase.org. MSBase is a data processor. Data access to external parties can be granted at the discretion of each MSBase Principal Investigator (data controller), who will need to be approached individually for permission.

Statistical methods

We used generalised boosted regression models (GBMs) to estimate inverse probability of treatment weight (IPW) and inverse probability of censoring weight (IPCW). Machine learning methods have been shown to outperform simple regression methods for iterative variable selection.18–20 GBMs can adjust for many covariates and capture complex and non-linear relationships without overfitting.

To avoid loss of information that would limit the generalisability of our results, all treatments including those with a small number of users (such as alemtuzumab, mitoxantrone and ocrelizumab) were included when calculating the ATE weights. These DMTs with a small number of patients were, thereafter, excluded before the analysis of head-to-head comparisons of therapies. Data were analysed with R version 3.6.3, using the package GBM.

Estimation of ATE weights

ATE comparing treatment A and treatment B is estimated as: E(D[t a, t b])=E(Y[t a]−Y[t b])=E(Y[t a])−E(Y[t b]), where t a is treatment A and t b is treatment B and Y is the outcome of interest.

To calculate weights for ATE models, we first created a dummy indicator for each treatment at each bin, then fitted GBMs for each treatment. In practice, the PS (pt(X)=pr(T [t]=1|X), the probability for a patient with characteristics X of receiving treatment t) was computed for each dummy treatment indicator at each 6-month bin given their covariate history (age, sex, pregnancy status, treatment history, history of relapses, MS duration, EDSS scores, MS course and year of birth). Stabilised weights for each patient i for each given treatment t at bin k were calculated as

Embedded Image

where A is the treatment status at bin k, B represents fixed confounding variables and L represents time-varying confounding variables. Thus, the weights reflect the probability of patients’ treatment status at each bin given their demographic and disease history.7

To account for censoring due to treatment switch, conversion to secondary progressive MS or study dropout, we calculated stabilised IPCWs as a function of baseline and time-varying covariates.

Embedded Image

where C is the censoring status at time k, B represents fixed confounding variables and L represents time-dependent confounding variables. The final ATE weight was calculated as a product of the two IPWs (w ik[t]=tw ik[tcw ik[t]).

Estimation of ATT weights

Let μt a, t b equal the counterfactual mean for patients who received treatment t a had they instead received treatment t b, μt a, t b=E(Y[t b]|T=t a). Then the ATT of treatment t a relative to t b is the difference between the factual and the counterfactual means, μt a, t aμt a, t b.

We used a subsample of the population with t a and one alternative treatment t b to calculate the weight for a patient receiving the target treatment t b as the ratio of the probability of receiving the treatment actually received (ta ) and the probability of receiving tb Embedded Image . This procedure is repeated for each treatment tb≠ta . Each patient is weighted by the inverse probability of receiving the treatment they actually received (ta ) relative to the probability of receiving the alternative treatment (tb ). Like ATE weights ATT weights are the product of the IPWs and IPCWs to take into account censoring due to treatment switch or study dropout.

Comparisons of treatment effects

Baseline characteristics of the study population were described using mean (SD) for continuous variables and n (%) for categorical variables. The standardised bias or absolute standardised mean difference was used as a balance metric to assess covariate balance between treatment groups before and after weighting (overall and per bin). For a covariate k and a treatment t, the balance metric is computed as:

Embedded Image

where PSBtk is the population standardised bias, Embedded Image tk represents the mean of a covariate k for the treatment t, Embedded Image kp is the mean of the covariate in the population and Embedded Image kp is population SD for the covariate k.

We used Cox proportional hazards models with the IPWs (MSM Cox model) to compare hazards of relapses, disability worsening and disability improvement among pseudo-cohorts treated with sufficiently represented therapies versus the reference therapy. For the ATE and ATT, the main reference was ‘glatiramer acetate’, with sensitivity analyses conducted against alternative references— fingolimod and natalizumab. In addition, the weighted models were further adjusted for MS duration, EDSS or age when the covariate balance between weighted groups exceeded 0.20. Sufficiently represented therapies were defined as treatments administered to at least 200 eligible individuals.

Results

Among 23 236 eligible patients in sufficiently represented treatment groups, the mean follow-up was 2.8 years (SD 2.2) with 6.5 EDSS visits (SD 6.3). At baseline (first recorded EDSS visit), interferon beta was the most common treatment (41%), followed by glatiramer acetate (11%, table 1). Fingolimod and natalizumab represented 4.5% and 5%, respectively, while dimethyl fumarate (1.5%) and teriflunomide (1.3%) were less common. Thirty-five per cent of eligible patients were untreated at baseline. Patients on teriflunomide tended to be older than those untreated, on interferon beta or on natalizumab. MS duration at baseline tended to be marginally longer for fingolimod, natalizumab, glatiramer acetate and teriflunomide and shorter in untreated patients. Baseline disability was greater in those treated with natalizumab than in the other treatment categories. Patients on natalizumab and fingolimod have used more DMTs before becoming eligible for this study and were more likely to have experienced relapses on DMT in the pre-baseline year. More than 900 patients were still treated with their index study therapy at 5 years from its commencement, allowing sufficient power for a comparison of treatment effectiveness over the 5-year period (online supplementary figure 2). When compared with participants included in this study, those who were excluded tended to be older, with longer MS duration at baseline, higher levels of disability, fewer MRI lesions and were more likely untreated (online supplemental table 2).

Table 1

Unweighted characteristics of study participants at the first visit

Covariate balance

Weights from the ATE models significantly improved covariate balance between each treatment and the pooled sample of alternative treatment states at individual 6-monthly bins (online supplemental figure 3) as well as overall (figure 1). Sufficient balance was maintained throughout most of the follow-up while the treatment groups were sufficiently populated. Standardised differences of <0.20 are considered acceptable, resulting in ≥92% overlap between compared cohorts.7 The covariates that were challenging to balance and showed standardised mean differences marginally >0.20 were EDSS for the natalizumab versus other therapies and relapses in the prior 12 months for fingolimod versus other therapies (online supplemental figures 3 and 4). The models were adjusted for these residual imbalances.

Figure 1

Unweighted and weighted covariate balance quantified with standardised difference between each treatment and the pooled population of all alternative treatments. AUC, area under EDSS-time curve; EDSS, Expanded Disability Status Scale; MS, multiple sclerosis; RRMS, relapsing-remitting multiple sclerosis.

Simultaneous comparison of multiple therapies: ATE models

Weights in the ATE models are calculated for dummy indicators of individual therapies versus all other therapies, therefore ATE models estimate differences among the DMT effects in the total study population. In comparison with glatiramer acetate, natalizumab (−56%; HR=0.44, 95% CI=0.40 to 0.50) followed by fingolimod (−40%; HR=0.60, 95% CI=0.54 to 0.66) had a greater effect on reducing relapses than the other therapies. The relative effects of dimethyl fumarate (−22%; HR=0.78, 95% CI=0.66 to 0.92), teriflunomide (−11%, HR=0.89, 95% CI=0.75 to 1.06) and interferon beta (−5%; HR=0.95, 95% CI=0.89 to 1.00) were quantitatively smaller and no-treatment was detrimental for relapse incidence (+35%; HR=1.35, 95% CI=1.27 to 1.44). Natalizumab was associated with a lower cumulative hazard of confirmed disability worsening than the other therapies. The hazards of disability worsening relative to glatiramer acetate were: natalizumab (HR=0.43, 95% CI=0.32 to 0.56), fingolimod (HR=0.85, 95% CI=0.67 to 1.06), dimethyl fumarate (HR=0.86, 95% CI=0.51 to 1.47), teriflunomide (HR=0.56, 95% CI=0.31 to 0.99), interferon beta (HR=1.08, 95% CI=0.96 to 1.23) and no-treatment (HR=1.04, 95% CI=0.89 to 1.21). Relative to glatiramer acetate, natalizumab (HR=1.32, 95% CI=1.08 to 1.60) led to a higher probability of disability improvement compared with fingolimod (HR=1.18, 95% CI=0.96 to 1.46), dimethyl fumarate (HR=1.15, 95% CI=0.82 to 1.60), teriflunomide (HR=0.70, 95% CI=0.44 to 1.11), interferon beta (HR=1.03, 95% CI=0.91 to 1.18) and no treatment (HR=0.91, 95% CI=0.78 to 1.05) (figure 2).

Figure 2

Comparison of multiple therapies using ATE. Results from Cox proportional hazards models with the inverse probability of treatment weights to compare hazards of relapses, disability worsening and disability improvement among pseudo-cohorts treated with sufficiently represented therapies versus the reference therapy. ATE, average treatment effect.

We conducted sensitivity analyses that were restricted to patients with MRI data, where the number of MRI lesions was included in the weights. Results were similar to those observed in the main analysis. However, the superiority of natalizumab for disability improvement was attenuated (online supplemental figure 5). The three sets of comparisons using different reference therapies— glatiramer acetate, fingolimod and natalizumab, showed results consistent with the primary analysis.

Pairwise comparisons of therapies: ATT models

In the ATT approach, characteristics of patients using other treatments were first weighted to match those of patients using glatiramer acetate (reference therapy). ATT results are pairwise comparisons between therapies generalisable only to the subpopulation treated with the therapy of interest (active therapy). Results suggest that patients on glatiramer acetate would have experienced fewer relapses if they were instead treated with several other treatments (figure 3): natalizumab (−32%; HR=0.68, 95% CI=0.60 to 0.78), fingolimod (−38%; HR=0.62, 95% CI=0.55 to 0.70) or interferon beta (−7%; HR=0.93, 95% CI=0.87 to 0.98). Patients treated with glatiramer acetate would also have experienced fewer events of disability worsening if they were treated with natalizumab (−39%; HR=0.61, 95% CI=0.44 to 0.86) and a greater chance of disability improvement if they were treated with natalizumab (HR=1.64, 95% CI=1.32 to 2.05) or fingolimod (HR=1.29, 95% CI=1.02 to 1.63).

Figure 3

Comparisons of treatments effects (ATT). Results from Cox proportional hazards models with the inverse probability of treatment weights to compare hazards of relapses, disability worsening and disability improvement for pairwise comparisons of treatment groups. ATT, average treatment effects among the treated.

Second, the characteristics of patients on other treatments were weighted to match those who were treated with natalizumab (reference category). Results suggest that patients on natalizumab would have experienced higher risks of relapses if they had been treated with any other of the included therapies. They would have had a lower chance of disability improvement if they were treated with the other therapies except for dimethyl fumarate, and a higher risk of disability worsening if they were treated with interferon beta (figure 3).

Third, fingolimod was set as a reference category and characteristics of patients treated with other treatments were weighted to match those of patients on fingolimod. Results indicate that patients who were treated with fingolimod would have experienced more relapses on other treatments, except for natalizumab (for which there was no evidence of a difference in this patient group). The chances for disability improvement in patients treated with fingolimod would have been higher if they were treated with natalizumab (HR=1.39, 95% CI=1.07 to 1.80) and lower if they were treated with teriflunomide, glatiramer acetate or if they were untreated. The only therapy for which we have found an indication of a difference in the hazard of disability worsening was teriflunomide, in which the risk of worsening would have been lower if it was used in the group treated with fingolimod.

Discussion

We have emulated a quasi-randomised trial simultaneously comparing effectiveness among six commonly used DMTs for MS and untreated state. Among the selected therapies compared with the ATE modelling approach, natalizumab ahead of fingolimod is most effective in preventing relapses, disability worsening and enabling disability improvement in relapsing-remitting MS. ATE comparisons reflect the differences in mean treatment effects generalised to the entire study population defined by the inclusion criteria. The results of this study are generalisable to a population with a relatively active relapsing MS, as close to 20% of the studied patients experienced a relapse within 3 months prior to commencing a new therapy. The ATT provides us with a complementary perspective—of pairwise comparisons among each of the three selected reference therapies (glatiramer acetate, natalizumab and fingolimod) and the other included therapies. Therefore, the ATE resembles a multiarm clinical trial of multiple therapies, while ATT estimates pairwise differences in treatment effectiveness among patients who are typically treated with the reference therapy.9 10

The pairwise comparisons corroborated the overall observations, supporting the superiority of natalizumab and, in some instances, fingolimod over the other included therapies in controlling MS activity. More specifically, among patients treated with natalizumab, who, together with patients treated with fingolimod, tended to have experienced more severe disease, we identified not only a superior effect of natalizumab on relapses but also a tendency to reduce the frequency of disability-worsening events and even an increased chance of disability improvement in comparison to their hypothetical treatment with other therapies. Further, of the sufficiently represented therapies, the only therapy that further improved relapse and disability outcomes among patients who were treated with fingolimod was natalizumab. The ATT also suggests that between higher-efficacy therapies used in a specific clinical context (such as failure of previous therapies), the differences may be relatively smaller than when generalised to a broader population.

The available results from randomised controlled trials mostly evaluate the efficacy of therapies compared with placebo over 2–3 years. Our present study not only corroborates the results of trials but also extends the conclusions to multiple direct pairwise comparisons and multiarm comparisons of the studied interventions over 5 years. While one meta-analysis of randomised trials did not report a superiority of natalizumab over fingolimod in reducing relapses over 2 years,6 another meta-analysis found evidence for a superiority of natalizumab over fingolimod for relapse outcomes and disability improvement.21 In keeping with our findings, the risks of relapse in both above-cited meta-analyses were lower in natalizumab or fingolimod compared with interferon beta, glatiramer acetate, dimethyl fumarate and teriflunomide.6 21 Pairwise comparisons of relapse frequencies between glatiramer acetate, interferon beta and teriflunomide did not find evidence for differences.6 The risk of 3-month confirmed disability worsening tended to be lower in natalizumab compared with any other DMTs included in the meta-analyses, reaching the pre-defined threshold for statistical significance for glatiramer acetate and interferon beta-1b.

In addition to the data from randomised trials and their meta-analyses, data from MS registries and cohorts have been used for pairwise or three-way comparisons between groups treated with some of the DMTs, balanced with propensity score matching or weighting. A study in the registry of the Swiss Federation for Common Tasks of Health Insurances reported the comparative effectiveness of natalizumab versus fingolimod over a median follow-up of 1.8 years in patients who switched therapy from interferon beta or glatiramer acetate.22 Patients who switched DMT to natalizumab had lower risks of relapses and were more likely to experience disability improvement while no differences were found in the proportion of patients free from EDSS progression.22 We have previously reported similar results for natalizumab versus fingolimod using the MSBase data over a mean 12-month follow-up.23 These conclusions were later corroborated in a large analysis combining data from three large registries—OFSEP, MSBase and Danish MS Registry.24 25 Several other observational studies reported similar relapse incidence among patients treated with fingolimod, dimethyl fumarate or teriflunomide, but with a small relative advantage among patients switching from glatiramer acetate or interferon beta to fingolimod versus dimethyl fumarate or teriflunomide (absolute difference of approximately 0.1 relapse per year).26–28

The application of the novel methodological approach, combining marginal structural models for estimation of causal associations between time-varying exposures to multiple therapies with generalised boosted models builds on the use of different components of this methodology in previous studies.16 19 29 Similar to the individual components of this design, the current design is broadly applicable to data from various settings, ranging from cohorts, registries and databases to repurposed data from randomised controlled trials.

The main limitation of this study is the risk of potential unmeasured confounding due to missing data—especially MRI information. Reassuringly, sensitivity analyses within a subgroup with MRI data available support the primary ATE analysis. While this study allows generalisation of its observations to a population of patients with a relatively active MS, the restricted size of the subgroups with low disease activity or diagnosed with clinically isolated syndrome did not allow us to explore the effectiveness of therapies specifically in these clinical scenarios. A potential heterogeneity in the acquisition of relapses and disability information represents another limitation. We have mitigated this risk by prospectively defining relapse in the MSBase Study Protocol and by requiring Neurostatus certification at each site. Further, our models cannot account for the delayed effects of previous treatments in preventing relapses and disability. To mitigate this limitation, we have used cumulative weights that use information about treatment exposure during previous bins included in the analysis. Potential effects of regression to the mean were equalised across the compared treatments by periodically adjusting the cumulative IPW and IPCW for the number of relapses recorded during the preceding 6 months. Informed censoring leading to attrition bias may represent a limitation where retention of longitudinal data in the study is conditional on persistence of study therapy. This was accounted for by weighting on the inverse probability of censoring. It is encouraging that our weighting strategy resulted in good covariate balance among treatments suggesting that MSMs are suitable for simultaneous comparisons of several MS therapies in the context of their complex causal relationships.

Conclusions

This study for the first time simultaneously compares and quantifies the effectiveness of six commonly used MS therapies and no treatment. Its results suggest that among patients with active relapsing MS, natalizumab followed by fingolimod offers an advantage over dimethyl fumarate, teriflunomide, interferon β and glatiramer acetate in reducing the risk of relapses and improving disability outcomes. However, the individual use of these therapies should be also guided by specific clinical scenarios and their safety profiles, which were not evaluated in this study. The methodological development proposed in this report allows simultaneous comparison of the effectiveness of multiple treatments using observational data while accounting for time-dependent confounding and attrition bias. We highlight the conceptual differences between the ATE and ATT approaches to causal inference and demonstrate how complementary clinical questions are answered by utilising both strategies. This approach allows us to inform neurologists about both the overall relative effectiveness of the compared therapies as well as comparisons between selected treatment pairs in populations that are typically treated with the therapy of interest.

Data availability statement

Data are available upon reasonable request. MSBase is a data processor, and warehouses data from individual principal investigators who agree to share their datasets on a project-by-project basis. Data access to external parties can be granted at the sole discretion of each MSBase Principal Investigator (the data controllers), who will need to be approached individually for permission.

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants and was approved by Melbourne Health (2006.044). Participants gave informed consent to participate in the study before taking part.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Twitter @ALugaresi, @doctoredu

  • Contributors ID performed the analyses, interpreted the results, drafted and reviewed the manuscript. TK obtained funding, conceptualised the study, contributed data, designed the analyses, interpreted the results, edited and reviewed the manuscript and acts as the study guarantor. CBM, SS, IR, EAS contributed to the study design, interpreted the results and edited and reviewed the manuscript. DH, EKH, FP, VS, SO, SE, MO, AL, RAl, AP, PD, MT, CB, FGM, PS, DF, PG, BY, AA, OG, JL-S, RB, RK, GLu, CM, EC, SHu, MJS, CSo, LK, SHo, MS, FG, KdG, PAM, RAm, AvdW, HB, JLS-M, SV, GLa, YS, RG, TC-T, OG, EA-M, AA-A, CSh, TMA-H, TC, APS, ITF contributed data, interpreted the results and edited and reviewed the manuscript.

  • Funding This study was financially supported by National Health and Medical Research Council of Australia [1129189, 1140766, 1080518].

  • Competing interests The authors report the following relationships: speaker honoraria, advisory board or steering committee fees, research support and/or conference travel support from Acthelion (EKH, RA), Almirall (MT, FG, RB, CRT, JLS-M), Bayer (MT, AL, PS, RA, MT, CB, JL-S, EP, VVP, RB, DS, RA, JO, JLSM, SH, CR, AGK, TC, NS, BT, MS, CAS), BioCSL (TK, AGK, BT), Biogen (TK, TS, DH, EKH, MT, GI, AL, MG, PD, PG, VJ, AVW, FG, PS, DF, RA, RH, CB, JLS, EP, VVP, FG, RB, RA, CRT, JP, JO, MB, JLSM, SH, CR, CSh, OGerlach, AGK, TC, BS, NS, BT, MS, HB), Biologix (RA), BMS/Celgene (EKH, AL), Genpharm (RA), Genzyme-Sanofi (TK, EKH, MT, GI, AL, MG, PD, PG, AVW, FG, PS, DF, RA, RH, MT, CB, JLS, EP, EP, VVP, FG, RB, RB, DS, CRT, JP, JO, MB, JLSM, SH, O Gerlach, AGK, HB), GSK (RA), Innate Immunotherapeutics (AGK), Lundbeck (EP), Merck / EMD (TK, DH, EKH, MT, GI, AL(Merck Serono), MG, PD, PG, VJ, AVW, PS, DF, RA, RH, MT, CB, JLS, EP, VVP, FG, RB, DS, RA, JO, MB, JLSM, CR, FM, O Gerlach, AGK, TC, BS, MS, HB), Mitsubishi (FG),Novartis (TK, TS, DH, EKH, MT, GI, AL, MG, PD, PG, VJ, AVW, FG, PS, DF, RA, RH, MT, CB, JLS, EP, VVP, FG, RB, DS, RA, CRT, JP, JO, MB, JLSM, SH, CR, FM, CSh, OG, AGK, TC, NS, BT, MS, HB), ONO Pharmaceuticals (FG), Roche (TK, EKH, AL, MT, CB, VVP, BT), Teva (TK, DH, EKH, MT, GI, AL, MG, PD, PG, VJ, FG, PS, DF, RH, MT, CB, JLS, VVP, RB, DS, RA, JP, JO, JLSM, CR, AGK, TC, MS, CAS), WebMD (TK), UCB (EP).

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.