Material and methods

PATIENTS

Our datasets consisted of the following groups of patients with MS with monthly MRI of the brain for a period of at least 6 months.

Group A

This group comprised 66 patients with RRMS not selected for the presence of MRI activity at entry. There were 49 women and 17 men, with a mean age of 35.1 (SD 9.0) years, a mean duration of the disease of 6.0 (SD 5.1) years, and a mean baseline expanded disability status scale (EDSS) score23 of 3.2 (SD 2.6).

Group B

This group consisted of 120 patients with RRMS selected for having at least one enhancing lesion at entry. There were 87 women and 33 men, with a mean age of 34.0 (SD 7.5) years, a mean duration of disease of 4.9 (SD 3.8) years, and a mean baseline EDSS of 2.4 (SD1.2).

Group C

This group comprised 81 patients with SPMS not selected for the presence of MRI activity at entry. There were 40 women and 41 men, with a mean age of 40.1 (SD 8.1) years, a mean duration of the disease of 13.4 (SD 7.3) years, and a mean baseline EDSS of 5.3 (SD1.1).

MRI

Serial dual echo conventional or fast spin echo (TR=2000–3500; TE=20–50/60–100) and postcontrast T1 weighted (TR=400–700, TE=5–25) images of the brain were acquired every month for at least six consecutive occasions. T1 weighted scans were acquired five to 10 minutes after the injection of 0.1 mmol/kg of gadolinium-DTPA. Slices were always axial and contiguous with slice thickness of 3 or 5 mm. The pixel size was about 1×1 mm for all the scans. For each patient, the same MRI acquisition methodology was kept constant throughout the entire study duration. Scanners were not changed or upgraded over the study duration. New enhancing lesions were counted by a group of experienced observers, who always analyzed the entire set of scans from each patient.

STATISTICAL METHODOLOGY

The outcome variable of this study was the number of new enhancing lesions seen on three or six consecutive MR scans. Treatment effect was always expressed as a percentage difference between “treated” and “untreated” patients. We included in the analysis only patients with a complete 6 month follow up or with, at maximum, one missing observation (they were 66 from group A, 115 from group B, and 78 from group C). Missing observations were managed by assuming that the new enhancing lesions possibly present on the missing scans were all seen on the next subsequent scans (the total number of new enhancing lesions was obtained by summing up lesions seen during then-1 months and the observation period wasn months). The mean numbers of new enhancing lesions seen over the follow up period in the three groups of patients were compared using the Kruskal-Wallis and the Wilcoxon rank sum tests. The sample sizes needed for MR monitored treatment trials in MS were calculated for each of the three patient groups for a parallel groups (PG) design, for a parallel group with a single baseline correction scan (PGB) design, and for a baseline versus treatment (BVT) design (BVT design is based on the acquisition from the same patients of two sets of scans, one before and one after the introduction of the treatment to be tested and, as a consequence, each patient serves as his or her own control), using a parametric simulation method based on the NB distribution. A brief description of the statistical properties of the NB model, the estimated NB parameters obtained for the present study, and a description of the simulation algorithm used are all reported in the and in another paper.6

Results

In table 1, the mean numbers of new enhancing lesions in each of the three patient groups are reported for 3 and 6 monthly MRI. The three patient groups had significantly different numbers of new enhancing lesions (p=0.01). At post hoc analysis, the difference of the mean numbers of new enhancing lesions was not statistically different between unselected patients with RRMS or SPMS, whereas both of these groups had fewer new enhancing lesions than those with RRMS selected for MRI activity at entry (p=0.04 vunselected patients with RRMS and p=0.001 vpatients with SPMS) for both durations of follow up.

The results of the power simulations are displayed in fig 1 (for a six scan follow up only), where data are fitted with exponential curves for different magnitudes of the treatment effect to allow sample size extrapolations for different experimental conditions. The numbers of patients in each treatment arm necessary to obtain statistical powers of 80% or 90% are presented separately for the three patient groups in tables 2-4. Eighty per cent and 90% are the standard levels of statistical powers usually required in MS clinical trials. Treatment effects ranging from 50% to 80% are reported in tables 2 and 3 for PG and PGB trials, and treatment effects ranging from 20% to 50% are reported in table 4 for BVT trials. These different ranges of treatment effects were decided a priori on the basis of what conventional wisdom considers as clinically relevant and sensible expectations for the different clinical trial designs.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Power estimates for parallel group (PG) (first column), parallel group with a baseline correction scan (PGB) (second column) and baseline versus treatment (BVT) (third column) design clinical trials of unselected patients with RRMS (group A, first row), patients with RRMS and MRI activity at study entry (group B, second row) and unselected patients with SPMS (Group C, third row) followed with 6 monthly enhanced MRI. The number of new enhancing lesions is used as the measure of outcome. Raw data are fitted with exponential curves for different magnitudes of the treatment effect to allow extrapolations for different trial designs.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Numbers of observed new enhancing lesions (black squares connected by lines) and numbers of those expected under the NB model assumptions (bars) for patients with RRMS (A) or SPMS (C) not selected for the presence of MRI activity at entry, and for patients with RRMS selected for having at least one enhancing lesion at entry (B).

As expected, for trials with patients with RRMS selected for having MRI activity at entry smaller sample sizes were required than for those with unselected RRMS and SPMS for all the tested study designs. For the PG design, the sample sizes needed for treatment trials of patients with RRMS selected for MRI activity at entry were about 70% (range 63%-78%) of those needed for similar trials of unselected patients with RRMS, and about 50% (range 46%-64%) of those needed for similar trials of unselected patients with SPMS (table 2). The difference between the clinical subgroups was smaller for PGB and BVT trials: for both these designs, the sample sizes needed for treatment trials of patients with RRMS and MRI activity at entry were on average about 75% (range 66%-84%) of those needed for similar trials of unselected patients with RRMS and about 65% (range 60%-77%) of those needed for similar trials of unselected patients with SPMS (tables 3 and 4).

The baseline correction (PGB), as previously reported,5 8 9 reduced the sample size of PG design trials, both for patients with RRMS and those with SPMS; however the gain in power decreased as the magnitude of the expected treatment effect increased. For instance, for a treatment effect of 50%, a PGB design required about 50% fewer patients than a similar PG trial; for a treatment effect of 60%, 40% fewer patients were required; and for a treatment effect of 70%, only 20% less patients were needed. In case a of an 80% treatment effect, the baseline correction resulted in virtually no gain (tables 2 and 3).

As expected, BVT trials required far fewer patients than PG or PGB trials (tables 2, 3 and 4). The BVT trials were sensitive to duration of follow up (table 4). For instance, to achieve an 80% power with a treatment effect of 20%, 80 or 48 patients with RRMS not selected for baseline activity, 60 or 35 patients with RRMS selected for activity at entry, and 80 or 60 patients with SPMS were needed if 3 or 6 monthly periods are employed.

Discussion

At present, virtually all clinical trials in MS use measures derived from enhanced MRI as primary or secondary outcomes to monitor treatment efficacy.10 11 14-20 As a consequence, a great deal of work has been spent in the past few years to achieve reliable estimates of the sample sizes and durations of follow up needed to perform enhanced MRI monitored clinical trials in MS with adequate statistical powers in different experimental conditions.5-9 Overestimations and underestimations of the numbers of patients and MRI needed to reject the null hypothesis with adequate powers are undesirable. On the one hand, patient and scan overestimations would result in unnecessarily high trial costs and patient inconvenience. On the other, patient and scan underestimations would lead to equivocal or even false negative trial results. In the past, when large serial MRI datasets were not available, power calculations were based on data from small samples of patients with MS,5-9 and, as a consequence, they were relatively unreliable9 and potentially inaccurate.6 More recently, the situation has changed for two reasons. Firstly, large serial MRI data sets coming from the placebo arms of several MS clinical trials are available.14-21 Secondly, a new modelling of MS lesion counts, which results in more accurate power estimates than the conventional approach, has been developed to study patients with MS.6

Against this novel background, we performed the present study to provide more reliable and accurate estimates of the sample sizes needed to perform MRI monitored clinical trials with adequate statistical power in patients with different MS phenotypes and for different trial designs, when the number of new enhancing lesions is the measure of outcome. The tables and fig 1 of the present paper should provide a valuable reference when planning future clinical trials in MS, as they are based on large MRI datasets and on a reliable parametric simulation procedure.6 Such an approach might become even more valuable in the near future, as it is likely that new experimental treatments will be tested against already approved treatments. In this scenario, the sample sizes needed to detect treatment effects should increase dramatically and adequate sample estimations would reduce the risks of running uninformative or extremely expensive trials.

This study also confirms that including MRI activity at study entry as a patient selection criterion allows a substantial reduction of patients needed to run a PG design trial. The presence of MRI activity at a given time point is associated with a higher likelihood of further MRI activity during the next few months.24 Thus, by reducing the number of uninformative (inactive) scans, smaller sample sizes and shorter durations of follow up will be required to achieve enough study power. Nevertheless, such an approach has a hidden cost, as it is likely that a significant number of clinically eligible patients would be screened, but on showing a negative enhanced scan, would not enter the trial. The major drawback of such an approach is, however, the increased risk of selecting patients during periods of relatively high disease activity which will inevitably decline during the period of the trial regardless of the treatment used. This effect is known as “regression to the mean” and its impact is proportional to the amount of activity required for the patients' eligibility and might be misleading if the design of the study lacks a control arm. In addition, the use of selection criteria makes the results of a trial less generalisable to all the potential patients that might benefit from the treatment.

In PG trials, the use of additional MRI for baseline correction, which reduces the between patient variability of new enhancing lesions, also results in smaller sample sizes than those needed for PG trials conducted without such an approach. This agrees with findings of previous studies5 7 8 and with recent work demonstrating that the addition of a second and sometimes of a third baseline scan results in a significantly reduced sample variance (A Smith, J Petkau, personal communication). Although the use of additional baseline scans may not always be ethical or feasible, as there might be concerns about withholding a treatment from patients for a certain period of time or about the additional costs, the addition of a single baseline scan 1 month earlier is unlikely to be problematic. Indeed, cost considerations are mitigated, as shown in the present study, by the fact that the acquisition of a baseline pretreatment scan substantially reduces the number of scans needed during the course of the study.

Although the algorithm used in the present study to perform power calculations was shown to provide a better fitting of the raw data than the previously used methodology,6 it is not without potential problems. The main assumption behind our calculations is that a given experimental treatment would be able to modify the number of new enhancing MS lesions, but not their distribution, which should always be fitted by the NB model. This assumption can only be verified by analyzing the data of the treated arms of clinical trials and further work is warranted to clarify this issue. Another caveat of any of the possible approaches to perform power calculations is that the sizes of the samples studied and the durations of the follow up periods should not only be determined by statistical considerations, but should also be based on the characteristic of the experimental treatment used. Finally, the well known limitation in the correlation of enhancing lesion activity with clinical measures, such as relapse rate and disability,24 is still widely regarded as justifying the use of the latter as the primary end points in definitive phase III clinical trials.