Background Per cent slowing of decline is frequently used as a metric of outcome in Alzheimer’s disease (AD) clinical trials, but it may be misleading. Our objective was to determine whether per cent slowing of decline or Cohen’s d is the more valid and informative measure of efficacy.
Methods Outcome measures of interest were per cent slowing of decline; Cohen’s d effect size and number-needed-to-treat (NNT). Data from a graphic were used to model the inter-relationships among Cohen’s d, placebo decline in raw score units and per cent slowing of decline with active treatment. NNTs were computed based on different magnitudes of d. Last, we tabulated recent AD anti-amyloid clinical trials that reported per cent slowing and for which we computed their respective d’s and NNTs.
Results We demonstrated that d and per cent slowing were potentially independent. While per cent slowing of decline was dependent on placebo decline and did not include variance in its computation, d was dependent on both group mean difference and pooled SD. We next showed that d was a critical determinant of NNT, such that NNT was uniformly smaller when d was larger. In recent AD associated trials including those focused on anti-amyloid biologics, d’s were below 0.23 and thus considered small, while per cent slowing was in the 22–29% range and NNTs ranged from 14 to 18.
Conclusions Standardised effect size is a more meaningful outcome than per cent slowing of decline because it determines group overlap, which can directly influence NNT computations, and yield information on the likelihood of minimum clinically important differences. In AD, greater use of effect sizes, NNTs, rather than relative per cent slowing, will improve the ability to interpret clinical trial results and evaluate the clinical meaningfulness of statistically significant results.
- ALZHEIMER'S DISEASE
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Contributors TG initiated the study and wrote the first draft of the manuscript. SL made the graphic, devised the statistical approach for determining Cohen’s d and made revisions in drafts. DPD and LSS substantively revised all drafts of the manuscript. TG acted as guarantor of this work.
Funding This work was funded by the following National Institute on Aging grants: P30AG066530, R01AG051346, R01AG052440, R01AG055422, R01AG062578 and R01AG062687.
Competing interests LSS reports personal fees from AC Immune, Alpha-cognition, Athira, Corium, Cortexyme, BioVie, Eli Lilly, GW Research, Lundbeck, Merck, Neurim, Novo-Nordisk, Otsuka, Roche/Genentech, Cognition Therapeutics, Takeda; grants from Biohaven, Biogen, Eisai, Eli Lilly and Novartis. DPD reports research support from the National Institute on Aging, Alzheimer’s Association, is a scientific adviser to Acadia, TauRx, Corium, Genentech, and is a member of the Data and Safety Monitoring Board of BioXcel. TG and SL have no conflicts of interest to disclose.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.