Background, Aim and Method A version of John Rankin's scale has been an outcome measure in many stroke studies. The original version was modified, and limited inter-rater agreement has led to scoring algorithms and post hoc score adjustments. These recommendations have improved scoring consistency. Nevertheless, only limited scale evaluation is achievable using traditional psychometric methods; in contrast, modern psychometric methods (e.g., Rasch analysis) can help by enabling a test of whether the six Rankin categories are ordered as intended (0–1–2–3–4–5). We applied this test to modified Rankin scale data from two phase 3 lubeluzole trials (n=515; n=591).

Results In combined data, and both samples, modified Rankin scale categories were disordered (not working as intended). Moreover, Rankin scale categories remained disordered even when scores were adjusted as suggested by others. Indeed, we have not found a published study where data analysis shows Rankin categories work as intended. This disordering affects the dichotomisation of Rankin data.

Conclusions Despite the appeal of apparent clinical simplicity, and approaches to improving scoring, the grading of poststroke sequelae mapped out by Rankin scales does not work. The most likely reason—ambiguity within category definitions. This finding threatens the validity of all studies using Rankin scales and provides one plausible explanation for some negative stroke treatment studies. Has the time come to throw in the towel?