Objective: To explore the use and interpretation of self reported on/off diary data for assessment of daily motor fluctuations in Parkinson’s disease.
Methods: 26 consecutive non-demented patients with fluctuating Parkinson’s disease received standardised training on how to fill out the four category CAPSIT-PD on/off diary, followed by four hours of clinical observation and four weeks of daytime on/off diaries every 30 minutes at home.
Results: Overall patient–clinician agreement in diary entries was good (κ = 0.62; weighted κ = 0.84). Agreement for individual diary categories was good for “off” and “on with dyskinesias” (κ = ⩾0.72), but moderate for “partial off” and “on” (κ = 0.49). The overall validity of patient kept diaries was supported by expected symptom severity variability across diary categories, as assessed in the clinic. One day’s home diary data failed to predict outcomes from the full four weeks for all diary categories, and data from three days failed to yield good prediction (predefined as R2 = ⩾~0.7) for the time spent in “off” and “partial off”. Data from one week yielded good prediction (R2 = ⩾0.74) in all instances except “partial off”, which could not be well predicted even when two weeks’ home diary data were considered (R2 = 0.52).
Conclusions: The data provide support for the overall accuracy and validity of the four category CAPSIT-PD on/off diary, but suggest that a three category diary format may improve accuracy and validity. Interpretation of diary data beyond the assessed time frame should be made with caution unless diaries have been kept for sufficiently long periods.
- Parkinson’s disease
- motor fluctuations
- self report
- CAPSIT-PD, core assessment program for surgical interventional therapies in Parkinson’s disease
- CDRS, clinical dyskinesia rating scale
- UPDRS, unified Parkinson’s disease rating scale
Statistics from Altmetric.com
- CAPSIT-PD, core assessment program for surgical interventional therapies in Parkinson’s disease
- CDRS, clinical dyskinesia rating scale
- UPDRS, unified Parkinson’s disease rating scale
A major aim in the management of advanced Parkinson’s disease is to decrease daily motor fluctuations and the time spent in the “off” phase and in the “on” phase with dyskinesias. While the severity of Parkinson’s disease symptoms during “off” and dyskinesias during “on” is typically estimated by means of a clinical rating scales,1–3 this approach does not yield information about the number of daily fluctuations or the time spent in various conditions. The most commonly employed method to obtain this type of information is to instruct the patient to keep on/off diaries, differentiating between various motor conditions at regular intervals throughout the day. Such diaries are important tools in clinical research involving patients with motor complications, and as end points in clinical Parkinson’s disease trials.
In order to yield valid and interpretable data, patient-kept on/off diaries need to be accurate.4–6 Accuracy may be achieved through standardised video based patient training.7 However, there is little methodological consistency between studies using on/off diaries. For example, some investigators have used diaries that only differentiate between two motor states (“on” and “off”), whereas others use diaries containing three states (“off”, “on”, and “on with dyskinesias”) or more (various severity levels of “off” and “on with dyskinesias”).8–10 The length of time that patients have kept diaries also varies considerably between study protocols, ranging from a few days to virtually continuous data collection over several years.9,11 However, the extent to which diary data can be extrapolated beyond the time period during which they were collected is unknown.
CAPSIT-PD (core assessment program for surgical interventional therapies in Parkinson’s disease) provides recommendations for patient selection and assessment in clinical trials of neurosurgical procedures for Parkinson’s disease.5 According to these guidelines, patients should keep daytime on/off diaries differentiating between four motor conditions (“off”, “partial off”, “on”, and “on with dyskinesias”) every 30 minutes during one week each month. The use of a standardised educational programme7 is also suggested in order to improve the accuracy of diary entries.5 However, empirical evaluation of these guidelines appears to be lacking.
Our objective in this study was to explore the use and interpretation of on/off diary data collected according to current guidelines for neurosurgical trials in Parkinson’s disease.5 Specifically, we evaluated patent and clinician agreement over diary entries, the validity of self assessed motor fluctuations, and the predictive value of diary data beyond the time period assessed.
The study was approved by the research ethics committee, Faculty of Medicine, Lund University, Sweden. Twenty six consecutive patients (15 men and 11 women) were recruited according to criteria resembling those of neurosurgical trials. The inclusion criteria were thus that there should be clinically diagnosed idiopathic Parkinson’s disease12 complicated by daily episodes of “off” phases and “on” phases with dyskinesias while on stable anti-parkinsonian drug treatment, and that participants were capable of independently keeping an on/off diary. Exclusion criteria were the presence of significant cognitive impairment (a mini-mental state examination13 score of 24 or less), ongoing psychiatric drug reactions (hallucinations, confusion, psychosis), or other significant comorbidities.
Patients received standardised training regarding on/off fluctuations and how to fill out the CAPSIT-PD on/off diary. The training consisted of the suggested5 patient training video on motor fluctuations for on/off diaries in Parkinson’s disease by Goetz et al,7 accompanied by written information with definitions of the various motor conditions and instructions on how to complete the diary.
In the written and oral information, motor conditions were defined as: “off” (poor or no effect of antiparkinsonian drugs; difficulties in moving; slowness, stiffness, and/or tremor); “partial off” (some effect of antiparkinsonian drugs; condition in between “off” and “on”; symptoms as in “off”, but mild); “on” (good effect of antiparkinsonian drugs; able to move without disabling slowness or stiffness); “on with dyskinesias” (“on” phases complicated by involuntary irregular, twisting, and/or jerky movements).
During the training session, any queries from the patients were explained. Training typically required 20 to 25 minutes per patient. Thereafter, patients underwent a four hour clinical observation period (in the afternoon) while continuing to take their regular drug treatment. During the observation period, patients and a movement disorder specialised clinician (JR or PH) independently completed the on/off diary every 30 minutes by selecting the category that best described the patient’s overall motor condition. In connection with each diary entry, patients were also assessed according to the motor examination section (part III) of the unified Parkinson’s disease rating scale (UPDRS III),1 the two timed motor tests advocated by CAPSIT-PD (number of hand/arm movement between two points 30 cm apart during 20 seconds, and the time and number of steps required to walk 2×7 metres),5 and the clinical dyskinesia rating scale (CDRS).3
Prestudy evaluation of inter-rater agreement for clinical assessments were found to be ⩾0.90 (intraclass correlation coefficient) for both the UPDRS III and the CDRS. Inter-rater agreement for diary entries was 0.77 and 0.93 (κ and weighted κ statistics, respectively). Following the clinical observation period, patients were instructed to continue recording their motor condition every 30 minutes during the waking part of the day for the next four weeks, and to return the completed home diaries by mail. Daily time spent in the various motor conditions according to the home diaries was calculated as percentages of the waking part of the day. The number of daily fluctuations was recorded as the number of transitions between motor conditions.
Data were checked for assumptions underlying the use of parametric and non-parametric statistics and analysed accordingly.14 Overall agreement between patient and clinician on/off diary entries during clinical observation was assessed by the quadratic weighted κ statistic (κw). Absolute chance corrected proportional agreement for the whole diary, as well as for the individual motor conditions, was assessed by the κ statistic. Agreement was interpreted as poor (<0.20), slight (0.21 to 0.40), moderate (0.41 to 0.60), good (0.61 to 0.80), or very high (0.81 to 1.0).15 The validity of patient kept diaries was assessed by evaluating the variability in symptom severity across diary categories from the clinical observation period by Kruskal–Wallis analysis of variance with post hoc Mann–Whitney U tests (for differences between individual diary categories), corrected for multiple comparisons.16 Assessed symptom scores were overall Parkinson’s disease motor symptomatology (UPDRS III), resting tremor (item 20 of the UPDRS), rigidity (item 22 of the UPDRS), postural stability (item 30 of the UPDRS), upper limb bradykinesia (hand/arm movement between two points), gait (time to walk 2×7 metres), and dyskinesias (hyperkinesias and dystonia scores of the CDRS). Symptom severity scores were expected to decrease across diary categories from “off” to “partial off” to “on”/“on with dyskinesias”, except for hyperkinesias (which were expected to increase), postural stability (which is less responsive to dopaminergic treatment17) and dystonia (which occurs both during “off” and hyperkinetic “on” phase dyskinesias18).
The predictive value of diary data beyond the assessment period was evaluated by extracting four different time periods from the full four weeks of home diaries: the first day, the first three days, the first week, and the first two weeks. The mean percentages of time spent in the various motor conditions and the mean number of daily motor fluctuations during the respective time periods were then analysed by linear regression analyses using the data obtained from the full four weeks diaries as the dependent variable. Good prediction was predefined as an R2 value around or above 0.70. The α level of significance was set at 0.05. Probability (p) values are two tailed.
All 26 patients completed training and the subsequent four hour clinical observation period. One patient declined to participate in the home diary part of the study and three patients only completed home diaries for the first two weeks. Among the remaining 22 patients, home diary data were missing for one day in three instances. Patient characteristics are summarised in table 1.
Overall agreement between the independent patient and clinician diary entries during the four hour observation period was 0.84 (κw) and absolute chance corrected agreement was 0.62 (κ). Examination of the levels of absolute chance corrected agreement for the respective motor conditions revealed similar and good values for “off” and “on with dyskinesias” (κ = 0.78 and 0.72, respectively), whereas agreement was moderate for “partial off” and “on” (κ = 0.49 in both instances).
Variability in symptom severity by self assessed motor fluctuations is shown in table 2. Overall, score variability followed an expected pattern across patients’ self rated motor conditions according to the on/off diary. Most symptom scores were significantly different in “partial off” and “on” as compared with “off”, and in “on with dyskinesias” as compared with “off” and “partial off”, whereas only the total UPDRS motor score and tremor score differed between “on” and “partial off”.
Patient-kept home diaries during the four weeks following clinical observation showed that patients spent a mean (SD) of 16.1 (12.0)% of their waking time in “off”, 26.5 (10.3)% in “partial off”, 42.2 (17.8)% in “on”, 13.7 (14.3)% in “on with dyskinesias”, and experienced an average of 9.4 (5.1) daily fluctuations. Results from the regression analyses are shown in table 3, and example plots are provided in fig 1.
One day’s home diary data were insufficient to predict outcomes from the full four weeks in all instances (R2 = −0.011 to 0.115) and three consecutive days failed to yield good prediction for the time spent in “off” (R2 = 0.489) and “partial off” (R2 = 0.095). Data from one week yielded good prediction (R2 = ⩾0.736) in all instances except for the time spent in “partial off” (R2 = 0.301), which failed to yield good prediction even when two weeks’ home diary data were considered (R2 = 0.521). Similar observations were made when data from one day, three days, and one week were regressed to data from two weeks (data not shown). Again, data from one day had virtually no predictive value, and data from three days showed only modest to moderate predictive value for “off” and “partial off” (R2 = 0.549 and 0.269, respectively), whereas three day data from the other categories showed good prediction of the subsequent two weeks (R2 = 0.730 to 0.829). For prediction of one week’s motor fluctuations, three days’ self report yielded good prediction in all instances except “partial off”, which gave a borderline value (R2 = 0.662), whereas one day’s scoring predicted no more than 18% of the outcomes of one week (data not shown).
This study has provided new insights into the use and interpretation of on/off diaries in advanced Parkinson’s disease when used according to the CAPSIT-PD guidelines,5 and has implications for the design and conduct of clinical trials. Our findings provide overall support for the accuracy and validity of the CAPSIT-PD on/off diary, whereas interpretation of diary data beyond the assessed time frame should be made with caution unless diaries have been kept for sufficiently long periods.
Evidence that a brief standardised training session can yield good agreement between patients and clinicians in categorising parkinsonian motor conditions was obtained. For the full diary, we observed good to very high levels of absolute chance corrected and overall agreements, respectively. Agreement was also good for the categories “off” and “on with dyskinesias”, but only moderate for “partial off” and “on”. Goetz et al used a very similar patient training procedure, including the same instruction video, but a three category diary (“off”, “on”, and “on with dykinesias”).7 Although they assessed the percentage overall agreement—which typically yields overestimates as it fails to take agreement by chance into consideration14—it appears that they obtained a somewhat better agreement (93%) than that observed here with a four category diary. This may thus be in favour for a three category on/off diary over the four category one advocated by CAPSIT-PD. However, the patient instruction video7 suggested for use with the CAPSIT-PD on/off diary5 does not specifically illustrate the “partial off” condition. Although we attempted to circumvent this shortcoming by providing explicit written and verbal instructions, this may have contributed to the lower patient–clinician agreement observed for this category. The fact that agreement was also compromised for the “on” category may suggest particular difficulties in differentiating between “partial off” and “on”.
Nevertheless, recent data from the Parkinson study group22 indicate that good agreement can also be reached with larger numbers of categories (“off”, “partial off”, “on”, “mild dyskinesias”, and “severe dyskinesias”), although the amount of training needed to achieve this appears substantial. Whereas our patients received a brief 20 to 25 minute standardised training, patients participating in the study by the Parkinson study group were given extensive baseline instructions combined with a training and practice session where complete patient–investigator agreement was required for study entry.22 It thus seems reasonable to suggest that the choice of diary in a particular study setting should be made with these considerations in mind. Hence, if study objectives advocate careful monitoring of different levels of day to day motor fluctuations, an on/off diary such as that used by the Parkinson study group can yield valid data, assuming that patients and investigators are willing to invest in an extensive prestudy selection training session. In other situations, it appears more feasible to apply a diary differentiating between fewer categories, such as the three category diary (“off”, “on”, and “on with dyskinesias”) used by Goetz et al7 and recommended by Melamed et al.6 Another method which, depending on study objectives, could be considered is to use an activity based diary, which has recently shown promise,23 or one that differentiates between whether patients perceive dyskinesias as troublesome or not.24 The latter approach, however, has yet to be validated against clinician derived assessments.
Motor symptom severity scores showed an overall expected pattern across patient rated diary categories, thus providing general support for the validity of the CAPSIT-PD on/off diary in terms of external criteria. However, all scores did not show significant differences between all diary categories. The majority of unexpected findings involved the “partial off” category and may thus largely be explained by the observed problems with categorisation of this condition.
Our observations indicate that CAPSIT-PD on/off diaries should be kept for at least one week at a time in order to yield representative data that allow for valid conclusions beyond the assessed time period in patients with advanced fluctuating disease. This type of evaluation has not been presented before, and our observations provide overall support for the current recommendations that diaries should be kept for one week per month in clinical neurosurgical trials.5 Although shorter time periods (in this case, three days) provided good predictive values regarding the time spent in “on” and “on with dyskinesias”, as well as the number of daily fluctuations, prediction was only moderate for the time spent in “off”. The predictive value for the time spent in “partial off” was only moderate even when two weeks’ data were considered, again indicating some problems related to this category and possibly arguing for the use of a three category diary instead.
A further aspect of the collection of on/off diary data, not studied here, that may affect data validity is the frequency whereby patients are requested to record their motor condition. While most protocols, including CAPSIT-PD, appear to advocate that diaries should be kept every 30 minutes, other protocols have used alternative intervals—for example, once an hour.9 The frequency required to obtain representative and valid data is related to how often patients fluctuate, with a need for shorter intervals in those who experience more frequent motor oscillations. Post hoc analyses have provided some evidence that longer intervals (one hour) can yield both overestimations and underestimations of the time spent in various motor conditions, as compared with virtually continuous recordings.4 However, this aspect will need to be addressed in separate and specifically designed studies to allow firm recommendations. Until then, the commonly recommended5,6 30 minute interval appears a feasible compromise, at least for patients with advanced disease such as those enrolled in clinical trials of neurosurgical interventions.
Our analyses provide support for the overall accuracy and validity of on/off diary data when collected according to the CAPSIT-PD guidelines. However, we also found indications of some problems with the recommended four category diary. Together with previous findings in published reports, our observations thus suggest that the quality and reliability of on/off diary data in general probably benefit from reducing the number of categories from four to three (that is, “off”, “on”, “on with dyskinesias”), unless extensive patient training is provided. Furthermore, unless diaries have been kept for sufficiently long periods, the representativeness of diary derived data can be challenged, and interpretation beyond the assessed time frame should be made with caution. More, and comparative, studies on the use of various on/off diaries in Parkinson’s disease are needed to optimise their use and interpretation in clinical trials and practice. Until then, the current data provide useful guidelines for clinical trial design and interpretation.
This study was supported by the Swedish Research Council, the Skane County Council Research and Development Foundation, and the Söderberg Foundation.
We thank the patients for their cooperation, and Dr H Widner and Dr P Nyberg for valuable discussions.
Competing interests: none declared
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.