METHODS

The study was approved by the research ethics committee, Faculty of Medicine, Lund University, Sweden. Twenty six consecutive patients (15 men and 11 women) were recruited according to criteria resembling those of neurosurgical trials. The inclusion criteria were thus that there should be clinically diagnosed idiopathic Parkinson’s disease¹² complicated by daily episodes of “off” phases and “on” phases with dyskinesias while on stable anti-parkinsonian drug treatment, and that participants were capable of independently keeping an on/off diary. Exclusion criteria were the presence of significant cognitive impairment (a mini-mental state examination¹³ score of 24 or less), ongoing psychiatric drug reactions (hallucinations, confusion, psychosis), or other significant comorbidities.

Patients received standardised training regarding on/off fluctuations and how to fill out the CAPSIT-PD on/off diary. The training consisted of the suggested⁵ patient training video on motor fluctuations for on/off diaries in Parkinson’s disease by Goetz et al,⁷ accompanied by written information with definitions of the various motor conditions and instructions on how to complete the diary.

In the written and oral information, motor conditions were defined as: “off” (poor or no effect of antiparkinsonian drugs; difficulties in moving; slowness, stiffness, and/or tremor); “partial off” (some effect of antiparkinsonian drugs; condition in between “off” and “on”; symptoms as in “off”, but mild); “on” (good effect of antiparkinsonian drugs; able to move without disabling slowness or stiffness); “on with dyskinesias” (“on” phases complicated by involuntary irregular, twisting, and/or jerky movements).

During the training session, any queries from the patients were explained. Training typically required 20 to 25 minutes per patient. Thereafter, patients underwent a four hour clinical observation period (in the afternoon) while continuing to take their regular drug treatment. During the observation period, patients and a movement disorder specialised clinician (JR or PH) independently completed the on/off diary every 30 minutes by selecting the category that best described the patient’s overall motor condition. In connection with each diary entry, patients were also assessed according to the motor examination section (part III) of the unified Parkinson’s disease rating scale (UPDRS III),¹ the two timed motor tests advocated by CAPSIT-PD (number of hand/arm movement between two points 30 cm apart during 20 seconds, and the time and number of steps required to walk 2×7 metres),⁵ and the clinical dyskinesia rating scale (CDRS).³

Prestudy evaluation of inter-rater agreement for clinical assessments were found to be ⩾0.90 (intraclass correlation coefficient) for both the UPDRS III and the CDRS. Inter-rater agreement for diary entries was 0.77 and 0.93 (κ and weighted κ statistics, respectively). Following the clinical observation period, patients were instructed to continue recording their motor condition every 30 minutes during the waking part of the day for the next four weeks, and to return the completed home diaries by mail. Daily time spent in the various motor conditions according to the home diaries was calculated as percentages of the waking part of the day. The number of daily fluctuations was recorded as the number of transitions between motor conditions.

Data were checked for assumptions underlying the use of parametric and non-parametric statistics and analysed accordingly.¹⁴ Overall agreement between patient and clinician on/off diary entries during clinical observation was assessed by the quadratic weighted κ statistic (κ_w). Absolute chance corrected proportional agreement for the whole diary, as well as for the individual motor conditions, was assessed by the κ statistic. Agreement was interpreted as poor (<0.20), slight (0.21 to 0.40), moderate (0.41 to 0.60), good (0.61 to 0.80), or very high (0.81 to 1.0).¹⁵ The validity of patient kept diaries was assessed by evaluating the variability in symptom severity across diary categories from the clinical observation period by Kruskal–Wallis analysis of variance with post hoc Mann–Whitney U tests (for differences between individual diary categories), corrected for multiple comparisons.¹⁶ Assessed symptom scores were overall Parkinson’s disease motor symptomatology (UPDRS III), resting tremor (item 20 of the UPDRS), rigidity (item 22 of the UPDRS), postural stability (item 30 of the UPDRS), upper limb bradykinesia (hand/arm movement between two points), gait (time to walk 2×7 metres), and dyskinesias (hyperkinesias and dystonia scores of the CDRS). Symptom severity scores were expected to decrease across diary categories from “off” to “partial off” to “on”/“on with dyskinesias”, except for hyperkinesias (which were expected to increase), postural stability (which is less responsive to dopaminergic treatment¹⁷) and dystonia (which occurs both during “off” and hyperkinetic “on” phase dyskinesias¹⁸).

The predictive value of diary data beyond the assessment period was evaluated by extracting four different time periods from the full four weeks of home diaries: the first day, the first three days, the first week, and the first two weeks. The mean percentages of time spent in the various motor conditions and the mean number of daily motor fluctuations during the respective time periods were then analysed by linear regression analyses using the data obtained from the full four weeks diaries as the dependent variable. Good prediction was predefined as an R² value around or above 0.70. The α level of significance was set at 0.05. Probability (p) values are two tailed.

RESULTS

All 26 patients completed training and the subsequent four hour clinical observation period. One patient declined to participate in the home diary part of the study and three patients only completed home diaries for the first two weeks. Among the remaining 22 patients, home diary data were missing for one day in three instances. Patient characteristics are summarised in table 1.

Overall agreement between the independent patient and clinician diary entries during the four hour observation period was 0.84 (κ_w) and absolute chance corrected agreement was 0.62 (κ). Examination of the levels of absolute chance corrected agreement for the respective motor conditions revealed similar and good values for “off” and “on with dyskinesias” (κ = 0.78 and 0.72, respectively), whereas agreement was moderate for “partial off” and “on” (κ = 0.49 in both instances).

Variability in symptom severity by self assessed motor fluctuations is shown in table 2. Overall, score variability followed an expected pattern across patients’ self rated motor conditions according to the on/off diary. Most symptom scores were significantly different in “partial off” and “on” as compared with “off”, and in “on with dyskinesias” as compared with “off” and “partial off”, whereas only the total UPDRS motor score and tremor score differed between “on” and “partial off”.

Patient-kept home diaries during the four weeks following clinical observation showed that patients spent a mean (SD) of 16.1 (12.0)% of their waking time in “off”, 26.5 (10.3)% in “partial off”, 42.2 (17.8)% in “on”, 13.7 (14.3)% in “on with dyskinesias”, and experienced an average of 9.4 (5.1) daily fluctuations. Results from the regression analyses are shown in table 3, and example plots are provided in fig 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Scatterplots of the daily mean per cent time spent in the “off” phase according to home diaries from four weeks (y axis) plotted against data from diaries kept for (A) one day, (B) three days, (C) one week, and (D) two weeks (x axis).

One day’s home diary data were insufficient to predict outcomes from the full four weeks in all instances (R² = −0.011 to 0.115) and three consecutive days failed to yield good prediction for the time spent in “off” (R² = 0.489) and “partial off” (R² = 0.095). Data from one week yielded good prediction (R² = ⩾0.736) in all instances except for the time spent in “partial off” (R² = 0.301), which failed to yield good prediction even when two weeks’ home diary data were considered (R² = 0.521). Similar observations were made when data from one day, three days, and one week were regressed to data from two weeks (data not shown). Again, data from one day had virtually no predictive value, and data from three days showed only modest to moderate predictive value for “off” and “partial off” (R² = 0.549 and 0.269, respectively), whereas three day data from the other categories showed good prediction of the subsequent two weeks (R² = 0.730 to 0.829). For prediction of one week’s motor fluctuations, three days’ self report yielded good prediction in all instances except “partial off”, which gave a borderline value (R² = 0.662), whereas one day’s scoring predicted no more than 18% of the outcomes of one week (data not shown).

DISCUSSION

This study has provided new insights into the use and interpretation of on/off diaries in advanced Parkinson’s disease when used according to the CAPSIT-PD guidelines,⁵ and has implications for the design and conduct of clinical trials. Our findings provide overall support for the accuracy and validity of the CAPSIT-PD on/off diary, whereas interpretation of diary data beyond the assessed time frame should be made with caution unless diaries have been kept for sufficiently long periods.

Evidence that a brief standardised training session can yield good agreement between patients and clinicians in categorising parkinsonian motor conditions was obtained. For the full diary, we observed good to very high levels of absolute chance corrected and overall agreements, respectively. Agreement was also good for the categories “off” and “on with dyskinesias”, but only moderate for “partial off” and “on”. Goetz et al used a very similar patient training procedure, including the same instruction video, but a three category diary (“off”, “on”, and “on with dykinesias”).⁷ Although they assessed the percentage overall agreement—which typically yields overestimates as it fails to take agreement by chance into consideration¹⁴—it appears that they obtained a somewhat better agreement (93%) than that observed here with a four category diary. This may thus be in favour for a three category on/off diary over the four category one advocated by CAPSIT-PD. However, the patient instruction video⁷ suggested for use with the CAPSIT-PD on/off diary⁵ does not specifically illustrate the “partial off” condition. Although we attempted to circumvent this shortcoming by providing explicit written and verbal instructions, this may have contributed to the lower patient–clinician agreement observed for this category. The fact that agreement was also compromised for the “on” category may suggest particular difficulties in differentiating between “partial off” and “on”.

Nevertheless, recent data from the Parkinson study group²² indicate that good agreement can also be reached with larger numbers of categories (“off”, “partial off”, “on”, “mild dyskinesias”, and “severe dyskinesias”), although the amount of training needed to achieve this appears substantial. Whereas our patients received a brief 20 to 25 minute standardised training, patients participating in the study by the Parkinson study group were given extensive baseline instructions combined with a training and practice session where complete patient–investigator agreement was required for study entry.²² It thus seems reasonable to suggest that the choice of diary in a particular study setting should be made with these considerations in mind. Hence, if study objectives advocate careful monitoring of different levels of day to day motor fluctuations, an on/off diary such as that used by the Parkinson study group can yield valid data, assuming that patients and investigators are willing to invest in an extensive prestudy selection training session. In other situations, it appears more feasible to apply a diary differentiating between fewer categories, such as the three category diary (“off”, “on”, and “on with dyskinesias”) used by Goetz et al⁷ and recommended by Melamed et al.⁶ Another method which, depending on study objectives, could be considered is to use an activity based diary, which has recently shown promise,²³ or one that differentiates between whether patients perceive dyskinesias as troublesome or not.²⁴ The latter approach, however, has yet to be validated against clinician derived assessments.

Motor symptom severity scores showed an overall expected pattern across patient rated diary categories, thus providing general support for the validity of the CAPSIT-PD on/off diary in terms of external criteria. However, all scores did not show significant differences between all diary categories. The majority of unexpected findings involved the “partial off” category and may thus largely be explained by the observed problems with categorisation of this condition.

Our observations indicate that CAPSIT-PD on/off diaries should be kept for at least one week at a time in order to yield representative data that allow for valid conclusions beyond the assessed time period in patients with advanced fluctuating disease. This type of evaluation has not been presented before, and our observations provide overall support for the current recommendations that diaries should be kept for one week per month in clinical neurosurgical trials.⁵ Although shorter time periods (in this case, three days) provided good predictive values regarding the time spent in “on” and “on with dyskinesias”, as well as the number of daily fluctuations, prediction was only moderate for the time spent in “off”. The predictive value for the time spent in “partial off” was only moderate even when two weeks’ data were considered, again indicating some problems related to this category and possibly arguing for the use of a three category diary instead.

A further aspect of the collection of on/off diary data, not studied here, that may affect data validity is the frequency whereby patients are requested to record their motor condition. While most protocols, including CAPSIT-PD, appear to advocate that diaries should be kept every 30 minutes, other protocols have used alternative intervals—for example, once an hour.⁹ The frequency required to obtain representative and valid data is related to how often patients fluctuate, with a need for shorter intervals in those who experience more frequent motor oscillations. Post hoc analyses have provided some evidence that longer intervals (one hour) can yield both overestimations and underestimations of the time spent in various motor conditions, as compared with virtually continuous recordings.⁴ However, this aspect will need to be addressed in separate and specifically designed studies to allow firm recommendations. Until then, the commonly recommended^5,6 30 minute interval appears a feasible compromise, at least for patients with advanced disease such as those enrolled in clinical trials of neurosurgical interventions.

Conclusions

Our analyses provide support for the overall accuracy and validity of on/off diary data when collected according to the CAPSIT-PD guidelines. However, we also found indications of some problems with the recommended four category diary. Together with previous findings in published reports, our observations thus suggest that the quality and reliability of on/off diary data in general probably benefit from reducing the number of categories from four to three (that is, “off”, “on”, “on with dyskinesias”), unless extensive patient training is provided. Furthermore, unless diaries have been kept for sufficiently long periods, the representativeness of diary derived data can be challenged, and interpretation beyond the assessed time frame should be made with caution. More, and comparative, studies on the use of various on/off diaries in Parkinson’s disease are needed to optimise their use and interpretation in clinical trials and practice. Until then, the current data provide useful guidelines for clinical trial design and interpretation.