METHODS

Patients and carers

The Atlantic Canada Alzheimer’s Disease Investigation of Expectations (ACADIE) study targeted community dwelling Alzheimer’s patients judged to have dementia of mild to moderate severity according to standard criteria.^17,18 Eligible patients, together with their primary carers, were recruited at six study centres across Atlantic Canada. Calculations based on a summary of donepezil efficacy results on the Alzheimer’s Disease Assessment Scale-Cognitive (ADAS-cog),¹⁹ with a standard table,²⁰ and assuming α=0.05 in a two tailed test were used to determine that a sample size of 90 would allow 80% power to detect an effect size of 0.40. Allowing for 10% attrition, recruitment of 100 patients was planned.

Design and instruments

Participants were recruited to participate in a 52 week open label, prospective study of treatment with donepezil hydrochloride. At baseline, after obtaining informed consent, diagnostic verification and standard psychometric testing, patients were given 5 mg donepezil tablets, to be taken once daily, for the first 12 weeks of the study. Follow up assessments were conducted at 12, 24, 36, and 52 weeks. If clinically indicated, the donepezil dose was increased to a daily maximum of 10 mg at (or subsequent to) the 12 week follow up visit. At each time point, patients and carers underwent a battery of standard clinical assessments and participated in an open-ended home interview conducted by trained field researchers (masters or doctoral level social scientists). All home visits were audio recorded. The findings of the clinicians were blinded from the findings of the field researchers, and vice versa.

GAS ^{14,16,21–27} was the primary outcome measure. Like the CIBIC-plus, GAS is a clinimetric measure (that is, one that is based on clinical judgments about individual patients) designed to capture individual treatment effects. Briefly, GAS uses a five point scale, which is individualised according to patients’ needs. The scale is anchored at a baseline value, which represents the pretreatment status for individual patient problems. The scale extremities describe probable better and worse outcomes.

The GAS process begins with the selection of individualised problem areas, typically elicited by a phrase such as “Please tell me what it is like to have Alzheimer’s disease. How is it a problem for you?” Each problem needing treatment is then described in observable terms and recorded on the GAS follow up guide at the 0 level (the baseline status). Next, anticipated outcome levels are set. Expectations for possible improvements are designated on the GAS follow up guide as +1, an outcome that is a little better than baseline, and +2, an outcome that is a lot better than baseline. Similarly, potential declines are specified. A little worse than baseline is recorded as −1 and a lot worse than baseline is recorded as −2. As patients are seen subsequently, their follow up status is reviewed and scored. Patients’ goal attainment (that is, their score on each goal) is determined by the extent to which their status at follow up conforms to one of the predetermined levels (where 0=no change).

In this study, we set two groups of goals. The treating physicians and nurses set “clinician goals” during the baseline office visits and recorded them using their own words, based upon patient examinations, clinical observations, and information provided by the patient and carer informants. Field researchers conducted in depth home interviews with patients and carers within one week of the baseline clinic visit. During home visits, patients and their carers discussed their experiences of AD, its effect on their daily lives, and their expectations for treatment. Field researchers assisted patients and carers in the identification of treatment goals. This resulted in the development of separate “patient/carer goals” recorded by the field researcher using the patients’ and carers’ own words. In each case, patients, carers, and physicians could choose to set as many or as few goals as they wished. The clinical team met after each follow up visit to review and score the goals set at baseline. Field researchers saw patients and their carers at home within one week of each clinic visit to review and score the GAS follow up guides.

Based on our earlier experience,¹⁴ we specified that inquiries be made, but not limited to, four general domains: cognition, function, behaviour, and leisure. Within domains, patients/carers and physicians could choose to set as many goals or as few (including none) as they wished (that is, it was not a requirement of the study that patients have a goal set in each domain). Individualised goals were categorised into domains at the study centre, where social interaction emerged as a fifth domain.

For example, a typical goal such as memory fell into the domain of cognition. A description of a memory problem might be “Dad asks me the same question over and over—more than five times a day on most days. He has trouble shopping now, even with a list because he has problems remembering whether he has picked up the items and must constantly recheck the cart. He often doesn’t remember the names of the grandchildren when they come over.” Using this as the baseline level of performance (scored as “0” on the GAS follow up guide, table 1), descriptions of better and worse outcomes were then elicited from the patient and/or carer. A worse outcome might be that the patient repeats questions more than 10 times per day and/or that he cannot remember to ask for the list and can no longer use it with prompting; a still worse outcome would be that he would no longer recognise his daughter, who is his major carer. Better outcomes would be that he asked questions less frequently, that he is able to use a list, and no longer needs to check the cart. From this inventory, five states can be demonstrated: the present state, and states somewhat and much better/worse, respectively.

Patient/carer goals identified at baseline were also weighted on a visual analogue scale in order of relative importance. This step is optional; clinician goals were not weighted (all goals were set with weight=1). Note that goal attainment is scored by the terms specified at the outset. A global score, summarising attainment across a set of goals, is calculated as 50+{[10ε(w_ix_i)]/[0.7εw_i²+0.3(εw_i)²]^1/2}, where x_i=the individual attainment level, and w_i=the weight of that level. As each of the attainment levels is 0, so is their product, and thus the baseline score is 50. This is a mathematically equivalent modification of the conventional method,²⁸ which sets the baseline status at “−1,” and has the “0” level represent the treatment goal. This modification incorporates more levels of decline, which seemed preferable, given that deterioration is likely.

In addition to GAS, other standardised measures were used. Cognition was assessed using the Mini-Mental State Examination (MMSE) ²⁹ and the Alzheimer’s Disease Assessment Scale-Cognitive (ADAS-cog).^19,30 To measure various aspects of physical function we used the Physical Self-Maintenance Scale (PSMS),³¹ Instrumental Activities of Daily Living (IADL)³¹ and the Functional Activities Questionnaire (FAQ).^32,33 Depression in patients was measured using the Cornell Depression Scale (CDS)³⁴ and the Centre for Epidemiologic Studies Depression Scale (CES-D)^35,36 was used for carers. The Clinician’s Interview-Based Impression of Change-Plus caregiver input (CIBIC-plus)³⁷ was administered by the treating physician. The CIBIC-plus is a seven point Likert scale of change from baseline.

Analyses

The analyses of treatment effects used the intention to treat last observation carried forward (ITT-LOCF) principle. Mean change and 95% confidence intervals from baseline were calculated at each follow up visit for all outcome measures. A one way repeated measures analysis of variance was performed on the change scores as a function of time for the primary outcome measures (patient/carer GAS and clinician GAS) with the Bonferonni adjustment for multiple comparisons. Otherwise the mean change values and their 95% confidence intervals are presented. To assess construct validity, Spearman correlation coefficients were calculated for GAS change scores and compared with the secondary efficacy measures; small to moderate correlations were expected. Statistical tests were interpreted at the 5% significance level.

All subjects provided written informed consent. This project was approved by the Research Ethics Committee of the Queen Elizabeth II Health Sciences Centre, Halifax, Nova Scotia, and by comparable committees of the other investigating sites.

RESULTS

Enrolment

Of 108 patients enrolled, 88 (81%) successfully completed 52 weeks of treatment with donepezil. Premature discontinuation occurred because of: adverse events (n=10), withdrawal of consent (n=5), insufficient clinical response (n=2), death (n=1), and other (n=2). Of the 108 patients enrolled, eight had no observation after first treatment, leaving 100 for analysis.

Most patients were elderly women with mild AD (table 2). Eighty two patients had treatment increased from 5 mg/day to 10 mg/day for at least one dose. At week 12, 69% of patients were receiving 10 mg/day; 67% at week 24; 66% at week 36; and 65% at week 52. Most carers were women (66%), spouses (48%), and younger (mean age=61 years) than the patient set (table 2).

Goals and their attainment

Patients and carers identified a total of 855 goals at baseline (mean (SD)=9 (3) per patient), while clinicians identified 342 (mean=3 (1)). Patients/carers set function goals most often (86% of patients), followed by cognition (83%), leisure (76%), behaviour (58%), and social interaction (49%). Clinicians set more cognition goals (85%), followed by function (68%), behaviour (57%), social interaction (24%), and leisure (20%).

Statistically significant improvements in the global patient/carer GAS scores (fig 1) were seen to week 36 (mean change=3.19, p=0.03); however, by week 52 there was no significant difference from baseline (mean change=1.62, p=0.74). The global GAS score for clinician identified goals improved significantly from baseline to week 24 (mean change=2.39, p=0.04), but was not significantly different from baseline thereafter (week 52 mean change=0.43, p=1.00).

Comparisons with standard measures

While improvement in global GAS scores was observed as noted, the standard cognitive measures (figs 1 and 2) showed significant improvements only at week 12 (MMSE mean change=0.86 and ADAS-Cog mean change=−1.17). Decline from baseline was observed for both measures thereafter (for example, week 52: mean change=−1.04 and mean change =3.07, respectively). The functional assessments showed patterns of initial maintenance of functional performance, followed by later decline (fig 2), with slight differences by level of function (chiefly IADL decline after weeks 24–36, whereas ADL function was more stable). Neither patient (CSD) or carer (CES-D) measures of depression demonstrated much change (fig 2).

Clinician global GAS scores were highly correlated (table 3) with the CIBIC-plus at weeks 12 and 52 but not with the ADAS-cog results. Patient/carer global GAS scores were only moderately correlated with the CIBIC-plus. Low correlations were observed between patient/carer GAS and the ADAS-cog.

Low correlations were observed between the GAS domains of cognition, function, and behaviour and corresponding secondary efficacy measures at week 12, for both clinicians and patient/carers. Moderate correlations appeared between clinician assessed GAS cognition goals and the MMSE (r=0.51) and the ADAS-cog (r=−0.43) at week 52. Moderate correlations were also observed between clinician function goals and the PSMS (r=−0.53), patient/carer function goals and the PSMS (r=−0.47), and patient/carer function goals and IADL scores (r=−0.44). All remaining correlation comparisons at week 52 were low.

Patterns of goal attainment, by domain

Within goal areas (fig 3), patients and carers generally observed improvement, and each domain tended to agree with the others, and thus with the global GAS score. The largest and greatest treatment effects were observed in behaviour.

The pattern of treatment effects observed by physicians was somewhat more complex. They recorded behaviour and social interaction goals above baseline at each time point (fig 4) but in contrast with the patient/carer goals, goal attainment did not proceed in the same direction for each domain. Clinician identified leisure goals changed little, on average. Similar to the standard cognitive measures, clinician identified cognition and function goals had been met at week 12 but these initial gains were not maintained.

Adverse events

Of the 108 patients, 96% (n=104) experienced at least one adverse event. The most frequently reported were: pain (n=27), diarrhoea (n=24), nausea (n=23), dizziness (n=20), headache (n=19), dyspepsia (n=15), and influenza-like symptoms (n=15). Ten patients withdrew because of adverse events, which included: anxiety (n=2), weight loss (n=2), diarrhoea (n=1), pacing (n=1), transient ischaemic attack (n=1), confusion (n=1), agitated depression (n=1), and foot pain (n=1). Twenty two serious adverse events were reported in 16 patients, one of who died of a myocardial infarction. No serious adverse events occurred more than once and none had a clear relation to the study drug.

DISCUSSION

In this 12 month, open label study, treatment with donepezil resulted in clinically detectable effects in patients with mild to moderate AD, which largely met the goals of patients, carers, and clinicians for six months. Thereafter, treatment accorded with expectations to varying degrees. The extent of goal attainment was not reflected in standard measures used to evaluate antidementia drugs.

Patients and carers set more goals than physicians, perhaps a product of the more open-ended nature of the field researcher interviews. Physicians seemed to lump items that patients and carers tended to split. For example, whereas patients and carers might describe goals with respect to repetitive questions, needing lists and having to have help in following routines, physicians typically would group all of these under a single goal of memory. In addition, patients and carers set more goals in the domains of leisure, social interaction, behaviour, and function, and fewer in cognition. This probably reflects competing understandings of the disease. Physicians see patients episodically and they seem to think of pathophysiological processes and manifestations that they can test, or about which they are used to inquiring.³⁸ These differences, however, do suggest that physicians might wish to explore these other areas more systematically if they are not to miss important treatment effects. At the same time, physicians are able to offer some objectivity. They can help patients and carers sort out patterns in their observations, and not be unduly influenced by a single memorable event or by incidents that are not representative but whose saliency is raised by having happened recently. This objectivity, however, must be to some extent traded off with physicians’ more episodic interactions, and their differing priorities in understanding the disease and its treatment effects. Through qualitative analyses we plan to explore these differences more systematically.

It seems, however, that as each party brings information to bear, physicians can provide a useful role by collaborating with patients and carers in understanding disease treatment. Additional qualitative analyses will allow us to comment more fully on these differences. We plan to investigate further what we see as another useful finding of this study: attainment of treatment goals does not accord with a simplistic model in which successful treatment is interpreted as meaning that all declines are recovered. For example, the analysis of the clinician goal attainment scores by domain shows that treatment, which overall rated a clinical impression of improvement, nevertheless encompassed areas in which no clinically important change was seen, and even others in which decline occurred. This accords with earlier observations that suggest that judgement about treatment effects trade off between domains.^11,14 Understanding such trade offs, providing a role for patient and carer involvement, and clarifying treatment expectations will be necessary if the recommendation that a global clinical measure be used in routine monitoring of cholinesterase inhibitor use in AD is taken up.¹⁰

Whether such trade offs account for the at best modest correlations between GAS scores and the scores on standard tests is not clear. In this study, with the exception of the CIBIC-plus, most correlations between other measures and GAS were in the low to moderate range, although they tended to increase over time. By week 52, the clinician GAS correlation with other measures ranged from 0.40–0.54, and the patient/carer GAS correlated between 0.21–0.51 with standard measures. How to interpret this range is not clear. On the one hand, no correlation might mean that the measure either is not valid, or is tapping entirely different information. On the other hand, high correlation might mean that the new measure adds little value to existing information. In earlier studies, we have seen similar correlations, which we have interpreted as supporting validity, but adding new information.^25,26 Still, none of the usual measures represents a so called “gold standard”, and their clinical importance is suspect. In consequence, we see that replication offers the best guide to more completely understanding the validity of this approach.

Our study is subject to important limitations. It is not a controlled trial. Our question was not whether donepezil is more effective than placebo over 3, 6, 9, or 12 months, but rather whether treatment of AD meets a priori expectations. Our understanding is that, up to six months, the question of whether cholinesterase inhibitors provide statistically significant change in cognitive measures under double blind conditions has been answered ^{5–7,39–41} and new data seem to extend this time frame to about 12 months.^42,43 We are seeking to help translate whether the treatment effects observed in earlier studies have meaning for treating physicians, and for patients and carers. In this context, the patients enrolled in the study seem comparable to those in the treatment arms of the double blinded studies. As in an earlier clinical practice study ⁸ they were slightly older (76 (8) years) than patients from three pivotal double blinded trials (combined mean age =73 (5) years). Our study had more women (71% versus 60% overall).^39–41 As in other studies, most of our patients (75%) had mild AD (that is, 80% overall). Considering the ADAS-cog, we found that the mean difference in scores on individually optimal doses (that is, either 5 mg or 10 mg of donepezil) was −1.17 at 12 weeks and 0.72 at 24 weeks. These estimates are comparable with other published reports of −0.67 points (24 week/end point analysis; 5 mg) and −1.06 (24 week/end point analysis; 10 mg) and about 0.35 points (24 week/end point analysis 5 mg) and about −1.5 (24 week/end point analysis 10 mg) and about −0.2 (12 weeks/end point analysis 5 mg) and about −0.3 (12 week/end point analysis 10 mg).

Patients and their carers bring a wide range of expectations in relation to AD and treatment effects. Still, patient and carer involvement in goal setting seems both feasible and valid. For example, there were only a small number of goals (about 1%) where donepezil was unlikely, a priori, to have had a treatment effect. These included goals set for mobility, incontinence, pain, and headaches.

The scoring instructions for GAS were specified so that a problem area was identified and the current level of functioning (that is, the present problem) was described at the “0” level. In 106 patient/carer goals (12%), however, while future problems were anticipated, the present level of function was satisfactory. In such cases, goals identified by patient/carer dyads who felt that maintenance was the best that could be hoped for were scored as “0”. Where no improvement could be anticipated, there is no room to capture improvement. Similarly, in the 97 goals (11%) where patient/carer dyads were unwilling to specify a worse outcome, the baseline was scored as “0” and only unspecified worsening could be captured. Overall, these effects are likely to be largely offsetting, but they do reveal the tension in trying to incorporate patient and carer preferences in the face of a neurodegenerative illness where deterioration is a likely outcome. This can influence not only the appreciation of the present state, but can sometimes reveal situations in which people, perhaps as an aspect of coping, choose not to specify how things might get worse. The neurodegenerative nature of AD also means that the ITT-LOCF strategy might be less cognitive if patients who dropped out early had observations carried forward that did not reflect subsequent decline. We therefore also conducted an observed cases analysis, which did not change any of the main effects here observed.

Our data also have some strengths. We systematically incorporated the expectations of patients and carers, and their treating physicians into an evaluation of treatment. As noted, despite the broad range of expectations that patients and carers bring to the prospect of treatment for AD, they largely set goals that reflect a realistic understanding of what might be possible. Importantly, maintenance often is explicitly understood as the best outcome of treatment. Nevertheless, many of the individual goals hope for a reversal to a previous state, suggesting that the expectations are complex. Maintenance is often desirable, but when people are confronted with describing what might be better, they are likely to specify “the way things were.” The previous state strongly influences the expectation of likely future outcomes.

These data are relevant to understanding the CIBIC-plus. Like it, GAS relies on clinical judgments about individual patients with caregiver input. Recent attempts to standardise the CIBIC-plus, however, have seen increasing standardisation of the domains of inquiry. By contrast, GAS incorporates patient and carer preferences as the basis for evaluation of treatment effects. For example, the test of recent memory would not be whichever standard test a given clinician uses, which is commonly how such inquiries are operationalised using the CIBIC-plus.¹¹ Rather, the inquiry might be about repetitive questioning, recalling visits, or not having to constantly recheck a grocery list, if that is how the memory problem is seen by patients and carers. Like the CIBIC-plus, GAS probably works best when used in conjunction with, and not as a substitute for, standard measures.

To date, almost all antidementia drug trials have used standard scales and staging systems exclusively. While these allow for important insights, they do not provide an accurate portrayal of the expectations, motivations, and experiences of the patients, their primary carers, and families. In addition, these standard instruments largely assume that as AD gets better the result is chiefly a reversal of the decline in signs and symptoms that mark the disease as it progresses. This might not be the case—what seems more likely is that patients exhibit new combinations of symptoms and signs, as some that were present at baseline improve, while others worsen and still others stay the same.^11,44 In consequence, we run the risk that new antidementia drugs could bring about systematic and reproducible benefits, which might be missed because our current scales are not sensitive to the new profiles experienced by patients. Our study suggests that important treatment effects might not be reflected in the standard tests now in use as outcome measures in antidementia drug trials and treatments. Judging by its range of correlations with standard measures, GAS seems to be tapping into different constructs than the usual domains of inquiry now used to evaluate treatment effectiveness. At the same time, the extent of correlation is in the range that suggests convergent construct validity of the measures. Validity is likewise suggested by the higher patient/carer-clinician GAS scores. In consequence, we suggest that GAS can supplement the CIBIC-plus (but not replace it) so as to explicitly incorporate patient preferences into our understanding of whether treatment is beneficial.