Delay in Nursing Home Placement with Donepezil

To the Editor: The claim of Geldmacher et al.1 of risk reductions of 40% to 50% in nursing home placement (NHP) from long-term use of donepezil could, if true, dramatically change the natural history of Alzheimer's disease (AD). Unfortunately, the information they provide to support this claim is weak, incomplete, and problem plagued. The essential question is whether drug use delays NHP or whether natural destiny or ability to stay out of a nursing home creates the opportunity to continue to use the drug. We present several concerns relating to important bias in the design and analysis of the study that seriously question the validity of the results and conclusions. In part, this is based on having read (1) the study protocol E-0403 that was submitted to institutional review boards and (2) Delay in Nursing Home Placement of Alzheimer's Disease Patients Treated in the Aricept Clinical Program Final Report, by Provenzano et al. from the Battelle Centers for Public Health Research and Evaluation, February 5, 2001, the Pfizer- and Eisai-commissioned report on which Geldmacher's publication is based. Pfizer sent this report—which the Journal's readers and reviewers probably had not seen—to the Wall Street Journal, which provided it to the first author of this letter and others for preparation of a news story. The first major problem arises in the comparability of the groups constructed for analysis. The authors' large effects were attained, in great part, by defining extreme reference groups consisting of patients randomized to donepezil or placebo in the three phase II/III trials who mainly did not complete them or, if they did, did not continue to receive open-label medication after the trial. Therefore, the authors created a reference group of patients who are mainly intolerant to medication or who could not comply with the protocols. The comparison groups had much-longer donepezil use at doses of mainly 10 mg with 80% compliance or greater, and were observed over long periods. Thus, they created comparison groups of patients who are excellent at adhering to taking their medication, tolerate medication well, do not drop out, and believe they are experiencing symptomatic improvement. These extreme comparisons, defined mainly by survivability, seriously bias analyses to assess survival and contain imbalances that cannot be easily rectified or rendered inconsequential by adjusting for covariates in the usual manner with Cox proportional hazards models or by the kind of sensitivity analyses conducted by the authors. A table of covariates was created from the unpublished Battelle report to illustrate the immense differences between the groups, because except for cholinesterase inhibitor use, the distribution of these covariates was not included in the publication. Table 1 shows the extreme lack of comparability of the groups in the primary model in the publication, called the C/24 treatment construct (Model 1 in the report), defined in the Provenzano et al. report as “completed an acute study and 24 weeks in an open label study on five or more mg/d for 80% of the period.” In addition, there were eight other models of donepezil usage analyzed. The relative risks (more correctly, hazard ratios), but not the covariates, for this treatment exposure model C/24, as well as for two other models, C/48 and C/96, were listed in Table 2.1 Remarkable imbalances are obvious across the authors' donepezil-use strata; there were nearly 2.5 times more men, nearly 1.5 times as many cholinesterase inhibitor users, and one-third as many spouse-caregivers in the minimal-use reference group than in the maximal-use group and a trend for the reference group to be older. These differences should have been presented to readers before presenting the Cox models summary hazard ratios. The immense imbalances in the known variables suggest that many other unknown variables are also likely to have differed between the groups, and covariate adjustments cannot correct for these unknown variables! A further problem is the lack of a detailed analysis plan, resulting, in reality, in data dredging. The protocol E-0403 document (p. 3) states: The direction of the statistical analysis will be driven by the preliminary statistical results that emerge as the analysis proceeds in an iterative fashion. … The kinds of modification that are anticipated include: (1) modifying the subsets of the patients included in or excluded from the analysis; (2) modifying variable definitions, for example, how the Aricept dose is defined; and (3) using different ways to treat lost-to-follow-up, nonresponse, noninformative and informative censored data, and the definition of the baseline. This extends to the outcomes used (p. 4). The analysis process proposed for this study has been designed to accommodate several alternative but agreed definitions for key variables. Each alternative variable definition is a potential candidate for analysis. Recommended or “preferred” definitions will be established through discussion and agreement among the Battelle researchers, the Pfizer project manager, and other researchers from Pfizer and Eisai, Inc. The selection of a recommended definition will be pragmatic and will take into account the availability and quality of relevant data. This degree of flexibility allows the data and its interpretation at many stages, rather than prespecified primary and secondary analyses in the protocol, to drive the analysis, as with trials submitted to regulatory authorities. Compounding this are the many analyses conducted (e.g., logistic regression analyses were also conducted but not mentioned in the publication, as well as the specifying of nine other models that were each analyzed), so that P-values should be adjusted for the number of analyses performed, whether reported or not. Hence, at the best, these results should be interpreted as hypothesis generating. If the authors have confidence in their results, then they should conduct a randomized trial with physicians, patients, and caregivers who remain uncertain of the benefits. A recently completed placebo-controlled trial of donepezil in the United Kingdom (AD 2000), which, over the course of 4 years, showed no difference in NHP or functional level between donepezil and placebo, although showing a small effect for Mini-Mental State Examination (MMSE) scores (Bentham J, Gray A. Poster at the World Alzheimer's Congress, Stockholm, 2002), has rendered obsolete arguments that such a trial is not feasible. This result is irreconcilable with the benefits found in the Geldmacher et al. analyses. There are also discrepancies between other presentations and this publication (e.g., a poster presented at the American Geriatrics Society annual meeting in 2001 with different authors).2 The minimal utilization sample size was 113 in the publication but 95 for least exposure in the abstract. In the abstract, the risks of dementia-related NH placement is 0.453 and in the publication 0.574. Interpreting these discrepancies is difficult in the absence of a prespecified analysis plan. The length of time since the individual trials were performed and completed affected the number of subjects available for analysis, the earliest trial providing less robust evidence with only 52.2% of subjects followed, compared with the latest with 74.6% followed. The influence of this factor should have been assessed using the individual trials as covariates. Assessment up to spring 2000, long after completion of the last trials in early 1997, may well have resulted in the memory of completers being better and providing more favorable outcomes than noncompleters and nonresponders. These patients also were healthier than the average AD trial patient. The minimal-use reference group survived in the community 40.4 to 50.5 months using any of three definitions of NHP, although deaths occurring before potential NHP were censored. Yet other work suggests that expected median survival until NHP is about 28 months for AD patients, aged about 74, with MMSE scores of about 19 to 20, and 75% would be expected to be institutionalized in about 47 months.3 Another problem is that the known status of patients at the end of the study represented just 671 of 1,115 (60%) of those who were randomized to the double-blind trials. A 40% loss of the potential patient population represents such a large proportion that a convincing sensitivity analysis incorporating this level of loss would almost certainly alter the effect to such an extent that little confidence would be placed in the findings. This further illustrates the difficulty in comparing the constructed groups. A further problem lies in the ways patients were censored. The critical issue in survival analysis is that the censoring process is independent of the outcome of interest. There were 95 deaths (14%) out of the available cohort of 671 before a record of NHP. Sixty-two percent of these deaths were due to dementia, 24% from nondementia causes, and 14% of unknown cause. Hence, a conservative analysis should be reworked with all the deaths included with NHPs. This will not only provide an important sensitivity analysis, but also provide the clinician the necessary relevant information to balance the important risks and benefits and incorporate any hidden risks attached to therapy, should they exist. The anomaly of age not being a statistically significant risk factor for death due to dementia lends weight to evaluating all deaths. We estimate that this sensitivity analysis will render the result not significant, because the analysis on death due to dementia was not statistically significant. Simple Kaplan-Meier curves are sorely missing from this complicated Cox survival analysis. For example, if 12 months of treatment delays NHP by 18 months, will 3 years of treatment delay placement by 4.5 years? What would be a fairer comparison and analysis? To provide some ways forward before considering whether a randomized trial is necessary, we propose that the authors rework the analysis in a manner that provides fairer comparability between groups. Patients are assigned to groups only from those who have adequately completed the double-blind phase of the trial. Furthermore, the follow-up and exposure should begin from the end of the double-blind period and not take into account the (relatively short) double-blind period. A further refinement would be the use of a dose and time index to calculate a continuous variable to characterize donepezil and other cholinesterase inhibitor exposure and relate this to outcomes of NHP and death. Alternatively, an intention-to-treat analysis could be conducted in which the allocation of patients at randomization is maintained. This very conservative analysis has the great advantage of avoiding the thorny problem of lack of comparability of groups. The lack of comparability of the groups may be better made through use of propensity scores, as well as conventional covariate adjustment. However, this type of analysis requires much more information than appears in the protocol. Finally, if the effect were to be truly as large as the authors claim, it would represent a huge disease modification effect for donepezil that would defy plausible biological mechanisms and would probably be wrong given AD 2000 results! Donepezil and other cholinesterase inhibitors are efficacious over at least 6 months to a year, but this seriously flawed analysis does not provide a basis for the claim that long-term donepezil “may help patients live longer in community settings.” Such a statement is misleading.