The quest for a meaningful evidence base in psychiatry

While the rest of the medical profession moved swiftly and confidently into the era of evidence-based medicine, psychiatry was initially reluctant to follow, and slow to warm to its principles. However, more and more psychiatrists are now enthusiastically embracing an evidence-based approach and demanding “the evidence” for all therapeutic interventions. The trouble with this approach is that the evidence is often inconclusive, inconsistent and even contradictory, giving rise to the danger that those with specific interests can select the evidence to suit their needs. Studies failing to show advantages for newer agents are pounced upon by funding bodies and used to argue for the return to (cheaper) first generation antipsychotics (FGAs). At the same time these studies are disregarded by proponents of the newer agents, who point out the methodological flaws that are inherent in all clinical trials. Because much psychiatric symptomatology remains subjective, accumulating evidence based on objective measures is that much more difficult. Despite the encouraging progress in our ability to treat effectively most psychiatric disorders, there remain major shortcomings in clinical practice, and “real world” treatment outcomes are frequently unsatisfactory. Part of the problem lies in determining best practice based on the available evidence. The array of “evidence” being published each month in scientific journals can be bewildering. The article by Fleischhacker and Goodwin provides a timely and insightful discussion of some of the difficulties that psychiatrists experience when attempting to translate research findings into best practice. Thus, the randomized controlled trial (RCT) — the cornerstone of evidence-based medicine — is under siege in psychiatry, and has been criticized for, amongst other things, not accurately reflecting “real world” conditions 1. The high and increasing placebo response and dropout rates associated with randomized controlled trials have become the statisticians’ recurring nightmare, casting serious doubt on the validity of trial results. In an attempt to counter the shortcomings of RCTs, so-called “pragmatic” trials are appearing in the literature more frequently. However, these studies, with alluring acronyms that seem to promise much, such as CATIE 2, CUtLASS 3, CAFE 4, EUFEST 5 and STAR*D 6, are threatening to confuse the picture even more. They are proving just as difficult to interpret and are creating considerable controversy. It seems that, with each new study conducted, an additional batch of unanswered questions is generated. Fleischhacker and Goodwin argue for the retention of both RCTs and pragmatic trials, the latter at an earlier stage of drug development, before fixed opinions have been formed. By combining the advantages of the scientifically rigorous RCTs with those of the closer-to-real-world-practice experience of pragmatic trials, we can hopefully come closer to establishing which treatments are best for our patients. While this clearly makes sense, it alone might not be enough, as the difficulty is not just in obtaining evidence but also in interpreting the findings. Part of the problem may be that we have too many expectations from each individual trial — these trials are usually designed to address one or two questions — yet we often attempt to extrapolate the findings to other issues and other populations of patients. For example, the CATIE study found that the first generation antipsychotic (FGA) perphenazine performed surprisingly well against the second generation antipsychotics. However, it is potentially dangerous to generalize this finding to other FGAs such as haloperidol. No individual clinical trial, be it a randomized controlled or a pragmatic one, is designed to answer all the questions or to provide a basis for definitive treatment protocols — each study adds a little to the knowledge base. This means that the “evidence” on which practice is based will comprise a large pool of sometimes inconsistent knowledge. Clinicians need to be able to integrate justifiable conclusions from each new piece of knowledge into their daily practice, and accurate interpretation and translation will require a long-term cumulative approach, not a rash and sometimes opportunistic exclusive focus on each new piece of emerging data. However, we would argue that approaching the status quo with caution is not enough; that we need to do more to remove bias. Publication bias needs to be urgently addressed. The consequences of selective publication of positive results have become painfully clear in the controversy surrounding the use of antidepressants and other related drugs in adolescents 7. The various recently established clinical trial registers should go some way to preventing a recurrence of this situation, in which internationally accepted and implemented treatment guidelines appear to have been unknowingly based on incomplete data. The influence of the pharmaceutical and medical device industries, accused of impugning the integrity of medical science, needs to be carefully regulated 8–9. The quality of clinical data from RCTs may be improved by minimizing recruitment incentives (especially those for rapid recruitment), ensuring unimpeachable methodology, and using appropriate outcome measures. The rigorous training of investigators may improve the accuracy of clinical data collected. The usefulness of pragmatic trials may be increased by a broader selection of clinical contexts, including patient populations in developing countries and other low income settings. Perhaps most importantly, clinicians will need to maintain a non-dogmatic approach, a thorough knowledge of all of the evidence and sound clinical judgement, for which there is still no substitute.