Efficacy vs. Utility - Treatments of Mental Disorders

Efficacy vs. Utility

RCT data are extremely useful for allowing us to identify certain therapies as “empiri-cally supported treatments.” But some authors are skeptical about the reliance on RCT data (Nathan et al., 2000; Westen & Bradley, 2005; Westen, Novotny, & Thompson-Brenner, 2004; Zimmerman, Mattia, & Posternack, 2002). Specifically, they argue that an RCT might inform us about a therapy’s efficacy (whether it works in carefully designed tests) but might not tell us about its clinical utility (whether it works in more typical circumstances; APA Presidential Task Force on Evidence-Based Practice, 2006).

We have already considered one of the bases for this concern—the reliance on manualized therapy. A different concern hinges on the patient groups used in many RCT studies. We discussed earlier the need for reasonably uniform patient groups in an RCT—people whose disorders are relatively uncomplicated by other problems. This uniformity allows us to ask whether our therapy works for people in a well-defined group, but it undermines external validity. Many of the people who seek out therapy do not have a clear diagnosis, and, if they do, the diagnosis may be complex. That is because many forms of mental disorder have high rates of comorbidity—they occur together with some other disorder. Thus, for example, there is considerable comorbid-ity between anxiety and depression disorders—someone who has one often suffers from the other as well. As a result, the RCT data may tell us what happens in therapy with “pure” cases, but not what happens in the (much more common) mixed cases encountered in most clinics.

For these and other reasons, we must combine RCT data with other sorts of data, including studies of ordinary clinical practice, case studies, studies of the actual process of change within psychotherapy, and more (APA Presidential Task Force on Evidence-Based Practice, 2006; Shadish et al., 1997). This broader package of evidence will help us under-stand whether therapy as it is usually practiced, with more typical patients, has the desired effects; it will also tell us whether the worries about manualized therapy are justified.