Efficacy or Effectiveness

Peter ForsterMajor Depression, Treatments of Depression

Efficacy versus EffectivenessWhat is the difference between efficacy and effectiveness?

We tend to use these terms interchangeably in our posts on this site, but the two are different and the difference can be important.

Efficacy refers to the power of a treatment to achieve a given outcome assuming that the patient is perfectly compliant and completes the treatment. Effectiveness addresses the more “real world” results of treatment selection. Here is a reasonably good summary of the difference from a longer article about clinical trials.

“Clinicians and policymakers often distinguish between the efficacy and the effectiveness of an intervention. Efficacy trials (explanatory trials) determine whether an intervention produces the expected result under ideal circumstances. Effectiveness trials (pragmatic trials) measure the degree of beneficial effect under “real world” clinical settings. Hence, hypotheses and study designs of an effectiveness trial are formulated based on conditions of routine clinical practice and on outcomes essential for clinical decisions.Efficacy and effectiveness exist on a continuum. Generalizability depends largely on the viewpoint of the observer and the condition under investigation.”

An illustration of the difference is contained in an article in the American Journal of Psychiatry which purports to find that not following patient preferences when choosing a treatment for depression does not change outcomes in patients who have not had previous treatment for depression. This is the summary of the study…

“Adults aged 18–65 with treatment-naive major depression were randomly assigned with equal likelihood to 12 weeks of treatment with escitalopram (10–20 mg/day), duloxetine (30–60 mg/day), or CBT (16 50-minute sessions). Prior to randomization, patients indicated whether they preferred medication or CBT or had no preference. The primary outcome was change in the 17-item Hamilton Depression Rating Scale (HAM-D), administered by raters blinded to treatment.

A total of 344 patients were randomly assigned, with a mean baseline HAM-D score of 19.8 (SD=3.8). The mean estimated overall decreases in HAM-D score did not significantly differ between treatments (CBT: 10.2, escitalopram: 11.1, duloxetine: 11.2). Last observation carried forward remission rates did not significantly differ between treatments (CBT: 41.9%, escitalopram: 46.7%, duloxetine: 54.7%). Patients matched to their preferred treatment were more likely to complete the trial but not more likely to achieve remission.

Treatment guidelines that recommend either an evidence-based psychotherapy or antidepressant medication for nonpsychotic major depression can be extended to treatment-naive patients. Treatment preferences among patients without prior treatment exposure do not significantly moderate symptomatic outcomes.”

When I saw the summary I was taken aback. Decades of information from the literature on treatment of depression strongly argues that patient preferences do predict whether or not a person responds to treatment. And, in part based on that, we make every effort to do so.

A careful read of the article indicates that the story may be more complicated.

The study reports efficacy data rather than effectiveness. In the graphs that look at matched versus mismatched treatment choice you can see that patients who had a moderate or strong preference for one treatment or another were much more likely to complete the treatment. Since completing treatment is a big factor determining long term outcome in patients with depression, this difference is important.

In fact, mismatched patients, those who preferred CBT and received medications, and vice versa, where 20% less likely to complete treatment.

The second graph looks at percent remission. These graphs analyze data looking at last observation carried forward analysis. Those who dropped out are included in the analysis.

So this would seem to be an effectiveness trial in the sense that it factors in the reduced likelihood that a patient will complete the trial based on patient matching.

Those who were matched for treatment completed an average of 9 medication treatment sessions, those who were not matched completed 8 sessions (of a possible 9). The difference was larger for those who preferred medications but received therapy. Those who were matched with their preference completed 11 sessions and those who were not matched completed 14 sessions (out of a possible 16). 

One aspect of effectiveness is the need for ongoing treatment. This study followed patients for 3 months. But almost would agree that, even for those who have only had a single episode of depression, treatment should continue for at least six months.

If those who were mismatched and dropped out of treatment are considered to be much more likely to have a return of symptoms over the following three months, which is an important aspect of the effectiveness of a treatment for depression, how does this affect the conclusion? There is no way of answering the question because that data is not available.


Effects of Patient Preferences on Outcomes in the Predictors of Remission in Depression to Individual and Combined Treatments (PReDICT) StudyBoadie W. Dunlop, Mary E. Kelley, Vivianne Aponte-Rivera, Tanja Mletzko-Crowe, Becky Kinkead, James C. Ritchie, Charles B. Nemeroff, W. Edward Craighead, Helen S. Mayberg, and for the PReDICT Team. American Journal of Psychiatry 0 0:0
Gartlehner G, Hansen RA, Nissman D, et al. Criteria for Distinguishing Effectiveness From Efficacy Trials in Systematic Reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006 Apr. (Technical Reviews, No. 12.) 1, Introduction. Available from: https://www.ncbi.nlm.nih.gov/books/NBK44024/