Researchers have not been able to methodologically eliminate self-selection bias or utilize adequate controls in their studies of 12-step groups and Twelve-Step Facilitation.
When I read Facing Addiction in America: The Surgeon General’s Report on Alcohol, Drugs, and Health, I was surprised to see Twelve-Step Facilitation (TSF) included as an evidence-based behavioral treatment for addiction. I had just done a literature review on the efficacy of 12-step-based interventions, and found the evidence insufficient to support the prescription of 12-step groups as treatment. TSF is a standardized form of therapy where professional counselors try to engage their patients in participating actively in 12-step groups, in part by emphasizing 12-step philosophy during therapy sessions.
Twelve-step philosophy stipulates that addiction is a spiritual disease born of defects of character, and that 12-step groups are the only cure, involving faith in a higher power, prayer, confession, and admission of powerlessness. In contrast, the National Institute on Drug Abuse (NIDA) defines addiction as a disease of the brain – a medical condition requiring medical treatment. A spiritual disease concept is not the same as a medical disease concept. Twelve-Step Facilitation treats addiction as a spiritual and biopsychosocial disease, retaining the spiritual emphasis of 12-step philosophy.
TSF was classified as a professional behavioral treatment in the Surgeon General’s Report. How can a professional, medical treatment be based on a definition of addiction as a spiritual disease? Baffled, I knew I would not be able to understand if I got stuck in bias against Twelve-Step Facilitation. I had studied the research on 12-step groups, but had only dipped my toe into the research on TSF. The Surgeon General’s Report cites hundreds of studies, and over a dozen in support of TSF. So, I did what all good scientists must do: I set aside my bias, knowing that if I want truth, I must assume first that I am wrong and dig deeper.
I conducted a preliminary literature review to investigate the effectiveness of TSF as a treatment, and then examined each of the sources the Surgeon General’s Report cited in support of TSF. I looked at the methodology, results, and conclusions for each. In this article, I define “evidence-based” to mean any treatment supported by numerous scientific experiments with rigorous methods that include control groups, randomization of patients to treatments, and bias-free samples. I use “12-step approaches” to refer to all 12-step-based rehab programs, TSF, and 12-step mutual help groups.
The key to understanding research on TSF is to know why the treatment was created in the first place. Researchers had documented a correlation between 12-step group attendance and abstinence, but correlation is not causation and research had been limited in several ways:
- Studies evaluating the effectiveness of 12-step groups could not eliminate self-selection bias, which happens when group members are not randomly selected and participants opt in or select themselves, creating biased samples. The people participating in the studies had chosen to participate, and researchers could not determine whether successes observed were due to 12-step participation or qualities in the self-selected participants, such as greater motivation to enter recovery, more resources, or greater receptivity to messages of God, faith and/or acceptance. The people who chose not to participate, or who dropped out of the study, were not always accounted for. Researchers could not determine whether the correlation they observed was due to the treatment or to the characteristics of the people participating.
- Twelve-step groups have no standardized methods or conditions. Leaders of the groups are often laypeople in recovery from addiction themselves. The quality of social support in the group depends on the people who are participating. The literature is interpreted by the members, who create their own cultures around the interpretation. Twelve-step cultures also pass around other information and advice, which may or may not permeate every group. Each sponsor is a different layperson in recovery from addiction, with different character traits. Researchers could not control for all of these variables all of the time.
- Researchers struggled to maintain rigorous control groups throughout studies. At minimum, to determine whether 12-step groups have effect, researchers needed a no-treatment control group for each study. Ethically and logistically, they could not prevent people in the control groups from receiving treatment or from attending 12-step groups.
Twelve-Step Facilitation was developed by researchers working on Project MATCH, a well-known and extensive study funded by the National Institutes of Health. Project MATCH compared TSF to Motivational Enhancement Therapy (MET) and Cognitive Behavioral Therapy (CBT), seeking to establish what patient characteristics corresponded with the best results for each treatment. The study found there “was little difference in outcomes by type of treatment” based on the primary outcome measures of percent days abstinent and drinks per drinking day.
By standardizing methodology for TSF, Project MATCH made some headway on strengthening the quality of evidence, but they did not find a way around self-selection bias and they did not have a control group. Many patients, however, did drop out of the assigned treatments early on in the study. Two researchers later examined the outcomes of the zero-treatment dropout group, and found that “two-thirds to three-fourths of the improvement in the full treatment group was duplicated in the zero-treatment group.” This means that the people in Project MATCH’s treatment groups did not have significantly better abstinence outcomes than the people who dropped out of the study. Importantly, we do not know whether the dropout group sought treatment on their own, and it seems probable that they did. Based on their analysis, none of the interventions in Project MATCH seem to be effective, but without an actual control group the results are equivocal regardless.
Some researchers have sought to re-analyze other parts of the Project MATCH data, but their findings, while supportive of TSF, are subject to the same methodological limitations of the parent study. Many other studies cited by the Surgeon General’s Report seem to support TSF as effective for improving abstinence outcomes and/or for relatively increasing 12-step participation compared to treatment as usual (TAU), but none of these studies had control groups. The Surgeon General’s Report cited one source in support of TSF that was actually an article reviewing information about 12-step programs to educate social workers, not an experimental study. The Report also cited a study in support of TSF that examined two active referral interventions, 12-step peer intervention (PI) and doctor intervention (DI), compared to no intervention (NI). The study found that while the active referral interventions significantly increased participation in 12-step groups compared to no intervention, “abstinence rates did not differ significantly across intervention groups (44% [PI], 41% [DI] and 36% [NI]).” This study was the only one cited in the Surgeon General’s Report in support of TSF that approximates a control group, and it does not actually support the efficacy of TSF in increasing abstinence outcomes. The NI pseudo-control group still received a list of 12-step group meeting times and locations, but was not encouraged to attend. The PI group attended meetings twice as much as the NI group, and yet the researchers found no significant difference in abstinence outcomes. The DI group, essentially TSF, was less effective than the PI group at increasing attendance, and again, did not significantly improve abstinence.
My own literature review turned up articles the Surgeon General’s Report did not reference, both in support of TSF and not supporting TSF, but none of the studies I found had control groups either. Results of my literature review, including my assessment of the Surgeon General’s report sources, were therefore as ambivalent as the 2006 Cochrane Review, a systematic meta-study of all 12-step-based programs that found that, “No experimental studies unequivocally demonstrated the effectiveness of AA or TSF approaches for reducing alcohol dependence or problems.” Researchers have not been able to methodologically eliminate self-selection bias or utilize adequate controls in their studies of 12-step groups and TSF.
One study to date attempted to use instrumental variables models, a recently developed statistical method, to determine what percentage of increased 12-step group attendance can be attributed to TSF without self-selection bias. The researchers, Humphreys et al. 2014, re-analyzed data from five randomized clinical trials, though none of these clinical trials had control groups. They determined that TSF did significantly increase participation in 12-step groups for people who had not previously participated much or at all, though not for people who already had high levels of participation. Their methods were inadequate for determining whether increased participation is causally linked to increased abstinence, or whether increases in abstinence occurred without self-selection bias.
My review answers a question that cannot be answered by simply counting the number of studies apparently supporting or not supporting 12-step approaches like TSF. Are 12-step approaches evidence-based for treating addiction – supported by numerous scientific experiments with rigorous methods that include control groups, randomization of patients to treatments, and bias-free samples? No, 12-step approaches are not evidence-based, because no studies decisively demonstrate their effectiveness. In fact, research on 12-step approaches faces insurmountable obstacles. Practically speaking, researchers cannot ethically design and conduct experiments that eliminate self-selection bias and utilize adequate controls.
In medical science, if a treatment is ineffective or faces prohibitive methodological challenges, the treatment is either revised or abandoned. Twelve-step philosophy prohibits either approach. Twelve-step literature is comparable to the Bible for Christians or the Qur’an for Muslims; if the literature is removed, the identity of the group goes with it. The same basic text has been used for AA since the publication of its “Big Book,” Alcoholics Anonymous, in 1939. Twelve-step literature also explicitly states that, “Those who do not recover are people who cannot or will not completely give themselves to this simple program, usually men and women who are constitutionally incapable of being honest with themselves. There are such unfortunates. They are not at fault; they seem to have been born that way. They are naturally incapable of grasping and developing a manner of living which demands rigorous honesty. Their chances are less than average.” Twelve-step philosophy, by taking this position, is asserting that its methods can never be wrong. If the 12 Steps do not work for people, 12-step philosophy explicitly states it is their fault, and that the fault is inborn and irreversible. The 12 Steps and attendant literature, however, are not modified.
Research does support the concept that changing “people, places, and things” and finding a network of people with a culture of abstinence can improve chances of recovery. However, mutual help groups other than 12-step groups do exist that may provide the social support needed by people in recovery. A study by Zemore et al. compared the relative, personal satisfaction of participants in 12-step groups vs. 12-step alternatives: SMART Recovery, LifeRing, and Women For Sobriety. The study did not examine abstinence or drinking/using outcomes among participants, but interviewed all participants on demographics, level of participation in their mutual help group (MHG) of choice, and level of satisfaction and group cohesion they experienced in their MHG. They found that, “despite lower levels of in-person meeting attendance, members of all the 12-step alternatives showed equivalent activity involvement and higher levels of satisfaction and cohesion, compared to 12-step members.” The participants who engaged in alternative MHGs tended to be less religious, and to have higher levels of education and income. The study demonstrates that alternative MHGs not only exist, but are of comparable efficacy in terms of social support. People who are not religious may be able to make 12-step groups work for them as social support if they have no other choices, but other options will most often be available.
A study in 2001 by Humphreys and Moos found that TSF may reduce health care costs for people in recovery by emphasizing reliance on free 12-step groups, as opposed to cognitive behavioral therapy. Yet their conclusions that the study indicates people should be diverted from CBT to TSF because it is ultimately cheaper amounts to advocating malpractice. TSF itself is not free and is not decisively supported by evidence; twelve-step groups, while free, are not evidence-based treatment, and other available mutual help groups are equally free options for social support. Even if TSF were demonstrably effective at promoting abstinence for some people, 12-step philosophy is heavily spiritual (specifically Christian-based), so it would be unethical to recommend TSF simply because it might save money.