Advertisement
Research Article| Volume 120, 108176, January 2021

It's not just the money: The role of treatment ideology in publicly funded substance use disorder treatment

Published:October 20, 2020DOI:https://doi.org/10.1016/j.jsat.2020.108176

      Highlights

      • The majority of substance use disorder (SUD) treatment facilities do not offer medications for opioid use disorder (MOUD).
      • MOUD expansion efforts have typically focused on practical and structural barriers such as prescribers and cost.
      • Most specialty SUD treatment organizations in Philadelphia are adopting at least one MOUD.
      • A qualitative analysis indicates that practical and ideological barriers thwart the adoption and implementation of MOUD.
      • Facilitators to MOUD adoption include the presence of strong leadership, a redefinition of “recovery,” and rapid workforce turnover.

      Abstract

      Medications for opioid use disorder (MOUD) are a first-line treatment for opioid use disorder, yet national surveys indicate that most substance use treatment facilities do not offer MOUD. This article presents the results of a qualitative analysis of interviews with leaders from 25 treatment organizations in Philadelphia, Pennsylvania, that investigated attitudes and barriers toward MOUD. Most treatment organizations that we interviewed are adopting at least one MOUD, suggesting that Philadelphia exceeds the national average of organizations with MOUD capacity. Leaders indicated that both practical resources and ideological barriers thwart adoption and implementation of MOUD in publicly funded substance use disorder treatment agencies. Organizations that had recently adopted MOUDs revealed facilitators to MOUD adoption, such as strong leadership that champions the implementation to staff and redefining recovery from substance use disorders throughout the organization. This study's findings highlight that clients, clinicians, and leadership need to address both practical and ideological barriers to expanding MOUD use.

      Abbreviations:

      MOUD (Medications for opioid use disorder), DBHIDS (Department of Behavioral Health and Intellectual disAbility Services), CBH (Community Behavioral Health)

      Keywords

      1. Introduction

      Opioid overdoses in the U.S. opioid epidemic may have finally reached the apogee in 2017, yet the death toll remains staggeringly high and millions of Americans continue to suffer from opioid use disorder (OUD) (
      • Gladden R.M.
      • O’Donnell J.
      • Mattson C.L.
      • Seth P.
      Changes in opioid-involved overdose deaths by opioid type and presence of benzodiazepines, cocaine, and methamphetamine — 25 states, July–December 2017 to January–June 2018.
      ). A large and growing body of evidence supports medications for opioid use disorder (MOUD) as the most effective treatment (
      • Sordo L.
      • Barrio G.
      • Bravo M.J.
      • Indave B.I.
      • Degenhardt L.
      • Wiessing L.
      • Pastor-Barriuso R.
      Mortality risk during and after opioid substitution treatment: Systematic review and meta-analysis of cohort studies.
      ). In particular, methadone and buprenorphine, the two FDA-approved opioid agonists/partial agonists, reduce morbidity, all-cause mortality, and opioid related mortality (
      • Mattick R.P.
      • Breen C.
      • Kimber J.
      • Davoli M.
      Buprenorphine maintenance versus placebo or methadone maintenance for opioid dependence.
      ). Naltrexone is the newest FDA-approved MOUD; it is less well studied but has shown similar outcomes to agonist/partial agonist therapies (
      • Bisaga A.
      • Mannelli P.
      • Yu M.
      • Nangia N.
      • Graham C.E.
      • Tompkins D.A.
      • Sullivan M.A.
      Outpatient transition to extended-release injectable naltrexone for patients with opioid use disorder: A phase 3 randomized trial.
      ;
      • Comer S.D.
      • Sullivan M.A.
      • Yu E.
      • Rothenberg J.L.
      • Kleber H.D.
      • Kampman K.
      • O’Brien C.P.
      Injectable, sustained-release naltrexone for the treatment of opioid dependence: A randomized, placebo-controlled trial.
      ). Despite this robust evidence, according to a recent large national survey of outpatient substance use treatment facilities, only 36% of all organizations and only 33% of publicly funded organizations offer any MOUD (
      • Mojtabai R.
      • Mauro C.
      • Wall M.M.
      • Barry C.L.
      • Olfson M.
      Medication treatment for opioid use disorders in substance use treatment facilities.
      ). Nonetheless, these numbers represent a substantial increase in the past decade, largely driven by more facilities offering buprenorphine and naltrexone. Increasing the availability of MOUD is one of the most promising strategies to combat the opioid epidemic, and public agencies are increasing their efforts to address the gap between research evidence and current practices (
      • Barnett M.L.
      • Gray J.
      • Zink A.
      • Jena A.B.
      Coupling policymaking with evaluation — The case of the opioid crisis.
      ).
      A small body of literature compares characteristics of substance use facilities that do and do not offer MOUD. Interviews with 250 administrators of publicly funded treatment programs found that administrators most strongly endorsed regulatory, funding, and infrastructure barriers as reasons for not adopting MOUD (
      • Knudsen H.K.
      • Abraham A.J.
      • Oser C.B.
      Barriers to the implementation of medication-assisted treatment for substance use disorders: The importance of funding policies and medical infrastructure.
      ). Researchers are currently testing an implementation strategy called the Prescriber Recruitment Bundle (PRB) on the assumption that the primary barrier to MOUD adoption and implementation is structural, specifically physician prescribing capacity.
      Administrators of treatment programs were less likely to endorse ideological barriers (
      • Knudsen H.K.
      • Abraham A.J.
      • Oser C.B.
      Barriers to the implementation of medication-assisted treatment for substance use disorders: The importance of funding policies and medical infrastructure.
      ), defined here as a set of beliefs about OUD treatment that inhibit MOUD prescribing. This treatment ideology has also been called “intervention stigma,” which describes a specific stigma toward MOUD. Intervention stigma differs from “condition stigma,” which describes a stigma against people with a specific medical condition such as a substance use disorder (SUD) (
      • Madden E.F.
      Intervention stigma: How medication-assisted treatment marginalizes patients and providers.
      ;
      • Wakeman S.E.
      Medications for addiction treatment: Changing language to improve care.
      ). Ideological barriers to MOUD include the idea that abstinence from all substances (particularly agonist therapies) is the only path to recovery. Explicit in this ideology is that a medication for an SUD is “replacing one addiction for another” (
      • Galanter M.
      Combining medically assisted treatment and twelve-step programming: A perspective and review.
      ;
      • Olsen Y.
      • Sharfstein J.M.
      Confronting the stigma of opioid use disorder--and its treatment.
      ). The abstinence-only belief system stems from the 12-Step approach that emerged in the 1930s as a treatment for alcohol dependence, when there were no medications available to assist with recovery with substance use. This approach heavily influenced most treatment facilities across the country and created a normative legacy of abstinence-based substance use treatment, despite the later development of efficacious medications (
      • Galanter M.
      Combining medically assisted treatment and twelve-step programming: A perspective and review.
      ).
      In contrast to Knudsen and colleagues, who found the chief importance of logistical barriers, other studies have been equivocal about the association between treatment ideology and medication adoption. One national survey of executive directors and clinical supervisors from 547 outpatient substance use treatment units found that those who adopted buprenorphine were much less likely to endorse abstinence as the most important treatment goal (
      • Friedmann P.D.
      • Jiang L.
      • Alexander J.A.
      Top manager effects on buprenorphine adoption in outpatient substance abuse treatment programs.
      ). A similar nationally representative study surveying counselors' attitudes found that counselors' agreement with the 12-step philosophy was negatively associated with acceptability of agonist medications (
      • Aletraris L.
      • Edmond M.B.
      • Paino M.
      • Fields D.
      • Roman P.M.
      Counselor training and attitudes toward pharmacotherapies for opioid use disorder.
      ). Other studies have found no relationship (
      • Abraham A.J.
      • Knudsen H.K.
      • Rieckmann T.
      • Roman P.M.
      Disparities in access to physicians and medications for the treatment of substance use disorders between publicly and privately funded treatment programs in the United States.
      ;
      • Knudsen H.K.
      • Ducharme L.J.
      • Roman P.M.
      Research network involvement and addiction treatment center staff: Counselor attitudes toward buprenorphine.
      ).
      Understanding that MOUD is the treatment with the most evidence to support it, the City of Philadelphia Department of Behavioral Health and Intellectual disAbility Services (DBHIDS) enacted a new policy in July 2018 that affected all publicly funded SUD treatment organizations in the city. This policy removed barriers to reimbursement (such as prior authorization requirements for medically monitored residential care), provided enhanced rates, incentivized organizations through the potential of pay-for-performance, and mandated that all residential treatment facilities provide MOUD by January 2020. To our knowledge, Philadelphia is the first municipality to mandate MOUD. The Philadelphia MOUD mandate is the most recent in a suite of approaches DBHIDS has taken to remove structural barriers to care. Efforts thus far have had modest impact. Perhaps, ideological obstacles in the treatment community are a substantial barrier that these policies do not address.
      The current study assessed the perspectives and experiences of leaders from organizations that either have not adopted MOUD or have only recently adopted MOUD. To our knowledge this is the first study specifically targeting organizations that have not adopted MOUD, even in the presence of a forthcoming mandate to adopt MOUD. Understanding factors that lead to MOUD adoption or nonadoption can provide insight into barriers and facilitators to adopting and implementing MOUD for potentially reticent organizations and inform the development of targeted strategies to increase implementation of MOUD in specialty SUD treatment.

      2. Methods

      2.1 Setting

      Philadelphia's DBHIDS is a large, publicly funded behavioral health system that serves approximately 169,000 individuals annually through a network of approximately 200 provider agencies. DBHIDS, in conjunction with Community Behavioral Health (a 501[c][3] behavioral health managed care organization contracted by Philadelphia), is the single payer for all publicly funded substance use treatment services in Philadelphia. On July 1, 2018, DBHIDS announced major MOUD-related policy changes affecting all treatment organizations in the city. The policy removed barriers to reimbursement (such as waiving prior authorization requirements for medically monitored residential care), provided enhanced rates, incentivized organizations through the potential of pay-for-performance, and ultimately required that all organizations document how they would become MOUD-capable by January 2020. We conducted study interviews from December 2018 to June 2019 in the time period after the city announced the policy but before enactment.

      2.2 Participants and procedure

      We invited all drug and alcohol treatment agencies (n = 53) located in Philadelphia County to participate. Our recruitment strategy was multi-pronged, with the goal of oversampling nonadopting organizations. First, based on billing codes, DBHIDS identified seventeen agencies (23%) that, to their knowledge, had not adopted any MOUD. The Commissioner of DBHIDS emailed the executive directors of all nonadopter agencies to encourage them to participate in the study. Next, we contacted administrative and clinical leadership at all 53 agencies to ascertain interest and invited all organizations to participate in a study exploring perspectives about MOUD. Third, we advertised our study at a forum that CBH hosted and that leaders of most Philadelphia substance use treatment organizations attended. We interviewed leaders because we wanted to speak with the person at the agency who was best acquainted with the selection, promotion, operation, and implementation of MOUDs. We conducted interviews with leaders from 25 organizations from December 2018 to June 2019. We conducted thirteen interviews with leaders from organizations who were adopting agonist medications, five with leaders from organizations who were adopting antagonist medications only, and seven with leadership from nonadopting organizations. Leaders reported these adopter categorizations in the interview. We included all organizations in our analyses so that we could compare responses of adopters and nonadopters. We ceased recruitment when we reached thematic saturation (after the 20th interview), and we proceeded with five additional scheduled interviews. Participants gave written consent and we did not compensate them. The City of Philadelphia and University of Pennsylvania Institutional Review Boards approved all study procedurs.

      2.3 Qualitative interview

      We developed a semistructured interview guide consisting of two parts. The first set of questions were exploratory and open-ended. We asked about the agency's treatment philosophy and population, and how the organization had been affected by the opioid epidemic. We asked leadership's attitudes toward MOUD as a response to the opioid epidemic, and how and if their organization had capacity to deliver MOUD. If they had recently implemented MOUD, specific probes queried on how they accomplished and financed this. The second set of questions specifically concerned organizational and payer supports that their organization might need to adopt to implement or enhance MOUD capacity in their organization. Specific probes queried both practical and ideological barriers. The interview also included open-ended questions if the organization was uninterested in implementing MOUD to understand why they were uninterested.

      2.4 Data analysis

      An integrated approach, which uses an inductive process of iterative coding to identify recurrent themes, categories, and relationships, guided our analysis (
      • Bradley E.H.
      • Curry L.A.
      • Devers K.J.
      Qualitative data analysis for health services research: Developing taxonomy, themes, and theory.
      ). We identified a priori attributes of interest (e.g., adoption of agonist or antagonist medications, practical and logistical barriers, ideology, stigma, facilitators) and also used modified grounded theory (
      • Glaser B.
      • Strauss A.
      The discovery of grounded theory: Strategies for qualitative research.
      ) to inform our overall data analytic strategy given its exploratory nature. We used an iterative process to analyze and develop a codebook for the interview transcripts. We digitally recorded and transcribed the interviews, with NVIVO qualitative data analysis software supporting our analyses. Using the NVIVO qualitative data analysis software program, three members of the research team (RS, JY, and RV) separately coded a sample of 5 transcripts and compared their application of the coding scheme to assess the reliability and robustness of the coding scheme. Discussion resolved disagreements in coding, and we refined the codebook several times as we recorded and coded new interviews. No new codes emerged after the 20th interview, suggesting that we had reach saturation (
      • Glaser B.
      • Strauss A.
      The discovery of grounded theory: Strategies for qualitative research.
      ). We collected five additional previously scheduled interviews out of respect to the participants, and no new themes emerged. We then applied the revised codebook to all interviews. JY and RV coded all transcripts, and RS separately coded 25%. Reliability was excellent (κ = 0.96; [
      • Landis J.R.
      • Koch G.G.
      The measurement of observer agreement for categorical data.
      ]).
      Once we had analyzed the data, to ensure the trustworthiness of the findings, the researchers shared their findings through four different presentations at local and state conferences with multiple stakeholders (including members of the target population who had not participated in the study). All presentations concluded with a question and answer session, during which the lead author asked the audience to corroborate the themes that the study data produced, and identify any new concepts that the research team may have missed. This exercise identified no new themes, and the stakeholders who responded confirmed all concepts that the research team had produced. Additionally, stakeholders agreed that these themes were the most important emerging from the data. These conversations enhanced our interpretation of the data as well as the findings that we articulate in the Discussion section. As a result of this auditing process, the authors were able to have more confidence that their biases had not influenced the interpretation of the data.

      3. Results

      3.1 Sample and agency characteristics

      Leaders included 20 executive directors/CEOs and five clinical leaders (i.e., clinical director or chief medical officer). Leaders averaged 56.8 years of age (SD = 12.1). Fifteen of the participants identified as male, and 10 identified as female. Most participants (80%) identified as white, two participants identified as Hispanic/Latino, one participant identified as more than one race, and one participant identified as Black. Of the 25 organizations, 13 were adopting agonist/partial agonist medications (and 11 of those also were adopting antagonist medications), five organizations were adopting antagonist medications only, and seven were not adopting any medications. Unless noted, there are no differences in themes by adopter group.

      3.2 The components of effective treatment

      Most agency leaders asserted that MOUDs were not a standalone treatment for substance use disorders, “We don't just prescribe. We have to provide treatment with it.” That is, leaders characterized MOUDs as an adjunctive to psychosocial treatments. As one executive director noted, “It's not treatment. It [MOUD] doesn't take the place of psychological treatment.” While leaders agreed that MOUDs protected patients from overdose, they noted that medications by themselves did not treat the underlying substance use disorder. As one CEO remarked: “They're giving them more powerful drugs to protect them, but if you think that's how you're going to solve addiction, you got a problem there. It doesn't stop you from smoking crack or [synthetic marijuana] so you got to treat addiction. If you don't treat addiction they got drugs the docs don't have a drug for.” Additionally, leaders shared a concern that promoting MOUDs has caused the field to abandon psychosocial treatments: “Listen, what happened to treatment? This [MOUD] was supposed to be integrated with treatments.” Agency leaders listed many psychosocial treatments provided at their organizations. A list of 22 interventions the 25 organizations can be found in Table 1.
      Table 1List of psychosocial treatments that SUD treatment agencies reported (N = 25).
      List of psychosocial treatmentsNumber of agencies reported
      Agencies could identify as delivering multiple psychosocial treatments.
      12-step, or recovery support network12
      Cognitive Behavioral Therapy
      Multiple forms of Cognitive Behavioral Therapy were identified, including Relapse Prevention Model, Dialectical Behavior Therapy, Trauma-Focused Cognitive-Behavioral Therapy, Transdiagnostic Behavioral Therapy and Prolonged Exposure.
      10
      Motivational interviewing5
      Psychoeducation5
      Trauma-informed care3
      Therapeutic community3
      Matrix model2
      Contingency management2
      Family therapy1
      Spiritual therapy1
      Seeking safety1
      Reality therapy1
      Music/art/writing therapy1
      Insight-oriented therapy1
      Screening, Brief intervention, and Referral to treatment (SBIRT)1
      a Agencies could identify as delivering multiple psychosocial treatments.
      b Multiple forms of Cognitive Behavioral Therapy were identified, including Relapse Prevention Model, Dialectical Behavior Therapy, Trauma-Focused Cognitive-Behavioral Therapy, Transdiagnostic Behavioral Therapy and Prolonged Exposure.

      3.3 Practical barriers to MOUD

      3.3.1 Lack of resources

      Agency leaders agreed that a lack of financial resources impeded MOUD adoption and implementation. As one leader summarized, “Everything is lack of resources. Most agencies have a hard time running just on what exists right now.” A common explanation for lack of resources is low reimbursement rates in the publicly funded behavioral health system. Leaders also commented that their organizations provided many support services for which they were not reimbursed: “One of the biggest problems is the additional non-billable support services that are required in order to sustain treatment for this population. There's a lot of outreach that has to happen so people don't disappear.” Leaders also shared that low reimbursement rates and lack of resources made hiring additional medical staff to implement MOUD difficult or impossible. One leader questioned: “Where is the money for the psychiatrist [who] is going to write and manage that prescription? Where is the money for someone to come in and educate my staff?”

      3.3.2 Organizational discomfort

      According to agency leaders, implementing MOUD (particularly agonist/partial agonist medications) is challenging, “confusing,” and “daunting.” One nonadopter summarized, “I think from an administrative point of view it would be a nightmare to manage.” Leaders asserted that this uncertainty and discomfort stopped them from adopting MOUDs. As one leader commented: “When you talk about suboxone, there's a heavy burden around management of that. When the patient comes in, how long do you continue to see them? The induction period and all of that and I think for outpatient settings and some clinicians, it's much different even when they're used to treating different types of medical concerns.” Leaders frequently noted fears of regulatory scrutiny; one CEO commented: “People often don't want to prescribe it because they're afraid that they have to keep a record in a special way or the DEA may swoop. They'd rather not dabble in [prescribing MOUD].” Several leaders commented that trainings did not adequately alleviate anxiety: “Front-line staff are also uncomfortable and a little confused about what the obligations are even when they've had training in suboxone.” Several agencies indicated that a mentor agency or learning collaborative would be helpful. “It would be great to have a mentor agency or someplace out in the community where we could go with all of those day-to-day questions that come up in the management and administration of a program such as that. That would be something that would sway us in a direction more quickly.”

      3.3.3 Client choice/demand

      Leaders noted that a guiding philosophy in the organizational approach to SUD is “meeting the client where s/he is.” As such, they emphasized the importance of client choice in treatment, noting that a barrier to providing MOUD is that many patients do not want medications of any kind. As one leader remarked: “I find a lot of guys come in, and they don't want to be on any medication. They get their mindset that they want to be completely drug-free.” Client demand, as one leader explained, is the reason for low access to MOUD: “The access issue isn't that there aren't places willing to accept; it's that people aren't interested in what we have to offer.”

      3.4 Ideological barriers to agonist treatment

      Leaders described several ideological barriers to agonist treatment, which related to beliefs that patients, clinical staff, fellow organizational leaders, and the public hold. While adopter groups did not differ in the attitudes that different stakeholders held, leaders in antagonist-only adopting and nonadopting groups were more likely to endorse that they personally held these beliefs. Groups also differed in their description of their organizational response to the barrier.

      3.4.1 Substituting one drug for another

      Leaders noted a widespread belief that agonist treatment is “substituting one drug for another”, which impedes the uptake of agonist treatment for OUD. As one CEO explained, “There is still that hardcore principle to overcome that people think you're trading a drug for a drug and is that recovery.” Further, they noted that agonist treatment is often seen as inconsistent with traditional abstinence-based treatment: “Folks feel as though if you are going to be abstinent then you are going to not use anything and that the medication is just the crutch.” A related, pervasive belief that leaders expressed was that patients with an agonist medication aren't considered “clean.” As one staunch nonadopter remarked about a methadone clinic: “They [other organization] detox with methadone, which is not a detox. It's still an opiate!”

      3.4.2 Diversion anxiety

      All leaders saw the potential to divert buprenorphine as negatively affecting MOUD adoption and implementation. According to these leaders, buprenorphine is problematic because it can be sold or diverted to other drug users, or even to staff. This is sometimes viewed as “contributing to drug culture.” For antagonist-only-adopting and nonadopting agencies, this was the most commonly cited reason not to adopt buprenorphine. One leader summarized: “The main objection is abuse. I'd rather not mess with all that.” In contrast, agonist-adopting agencies had developed protocols to reduce the risk of diversion, such as limiting quantities of medication allowed or methods of delivery. As one leader described: “MAT as itself is a good principle if it's possible to keep it under control. We dispense [instead of writing prescriptions], and I think that's definitely the part of our managing the diversion.”

      3.4.3 Cognitive impairment

      Many leaders described a belief that being on agonist/partial agonist medications (or too high a dose) can cause patients to “nod off” or otherwise not engage in treatment or progress toward recovery. For antagonist-only-adopting and nonadopting agencies, leaders cited the cognitive impairment as a critically important reason not to adopt these medications. As one described: “Sure, being alive is better than not alive but, man, what kind of life is that?” There was also a belief among antagonist-only-adopting and nonadopting agencies that agonist agencies increase the dosages of agonist medication to increase retention. Leaders from agonist-adopting agencies described how dose management was part of daily practice: “If we see someone nodding it means there is possibly an interaction with their other medications (prescribed or not), or that we need to adjust their dose. This is something we check in on every single day with every single patient.”

      3.4.4 Agonist medications trigger staff in recovery and other patients

      Organizational leaders cited a belief that having agonist medications on site or patients on agonist medications can be triggering to (abstinent) staff in recovery and other patients. Leaders in antagonist-only and nonadopting organizations more frequently endorsed this belief. Patients on agonist medications are described as cognitively impaired and easily identifiable. As one nonadopting leader asserted: “It's a whole different person you're talking to than the guy who is drug-free.” These leaders also described how having “mixed” groups are damaging to the recovery of patients who do not use medications. One CEO remarked: “I can tell you what is dangerous. Having people in their facility that all they want to do is be on buprenorphine and they are sitting in a facility with people that are serious about what they're doing is a bad thing. Somebody nodding out and falling out of a chair in a group is a bad thing.” Last, the presence of medication onsite is considered dangerous because staff could take (or divert) it, or drug-free patients could react to those patients on medications. One leader remarked: “I too am in recovery, and the thought of bringing opiates into an opiate treatment facility is troubling to me but terrifying to the men in recovery. Our guys couldn't understand why you were giving drugs to people; it triggers them.” In contrast, agonist-adopting organizations took the opposite approach and purposefully mixed drug-free and patients using agonist medications. They used this practice to reduce stigma, re-define recovery, and encourage patients to learn from one another: “Ultimately we try to mix our groups together because in the name of our vision in stigma reduction by singling people out and saying ‘Well, you're on methadone, and you shouldn't be in groups with other people recovering in other ways.’ I find that we can cause that stigma to root even deeper. We try to mix all of our groups together and just deal with matters of recovery.”

      3.5 Antagonist medications

      Leaders had far fewer qualms about using antagonist medications. In fact, some thought that the antagonist medication Vivitrol is underused: “I feel like I personally don't know a ton about Vivitrol because none of my clients is on it. I don't know why it's not talked about more.” No leader raised practical or ideological concerns around Vivitrol. Leaders also noted that Vivitrol is acceptable among peers and staff in recovery: “The guys are fine with it, who is on it, who is not, they don't care.” The leaders considered Vivitrol safe due to its formulation: “You can't abuse Vivitrol, it was marketed that way.”

      3.6 Organizational facilitators to newly adopting agonist medications

      Six of the 13 agonist-adopting organizations could be classified as “newly adopting,” which we defined as a staged process of MOUD implementation over the prior two years. Three facilitating themes emerged from leaders in organizations that had adopted agonist MOUD.

      3.6.1 Presence of strong leadership and workforce turnover

      Strong leadership that championed MOUDs facilitated the adoption of medication at agonist/partial agonist-adopting-agencies. One CEO described: “I'm the CEO here, so what I say goes. If I put my foot down, jump up and down, insist demand and push, I get my way. That's tough because I have people who are married to an ideology that's against what I'm teaching.” Many leaders in these agencies described workforce turnover as an effective strategy for organizational change. As one leader remarked: “In May I fired my whole management team except for one. I was tired of the massive implementation and the drift, and we would drift back if we didn't have completely new personnel.” For those organizations who retained their workforce, leaders described their outreach efforts to those staff who included direct messaging about the new philosophy of the organization. As one leader described her efforts, “We've retained the traditional folks who don't have full support of MAT… everybody is entitled to their opinion but we need to support the organization and they need to support other people's paths to recovery. I told them this is the direction we are going.”

      3.6.2 Redefining recovery throughout the organization

      Leaders championed a “massive culture shift” at their agencies through changing the organizational view that recovery was something one could do with or without medications. As one leader expounded: “What had to happen organizationally is we had to change - not just approach, but our perspective of what abstinence based includes.” Leaders messaged staff directly and mixed peer groups across treatment modalities and organization-sponsored recovery events to increase awareness of different approaches to recovery. Leaders in transitioning organizations described group mixing as a catalyst for change in the organization. “…one way they did this was by mixing peers of all kinds to drill down this point that, ‘everybody's path to recovery is a valid one.’ Once they started talking to each other they found out they had a lot more in common than they didn't.”

      3.6.3 Experiencing improved retention and outcomes

      According to leaders from organizations that transitioned to medication, observing patient improvement was a major facilitator of that transition. As one CEO described: “Those of us working with the MAT population were just getting excited about the work. Then we started asking, ‘How can we do more of this,’ because this [MOUD] clearly is what seems to be helping these individuals.” Leaders noted improved outcomes and retention (e.g., “our retention rates have tripled”). Several noted improved financial status accruing from improved retention and billing.

      4. Discussion

      This qualitative study is among the first to explore the experiences of organizations that did and did not adopt MOUD. Our sample included nonadopters, adopters, and newly adopting organizations. Organizations were heterogeneous in which medications they were adopting and the stage of adoption or implementation. In general, nonadopting organizations fell into two categories: those who are cautiously willing to adopt MOUD but need financial and staffing support to do so, and those who oppose the use of agonist medications on ideological grounds. It is encouraging to note, however, that one-quarter of the organizations in the sample had recently transitioned from nonadopter to adopter. Despite our efforts to oversample nonadopting organizations for this study, most treatment organizations that we discussed here were adopting at least one MOUD, and most were delivering an agonist medication, suggesting that Philadelphia exceeds the national average of organizations with MOUD capacity.
      Consistent with prior studies (
      • Knudsen H.K.
      • Abraham A.J.
      • Oser C.B.
      Barriers to the implementation of medication-assisted treatment for substance use disorders: The importance of funding policies and medical infrastructure.
      ), we found that there are clear structural and financial barriers to MOUD adoption and implementation. Providers noted workforce constraints related to the affordability (and therefore availability) of prescribers. They also agreed that low reimbursement rates and an inability to bill for certain SUD services that they provided (such as services used to engage and retain clients) were significant financial barriers to overall organizational operations, including the provision of MOUD. We should note that not one organizational leader mentioned the x-waiver (required to prescribe buprenorphine) as a barrier to expanding the use of MOUD, contrary to recent commentaries postulating that waiver acquisition is a barrier to increased availability of prescribers (
      • Berk J.
      To help providers fight the opioid epidemic, “X The X Waiver” | Health Affairs.
      ;
      • Fiscella K.
      • Wakeman S.E.
      • Beletsky L.
      Buprenorphine deregulation and mainstreaming treatment for opioid use disorder: X the X waiver.
      ). More research is needed on how to optimize payment in publicly funded SUD treatment to encourage the delivery of MOUDs. A recent review of empirical studies of pay-for-performance (P4P) in behavioral health suggested that although more empirical examination is needed, P4P can be an effective strategy to improve outcomes (
      • Stewart R.E.
      • Lareef I.
      • Hadley T.R.
      • Mandell D.S.
      Can we pay for performance in behavioral health care?.
      ). And despite increased attention to value-based purchasing, research on other promising alternative payment mechanisms (such as bundled rates) in SUD treatment is nascent (
      • Quinn A.E.
      • Hodgkin D.
      • Perloff J.N.
      • Stewart M.T.
      • Brolin M.
      • Lane N.
      • Horgan C.M.
      Design and impact of bundled payment for detox and follow-up care.
      ). Bundled rates hold particular promise because they would facilitate agencies to provide traditionally unbillable services that may improve patient care. Paying for outcomes is viewed as an effective but underutilized incentive for behavioral health, but also presents special challenges unique to substance use treatment (
      • Hodgkin D.
      • Garnick D.W.
      • Horgan C.M.
      • Busch A.B.
      • Stewart M.T.
      • Reif S.
      Is it feasible to pay specialty substance use disorder treatment programs based on patient outcomes?.
      ;
      • Stewart R.E.
      • Marcus S.C.
      • Hadley T.R.
      • Hepburn B.M.
      • Mandell D.S.
      State adoption of incentives to promote evidence-based practices in behavioral health systems.
      ). More work is needed to operationalize this strategy, and greater consensus is required on performance and outcome measures (
      • Garnick D.W.
      • Horgan C.M.
      • Acevedo A.
      • McCorry F.
      • Weisner C.
      Performance measures for substance use disorders – What research is needed?.
      ). Leaders of organizations indicated that traditional trainings do not address provider and clinician discomfort with MOUD. Detailing and learning collaboratives are promising strategies (
      • Molfenter T.
      • Knudsen H.K.
      • Brown R.
      • Jacobson N.
      • Horst J.
      • Van Etten M.
      • Madden L.
      Test of a workforce development intervention to expand opioid use disorder treatment pharmacotherapy prescribers: Protocol for a cluster randomized trial.
      ) to increase familiarity and comfort with clinical practice and regulations.
      Addressing practical barriers may be necessary but not sufficient to increase use of MOUD in substance use treatment facilities. As our community partners hypothesized, ideology plays an important role in MOUD uptake throughout an organization, from patient demand to executive director leadership. Ideological objections primarily surround agonist medications (rather than antagonist medications) and relate to traditional definitions of “abstinence” and “recovery” in substance use treatment. While leaders of nonadopting organizations were more likely to say that they personally subscribed to the objections (such as MOUD is substituting one drug for another), almost all of the leaders agreed that ideological objections are a barrier to more agencies adopting MOUD. MOUD expansion efforts have typically focused on prescriber capacity and cost with far less attention on ideology. Future research should focus on how to address negative attitudes and misunderstandings about agonist/partial agonist medications at the consumer, clinician, and leadership levels. Presentations of research evidence and statistics of well-controlled randomized studies are unlikely to be persuasive (
      • Stewart R.E.
      • Beidas R.S.
      • Mandell D.S.
      Stop calling them laggards: Strategies for encouraging nonadopters to incorporate evidence-based practices.
      ) and are not sufficient for securing buy-in, particularly for those individuals for whom MOUD may not be in the “latitude of acceptance” (
      • Sherif C.W.
      • Kelly M.
      • Rodgers Jr., H.L.
      • Sarup G.
      • Tittler B.I.
      Personal involvement, social judgment, and action.
      ). In contrast, there is a large body of literature on the persuasive effects of narrative and case study that has relevant and promising applications (
      • Borgida E.
      • Nisbett R.E.
      The differential impact of abstract vs. concrete information on decision.
      ;
      • Stewart R.E.
      • Chambless D.L.
      Interesting practitioners in training in empirically supported treatments: Research reviews versus case studies.
      ;
      • Ubel P.A.
      • Jepson C.
      • Baron J.
      The inclusion of patient testimonials in decision aids: Effects on treatment choices.
      ).
      • Heley K.
      • Kennedy-Hendricks A.
      • Niederdeppe J.
      • Barry C.L.
      Reducing health-related stigma through narrative messages.
      found that narrative messages reduced OUD stigma and increased attributions of responsibility to groups external to the individual (such as pharmaceutical companies and doctors). Interestingly, stigma toward OUD was more malleable than other health-related stigma (such as childhood obesity and smoking). It is unknown if narrative can reduce MOUD-related stigma by disrupting negative mental models of MOUD, but this is a promising area for future research. More empirical work is needed to develop evidence-based strategies designed to address ideology or stigma surrounding agonist medications.
      Our findings also reinforce that research has understudied consumer preferences for MOUD and that these preferences are likely underestimated in facilities' decision to adopt MOUD. Most leaders ascribed low access to MOUD as at least in part an issue of consumer demand. Indeed, two leaders indicated that their MOUD facilities were opening “drug-free” tracks to deliver services matched to patient preferences. More patient-level research is needed, specifically on what information clients desire to make a decision about OUD treatment, and how organizations present treatment choices. Shared decision-making holds promise, particularly the patient decision aid for medication treatment for opioid use disorder as a tool to standardize the presentation of information about MOUD (
      • Mooney L.J.
      • Valdez J.
      • Cousins S.J.
      • Yoo C.
      • Zhu Y.
      • Hser Y.-I.
      Patient decision aid for medication treatment for opioid use disorder (PtDA-MOUD): Rationale, methodology, and preliminary results.
      ). There is burgeoning literature on peer interventions (e.g., (
      • McGuire A.B.
      • Powell K.G.
      • Treitler P.C.
      • Wagner K.D.
      • Smith K.P.
      • Cooperman N.
      • Watson D.P.
      Emergency department-based peer support for opioid use disorder: Emergent functions and forms.
      ;
      • Scott C.K.
      • Grella C.E.
      • Nicholson L.
      • Dennis M.L.
      Opioid recovery initiation: Pilot test of a peer outreach and modified recovery management checkup intervention for out-of-treatment opioid users.
      ). Peers (i.e., certified recovery specialists) are “live” case studies, and they can, through their work, vividly demonstrate the efficacy of MOUD and dispel “zombie” myths about medication, thereby increasing consumer demand for medications. Last, we can increase consumer demand for MOUD if we educate the public and consumer. Despite major media attention on the opioid epidemic, there has been far less coverage on the underuse of MOUD (
      • Kennedy-Hendricks A.
      • Levin J.
      • Stone E.
      • McGinty E.E.
      • Gollust S.E.
      • Barry C.L.
      News media reporting on medication treatment for opioid use disorder amid the opioid epidemic.
      ). National polls indicate that only half the public (49%) believes that there is a treatment for prescription painkiller use disorder that is effective (
      • Blendon R.J.
      • Benson J.M.
      The public and the opioid-abuse epidemic.
      ). Many major cities, including Philadelphia, have launched media campaigns (e.g., “Bupe Works”) intended to educate the public about and grow demand for MOUD. Future research should investigate how best to craft messages that disseminate research findings about OUD to educate and inform policymakers and the public (
      • Purtle J.
      • Lê-Scherban F.
      • Wang X.
      • Shattuck P.T.
      • Proctor E.K.
      • Brownson R.C.
      Audience segmentation to disseminate behavioral health evidence to legislators: An empirical clustering analysis.
      ).
      We can learn a great deal from partnering with these organizations. Detailed case studies of organizations that have newly adopted MOUD can contribute to our knowledge of malleable organizational and psychological factors associated with MOUD adoption and implementation. One salient finding from our sample is that those in top management targeted attitudes toward recovery; they brought peers together and discussed re-defining recovery in that organization. Regardless of their MOUD adoption status, these organizations have been at the forefront of the opioid crisis and form the backbone of community treatment in Philadelphia and many other municipalities. They have knowledge and expertise to share. Relatedly, providers indicated that they do not consider MOUD as the treatment but as an adjunctive to the psychosocial treatments that they provide at their organization. This distinction is also embodied in the definition of “medication-assisted treatment” that federal agencies such as SAMSHA and NIDA have put forth, as medications in combination with counseling and behavioral therapies. Indeed, some leaders noted that they preferred the term medication-assisted treatment to MOUD because they felt it more accurately described the role of medications. Providers described the treatment as working because they “see it work” in their agencies. Therefore, instead of showing non- or low-adopting agencies randomized trial data of MOUD use (that is unlikely to persuade them), it may be better to partner with these organizations and use their own administrative data to show that what they are doing is effective. There may be opportunities to deploy measurement-based care within these organizations, not only to learn what may be effective in treatment but also to improve outcomes at the agency by basing clinical care on their own client data.

      4.1 Limitations

      Several study limitations deserve mention. First, we conducted this study in one city, and findings may not generalize to top leadership or agencies in other cities, particularly those outside a publicly funded system. Second, we only included the perspectives of agency leadership; we may have gleaned additional insights if we had included stakeholders such as front-line staff and consumers. Third, we did not systematically characterize what stage of implementation participating agencies were in (i.e., exploration, preparation, implementation, and sustainment), thus we are unable to draw conclusions about specific themes that might emerge based on stage of implementation (
      • Aarons G.A.
      • Hurlburt M.
      • Horwitz S.M.
      Advancing a conceptual model of evidence-based practice implementation in public service sectors.
      ). Last, there may have been response bias. Leadership from agencies that responded to the interview request may have different perspectives than leadership from agencies that did not respond. We did not interview all nonadopters that DBHIDS identified. Reflecting the rapid shift in the Philadelphia treatment landscape, some of the identified nonadopters had begun an MOUD implementation process. We interviewed seven “true” nonadopters and found remarkable consistency among them regarding identified themes. Strengths of the study include the use of qualitative methods to collect nuanced information about leadership and organizational attitudes toward MOUD and the opportunity to evaluate the treatment landscape prior to an MOUD mandate in the City of Philadelphia.

      4.2 Conclusions

      The current study indicates that both resources and ideology thwart the adoption and implementation of MOUD in publicly funded SUD treatment; although, this sample of organizations held more MOUD-capacity than national averages indicate. Given the heterogeneity of organizations, organizations will have to develop tailored implementation strategies and tools to match their structural and ideological profiles. The information provided in the current study has the potential to inform organizations as they expand access to MOUD and to improve the quality of community-based treatment of SUDs.

      Ethics approval and consent to participate

      The Institutional Review Boards of the University of Pennsylvania and the City of Philadelphia approved all study procedures, and all ethical guidelines were followed.

      Availability of data and materials

      Data will be made available upon request.

      Funding

      This work was supported by K23DA048167.

      CRediT author statement

      Rebecca Stewart: conceptualization, methodology, formal analysis, investigation, writing – original draft, supervision Courtney Wolk: methodology, validation, writing-review & editing Geoffrey Neimark: conceptualization, writing–review & editing Ridhi Vyas: formal analysis.
      Jordyn Young: formal analysis, Chris Tjoa: conceptualization, visualization, writing–review & editing, Kyle Kampman, conceptualization, David T. Jones, conceptualization, resources David Mandell: conceptualization, methodology, writing–review & editing.

      Declaration of competing interest

      None declared.

      Acknowledgements

      We are especially grateful for the support that DBHIDS and CBH provided for this project. We thank Jim McKay for his data consultation and Nayoung Kwon for her comments on a prior version of the manuscript. We would also like to thank all of the organizational leaders who participated in the study, making it possible. The authors also thanked the National Institute on Drug Abuse (K23DA048167).

      References

        • Aarons G.A.
        • Hurlburt M.
        • Horwitz S.M.
        Advancing a conceptual model of evidence-based practice implementation in public service sectors.
        Administration and Policy in Mental Health. 2011; 38: 4-23https://doi.org/10.1007/s10488-010-0327-7
        • Abraham A.J.
        • Knudsen H.K.
        • Rieckmann T.
        • Roman P.M.
        Disparities in access to physicians and medications for the treatment of substance use disorders between publicly and privately funded treatment programs in the United States.
        Journal of Studies on Alcohol and Drugs. 2013; 74: 258-265
        • Aletraris L.
        • Edmond M.B.
        • Paino M.
        • Fields D.
        • Roman P.M.
        Counselor training and attitudes toward pharmacotherapies for opioid use disorder.
        Substance Abuse. 2016; 37: 47-53https://doi.org/10.1080/08897077.2015.1062457
        • Barnett M.L.
        • Gray J.
        • Zink A.
        • Jena A.B.
        Coupling policymaking with evaluation — The case of the opioid crisis.
        New England Journal of Medicine. 2017; 377: 2306-2309https://doi.org/10.1056/NEJMp1710014
        • Berk J.
        To help providers fight the opioid epidemic, “X The X Waiver” | Health Affairs.
        • Bisaga A.
        • Mannelli P.
        • Yu M.
        • Nangia N.
        • Graham C.E.
        • Tompkins D.A.
        • Sullivan M.A.
        Outpatient transition to extended-release injectable naltrexone for patients with opioid use disorder: A phase 3 randomized trial.
        Drug and Alcohol Dependence. 2018; 187: 171-178https://doi.org/10.1016/j.drugalcdep.2018.02.023
        • Blendon R.J.
        • Benson J.M.
        The public and the opioid-abuse epidemic.
        New England Journal of Medicine. 2018; 378: 407-411https://doi.org/10.1056/NEJMp1714529
        • Borgida E.
        • Nisbett R.E.
        The differential impact of abstract vs. concrete information on decision.
        Journal of Applied Social Psychology. 1977; 7: 258-271
        • Bradley E.H.
        • Curry L.A.
        • Devers K.J.
        Qualitative data analysis for health services research: Developing taxonomy, themes, and theory.
        Health Services Research. 2007; 42: 1758-1772https://doi.org/10.1111/j.1475-6773.2006.00684.x
        • Comer S.D.
        • Sullivan M.A.
        • Yu E.
        • Rothenberg J.L.
        • Kleber H.D.
        • Kampman K.
        • O’Brien C.P.
        Injectable, sustained-release naltrexone for the treatment of opioid dependence: A randomized, placebo-controlled trial.
        Archives of General Psychiatry. 2006; 63: 210-218https://doi.org/10.1001/archpsyc.63.2.210
        • Fiscella K.
        • Wakeman S.E.
        • Beletsky L.
        Buprenorphine deregulation and mainstreaming treatment for opioid use disorder: X the X waiver.
        JAMA Psychiatry. 2019; 76: 229-230https://doi.org/10.1001/jamapsychiatry.2018.3685
        • Friedmann P.D.
        • Jiang L.
        • Alexander J.A.
        Top manager effects on buprenorphine adoption in outpatient substance abuse treatment programs.
        The Journal of Behavioral Health Services & Research. 2010; 37: 322-337https://doi.org/10.1007/s11414-009-9169-z
        • Galanter M.
        Combining medically assisted treatment and twelve-step programming: A perspective and review.
        The American Journal of Drug and Alcohol Abuse. 2018; 44: 151-159https://doi.org/10.1080/00952990.2017.1306747
        • Garnick D.W.
        • Horgan C.M.
        • Acevedo A.
        • McCorry F.
        • Weisner C.
        Performance measures for substance use disorders – What research is needed?.
        Addiction Science & Clinical Practice. 2012; 7: 18https://doi.org/10.1186/1940-0640-7-18
        • Gladden R.M.
        • O’Donnell J.
        • Mattson C.L.
        • Seth P.
        Changes in opioid-involved overdose deaths by opioid type and presence of benzodiazepines, cocaine, and methamphetamine — 25 states, July–December 2017 to January–June 2018.
        Morbidity and Mortality Weekly Report. 2019; 68: 737-744https://doi.org/10.15585/mmwr.mm6834a2
        • Glaser B.
        • Strauss A.
        The discovery of grounded theory: Strategies for qualitative research.
        1999 (Aldine Transaction)
        • Heley K.
        • Kennedy-Hendricks A.
        • Niederdeppe J.
        • Barry C.L.
        Reducing health-related stigma through narrative messages.
        Health Commun. 2020; 35: 849-860https://doi.org/10.1080/10410236.2019.1598614
        • Hodgkin D.
        • Garnick D.W.
        • Horgan C.M.
        • Busch A.B.
        • Stewart M.T.
        • Reif S.
        Is it feasible to pay specialty substance use disorder treatment programs based on patient outcomes?.
        Drug and Alcohol Dependence. 2020; 206: 107735https://doi.org/10.1016/j.drugalcdep.2019.107735
        • Kennedy-Hendricks A.
        • Levin J.
        • Stone E.
        • McGinty E.E.
        • Gollust S.E.
        • Barry C.L.
        News media reporting on medication treatment for opioid use disorder amid the opioid epidemic.
        Health Affairs. 2019; 38: 643-651https://doi.org/10.1377/hlthaff.2018.05075
        • Knudsen H.K.
        • Abraham A.J.
        • Oser C.B.
        Barriers to the implementation of medication-assisted treatment for substance use disorders: The importance of funding policies and medical infrastructure.
        Evaluation and Program Planning. 2011; 34: 375-381https://doi.org/10.1016/j.evalprogplan.2011.02.004
        • Knudsen H.K.
        • Ducharme L.J.
        • Roman P.M.
        Research network involvement and addiction treatment center staff: Counselor attitudes toward buprenorphine.
        The American Journal on Addictions. 2007; 16: 365-371https://doi.org/10.1080/10550490701525418
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174https://doi.org/10.2307/2529310
        • Madden E.F.
        Intervention stigma: How medication-assisted treatment marginalizes patients and providers.
        Social Science & Medicine. 2019; 232: 324-331https://doi.org/10.1016/j.socscimed.2019.05.027
        • Mattick R.P.
        • Breen C.
        • Kimber J.
        • Davoli M.
        Buprenorphine maintenance versus placebo or methadone maintenance for opioid dependence.
        The Cochrane Database of Systematic Reviews. 2014; 2: CD002207https://doi.org/10.1002/14651858.CD002207.pub4
        • McGuire A.B.
        • Powell K.G.
        • Treitler P.C.
        • Wagner K.D.
        • Smith K.P.
        • Cooperman N.
        • Watson D.P.
        Emergency department-based peer support for opioid use disorder: Emergent functions and forms.
        Journal of Substance Abuse Treatment. 2020; 108: 82-87https://doi.org/10.1016/j.jsat.2019.06.013
        • Mojtabai R.
        • Mauro C.
        • Wall M.M.
        • Barry C.L.
        • Olfson M.
        Medication treatment for opioid use disorders in substance use treatment facilities.
        Health Affairs. 2019; 38: 14-23https://doi.org/10.1377/hlthaff.2018.05162
        • Molfenter T.
        • Knudsen H.K.
        • Brown R.
        • Jacobson N.
        • Horst J.
        • Van Etten M.
        • Madden L.
        Test of a workforce development intervention to expand opioid use disorder treatment pharmacotherapy prescribers: Protocol for a cluster randomized trial.
        Implementation Science. 2017; 12: 135https://doi.org/10.1186/s13012-017-0665-x
        • Mooney L.J.
        • Valdez J.
        • Cousins S.J.
        • Yoo C.
        • Zhu Y.
        • Hser Y.-I.
        Patient decision aid for medication treatment for opioid use disorder (PtDA-MOUD): Rationale, methodology, and preliminary results.
        Journal of Substance Abuse Treatment. 2020; 108: 115-122https://doi.org/10.1016/j.jsat.2019.08.006
        • Olsen Y.
        • Sharfstein J.M.
        Confronting the stigma of opioid use disorder--and its treatment.
        JAMA. 2014; 311: 1393-1394https://doi.org/10.1001/jama.2014.2147
        • Purtle J.
        • Lê-Scherban F.
        • Wang X.
        • Shattuck P.T.
        • Proctor E.K.
        • Brownson R.C.
        Audience segmentation to disseminate behavioral health evidence to legislators: An empirical clustering analysis.
        Implementation Science. 2018; 13: 121https://doi.org/10.1186/s13012-018-0816-8
        • Quinn A.E.
        • Hodgkin D.
        • Perloff J.N.
        • Stewart M.T.
        • Brolin M.
        • Lane N.
        • Horgan C.M.
        Design and impact of bundled payment for detox and follow-up care.
        Journal of Substance Abuse Treatment. 2017; 82: 113-121https://doi.org/10.1016/j.jsat.2017.09.012
        • Scott C.K.
        • Grella C.E.
        • Nicholson L.
        • Dennis M.L.
        Opioid recovery initiation: Pilot test of a peer outreach and modified recovery management checkup intervention for out-of-treatment opioid users.
        Journal of Substance Abuse Treatment. 2018; 86: 30-35https://doi.org/10.1016/j.jsat.2017.12.007
        • Sherif C.W.
        • Kelly M.
        • Rodgers Jr., H.L.
        • Sarup G.
        • Tittler B.I.
        Personal involvement, social judgment, and action.
        Journal of Personality and Social Psychology. 1973; 27: 311-328https://doi.org/10.1037/h0034948
        • Sordo L.
        • Barrio G.
        • Bravo M.J.
        • Indave B.I.
        • Degenhardt L.
        • Wiessing L.
        • Pastor-Barriuso R.
        Mortality risk during and after opioid substitution treatment: Systematic review and meta-analysis of cohort studies.
        BMJ. 2017; 357https://doi.org/10.1136/bmj.j1550
        • Stewart R.E.
        • Beidas R.S.
        • Mandell D.S.
        Stop calling them laggards: Strategies for encouraging nonadopters to incorporate evidence-based practices.
        Psychiatric Services (Washington, D.C.). 2019; (appips201900031)https://doi.org/10.1176/appi.ps.201900031
        • Stewart R.E.
        • Chambless D.L.
        Interesting practitioners in training in empirically supported treatments: Research reviews versus case studies.
        Journal of Clinical Psychology. 2010; 66: 73-95https://doi.org/10.1002/jclp.20630
        • Stewart R.E.
        • Lareef I.
        • Hadley T.R.
        • Mandell D.S.
        Can we pay for performance in behavioral health care?.
        Psychiatric Services. 2017; 68: 109-111https://doi.org/10.1176/appi.ps.201600475
        • Stewart R.E.
        • Marcus S.C.
        • Hadley T.R.
        • Hepburn B.M.
        • Mandell D.S.
        State adoption of incentives to promote evidence-based practices in behavioral health systems.
        Psychiatric Services. 2018; 69: 685-688https://doi.org/10.1176/appi.ps.201700508
        • Ubel P.A.
        • Jepson C.
        • Baron J.
        The inclusion of patient testimonials in decision aids: Effects on treatment choices.
        Medical Decision Making: An International Journal of the Society for Medical Decision Making. 2001; 21: 60-68
        • Wakeman S.E.
        Medications for addiction treatment: Changing language to improve care.
        Journal of Addiction Medicine. 2017; 11: 1-2https://doi.org/10.1097/ADM.0000000000000275