Advertisement

A randomized clinical trial evaluating the impact of counselor training and patient feedback on substance use disorder patients' sexual risk behavior

      Highlights

      • We examined counselor and patient interventions as delivered in real world settings.
      • Skills training helps substance use counselors talk to patients about sex risk.
      • Patients getting personalized feedback are likelier to discuss sex risk in counseling.
      • Neither patient feedback nor counselor training reduced risky sex at follow-up.
      • Emphasizing external validity may have weakened the intervention’s effect on behavior.

      Abstract

      Introduction

      High risk sex—such as sex with multiple partners, condomless sex, or transactional or commercial sex—is a risk factor in individuals with substance use disorders (SUDs). SUD treatment can reduce sexual risk behavior, but interventions to reduce such behavior in this context have not been consistently effective. This study sought to determine if the impact of treatment on sexual risk behavior can be increased.

      Methods

      In a nested 2 × 2 factorial repeated measures design, we examined outcomes of two interventions: training for counselors in talking to patients about sexual risk; and availability to both counselors and patients of a personalized feedback report based on patient self-report of sexual behavior. Counselors received either a brief, information-based, Basic Training, or a multi-session, skills-based Enhanced Training. Their patients completed an audio-assisted computerized assessment of sexual behavior and received either No Feedback or a Personalized Feedback Report (PFR). Four hundred seventy six patients participated. Patient follow-up occurred 3- and 6-months postbaseline. Primary patient outcome measures were Number of Unsafe Sex Occasions (USO) and whether patients reported talking about sex in counseling sessions (Discussed Sex), both in the past 90 days. Secondary outcomes included Number of Sexual Partners, Sex Under the Influence of Substances, and Perceived Condom Barriers.

      Results

      Patients of Enhanced-condition counselors compared to those of Basic-condition counselors were more likely to report talking about sex with their counselor at 6-month follow-up. Personalized feedback also increased the likelihood of reporting counselor discussions at 6-month follow-up. Neither the training nor the feedback condition affected USO, Number of Partners, or Sex Under the Influence.

      Discussion

      We discuss why these two interventions apparently altered counselor-patient communication about sexual risk behavior without affecting the behavior itself.

      Keywords

      1. Introduction

      Sexual transmission of HIV and other sexually transmitted infections (STIs) is of grave concern among people who use substances (
      Centers for Disease Control and Prevention
      Sexually transmitted diseases treatment guidelines, 2015.
      ,
      • Centers for Disease Control and Prevention
      CDC fact sheet: Today's HIV/AIDS epidemic.
      ,
      • Centers for Disease Control and Prevention
      Statistics overview.
      ). Sex, drugs, and alcohol mutually influence each other and amplify risks for relapse and disease transmission (
      • Fairbairn N.
      • Hayashi K.
      • Milloy M.
      • Nolan S.
      • Nguyen P.
      • Wood E.
      • Kerr T.
      Hazardous alcohol use associated with increased sexual risk behaviors among people who inject drugs.
      ;
      • Kral A.H.
      • Bluthenthal R.N.
      • Lorvick J.
      • Gee L.
      • Bacchetti P.
      • Edlin B.R.
      Sexual transmission of HIV-1 among injection drug users in San Francisco, USA: Risk-factor analysis.
      ;
      • Shoptaw S.
      • Montgomery B.
      • Williams C.T.
      • El-Bassel N.
      • Aramrattana A.
      • Metsch L.
      • Metzger D.S.
      • Kuo I.
      • Bastos F.I.
      • Strathdee S.A.
      Not just the needle: The state of HIV-prevention science among substance users and future directions.
      ;
      • Volkow N.D.
      • Montaner J.
      The urgency of providing comprehensive and integrated treatment for substance abusers with HIV.
      ). Unfortunately, condom use is low among those in SUD treatment. In two multisite companion HIV prevention trials of people in SUD treatment (
      • Calsyn D.A.
      • Hatch-Maillette M.
      • Tross S.
      • Doyle S.R.
      • Crits-Christoph P.
      • Song Y.S.
      • Harrer J.M.
      • Lalos G.
      • Berns S.B.
      Motivational and skills training HIV/sexually transmitted infection sexual risk reduction groups for men.
      ;
      • Tross S.
      • Campbell A.N.C.
      • Cohen L.R.
      • Calsyn D.
      • Pavlicova M.
      • Miele G.
      • Hu M.
      • Haynes L.
      • Nugent N.
      • Gan W.
      • Hatch-Maillette M.
      • Mandler R.
      • McLaughlin P.
      • El-Bassel N.
      • Crits-Christoph P.
      • Nunes E.V.
      Effectiveness of HIV/STD sexual risk reduction groups for women in substance abuse treatment programs: Results of a NIDA Clinical Trials Network Trial.
      ), 58 % of men and 53 % of women reported no condom use with casual partners (
      • Calsyn D.A.
      • Peavy K.M.
      • Wells E.A.
      • Campbell A.N.C.
      • Hatch-Maillette M.A.
      • Greenfield S.F.
      • Tross S.
      Differences between men and women in condom use, attitudes, and skills in substance abuse treatment seekers.
      ). In a study of women in methadone maintenance treatment, more than two-thirds reported inconsistent condom use during vaginal sex (
      • Engstrom M.
      • Shibusawa T.
      • El-Bassel N.
      • Gilbert L.
      Age and HIV sexual risk among women in methadone treatment.
      ).

      1.1 Entry into SUD treatment presents an opportunity to intervene in risky sexual behavior

      SUD treatment presents an ideal time to intervene in HIV and STI risk behaviors, because it inherently focuses on overall risk behavior. For example, medication for opioid use disorder treatment (MOUD) reduces injection and sexual risk (
      • Broome K.M.
      • Joe G.W.
      • Simpson D.D.
      HIV risk reduction in outpatient drug abuse treatment: Individual and geographic differences.
      ;
      • Gibson D.R.
      • Flynn N.M.
      • McCarthy J.J.
      Effectiveness of methadone treatment in reducing HIV risk behavior and seroconversion among injecting drug users.
      ;
      • Gottheil E.
      • Lundy A.
      • Weinstein S.P.
      • Sterling R.C.
      Does intensive outpatient cocaine treatment reduce AIDS risky behaviors?.
      ;
      • Kidorf M.
      • Brooner R.K.
      • Yan H.
      • Peirce J.
      Sexual-risk reduction following the referral of syringe exchange registrants to methadone maintenance: Impact of gender and drug use.
      ;
      • Latka M.H.
      • Wilson T.E.
      • Cook J.A.
      • Bacon M.C.
      • Richardson J.L.
      • Sohler N.
      • Cohen M.H.
      • Greenblatt R.M.
      • Andreopoulis E.
      • Vlahov D.
      Impact of treatment on subsequent sexual risk behavior in a multisite cohort of drug using women: A report from the women's interagency HIV study.
      ;
      • Metzger D.S.
      • Woody G.E.
      • O'Brien C.P.
      Drug treatment as HIV prevention: A research update.
      ;
      • Springer S.A.
      • Larney S.
      • Alam-mehrjerdi Z.
      • Altice F.L.
      • Metzger D.
      • Shoptaw S.
      Drug treatment as HIV prevention among women and girls who inject drugs from a global perspective: Progress, gaps, and future directions.
      ;
      • Wells E.A.
      • Clark L.L.
      • Calsyn D.A.
      • Saxon A.J.
      • Jackson T.R.
      • Wrede A.F.
      Reporting of HIV risk behaviors by injection drug using heterosexual couples in methadone maintenance.
      ). However, in adults who inject drugs, some studies have found sexual risk behavior less amenable to change during treatment than high-risk injection practices (
      • Hoffman J.A.
      • Klein H.
      • Clark D.C.
      • Boyd F.T.
      The effect of entering drug treatment on involvement in HIV-related risk behaviors. (1998).
      ;
      • Latka M.H.
      • Wilson T.E.
      • Cook J.A.
      • Bacon M.C.
      • Richardson J.L.
      • Sohler N.
      • Cohen M.H.
      • Greenblatt R.M.
      • Andreopoulis E.
      • Vlahov D.
      Impact of treatment on subsequent sexual risk behavior in a multisite cohort of drug using women: A report from the women's interagency HIV study.
      ). In a longitudinal analysis of 1393 patients enrolled in the Drug Abuse Treatment Outcome (DATOS) studies that included those injecting and those not injecting, the majority were engaged in high-risk sex at intake and showed reduction at 1-year follow-up (
      • Murphy D.A.
      • Brecht M.-L.
      • Herbeck D.
      • Evans E.
      • Huang D.
      • Hser Y.-I.
      Longitudinal HIV risk behavior among the Drug Abuse Treatment Outcome Studies (DATOS) adult sample.
      ). By 5-year follow-up, two-thirds had returned to baseline or escalated risky sex behavior. Thus, although SUD treatment can reduce risk behavior, risky sex often remains problematic.

      1.2 Brief STI/HIV prevention interventions are effective

      Meta-analyses of behavioral interventions for STI/HIV risk (
      • Albarracín D.
      • Gillette J.C.
      • Earl A.N.
      • Glasman L.R.
      • Durantini M.R.
      • Ho M.-H.
      A test of major assumptions about behavior change: A comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic.
      ;
      • Covey J.
      • Rosenthal-Stott H.E.S.
      • Howell S.J.
      A synthesis of meta-analytic evidence of behavioral interventions to reduce HIV/STIs.
      ;
      • Weissman J.
      • Kanamori M.
      • Devieux J.G.
      • Trepka M.J.
      • De La Rosa M.
      HIV risk reduction interventions among substance-abusing reproductive-age women: A systematic review.
      ) conclude the most effective programs are based on theoretical models addressing attitudes, education, and skills.
      • Covey J.
      • Rosenthal-Stott H.E.S.
      • Howell S.J.
      A synthesis of meta-analytic evidence of behavioral interventions to reduce HIV/STIs.
      reviewed 37 meta-analyses examining effectiveness of STI/HIV prevention interventions and determined that duration and number of sessions were not associated with intervention efficacy.
      • Meader N.
      • Li R.
      • Des Jarlais D.C.
      • Pilling S.
      Psychosocial interventions for reducing injection and sexual risk behavior for preventing HIV in drug users.
      reported consistent findings and further concluded that brief standard education interventions can be a cost-effective and resource-efficient alternative to intensive behavioral interventions.
      Brief motivational interventions employing personalized feedback based on risk assessment have been tested for a variety of health risk behaviors (
      • Frost H.
      • Campbell P.
      • Maxwell M.
      • O’Carroll R.E.
      • Dombrowski S.U.
      • Williams B.
      • Cheyne H.
      • Coles E.
      • Pollock A.
      Effectiveness of motivational interviewing on adult behaviour change in health and social care settings: A systematic review of reviews.
      ;
      • Lundahl B.W.
      • Kunz C.
      • Brownell C.
      • Tollefson D.
      • Burke B.L.
      A meta-analysis of motivational interviewing: Twenty-five years of empirical studies.
      ). Yet few clinical trials providing personalized feedback about sexual risk behavior to adults who use substances have been reported. The target populations in this small number of trials on sexual risk vary and include heavy-drinking college students (
      • Lewis M.A.
      • Patrick M.E.
      • Litt D.M.
      • Atkins D.C.
      • Kim T.
      • Blayney J.A.
      • Norris J.
      • George W.H.
      • Larimer M.E.
      Randomized controlled trial of a web-delivered personalized normative feedback intervention to reduce alcohol-related risky sexual behavior among college students.
      ), homeless young adults (
      • Tucker J.S.
      • D'Amico E.J.
      • Ewing B.A.
      • Miles J.N.V.
      • Pedersen E.R.
      A group-based motivational interviewing brief intervention to reduce substance use and sexual risk behavior among homeless young adults.
      ), patients receiving alcohol detoxification (
      • Brems C.
      • Dewane S.L.
      • Johnson M.E.
      • Eldridge G.D.
      Brief motivational interventions for HIV/STI risk reduction among individuals receiving alcohol detoxification.
      ), and PWID (
      • Tucker T.
      • Fry C.L.
      • Lintzeris N.
      • Baldwin S.
      • Ritter A.
      • Donath S.
      • Whelan G.
      Randomized controlled trial of a brief behavioural intervention for reducing hepatitis C virus risk practices among injecting drug users.
      ), with some studies finding reductions in risk (
      • Lewis M.A.
      • Patrick M.E.
      • Litt D.M.
      • Atkins D.C.
      • Kim T.
      • Blayney J.A.
      • Norris J.
      • George W.H.
      • Larimer M.E.
      Randomized controlled trial of a web-delivered personalized normative feedback intervention to reduce alcohol-related risky sexual behavior among college students.
      ), some finding no effect (
      • Brems C.
      • Dewane S.L.
      • Johnson M.E.
      • Eldridge G.D.
      Brief motivational interventions for HIV/STI risk reduction among individuals receiving alcohol detoxification.
      ;
      • Tucker T.
      • Fry C.L.
      • Lintzeris N.
      • Baldwin S.
      • Ritter A.
      • Donath S.
      • Whelan G.
      Randomized controlled trial of a brief behavioural intervention for reducing hepatitis C virus risk practices among injecting drug users.
      ;
      • Tucker J.S.
      • D'Amico E.J.
      • Ewing B.A.
      • Miles J.N.V.
      • Pedersen E.R.
      A group-based motivational interviewing brief intervention to reduce substance use and sexual risk behavior among homeless young adults.
      ), and others reporting mixed results (
      • Tucker T.
      • Fry C.L.
      • Lintzeris N.
      • Baldwin S.
      • Ritter A.
      • Donath S.
      • Whelan G.
      Randomized controlled trial of a brief behavioural intervention for reducing hepatitis C virus risk practices among injecting drug users.
      ;
      • Tucker J.S.
      • D'Amico E.J.
      • Ewing B.A.
      • Miles J.N.V.
      • Pedersen E.R.
      A group-based motivational interviewing brief intervention to reduce substance use and sexual risk behavior among homeless young adults.
      ). The paucity of studies in this area, specifically with patients in SUD treatment, suggests that we need to further test brief personalized feedback to address sexual risk in this population.

      1.3 SUD treatment counselors are ideal interventionists but lack comfort and skills

      SUD counselors are ideally placed to address risky sex practices with patients, due to their established relationship, regular contact, and understanding of a patient's circumstances (
      • Brown N.K.
      Clinical judgments of high-risk behavior during recovery.
      ). Yet many SUD treatment providers do not feel competent or comfortable assessing and discussing sexual risk with patients (
      • Haynes L.
      • Calsyn D.
      • Tross S.
      Addressing sexual issues in addictions treatment.
      ).
      • Mitchell C.G.
      • Oltean A.
      Integrating HIV prevention into substance user treatment: Current practices and challenges.
      found that one-third of counselors did not feel competent to carry out an HIV sexual risk assessment, less than one-third believed their patients were open to discussions about HIV risk behaviors, and almost half endorsed feeling uncomfortable working with patients who engage in behaviors that put others at risk. Moreover, 43 % of SUD treatment programs in the United States do not provide HIV education or counseling (
      • Satterwhite C.L.
      • Torrone E.
      • Meites E.
      • Dunne E.F.
      • Mahajan R.
      • Ocfemia M.C.B.
      • Su J.
      • Xu F.
      • Weinstock H.
      Sexually transmitted infections among US women and men.
      ).

      1.4 Providers can increase comfort and skills, and follow-up is needed to maintain these gains

      Training can improve clinicians' comfort and skill in discussing sexual issues. However, existing research has primarily studied medical providers in non-SUD treatment settings. Provider outcomes in these studies included more favorable attitudes toward risk assessment and intervention (
      • Bluespruce J.
      • Dodge W.T.
      • Grothhaus L.
      • Wheeler K.
      • Rebolledo V.
      • Carey J.W.
      • McAfee T.A.
      • Thompson R.S.
      HIV prevention in primary care: Impact of a clinical intervention.
      ;
      • Bradley-Springer L.A.
      • Everett M.R.
      • Rotach E.G.
      • Vojir C.P.
      Changes in clinician ability to assess risk and help patients determine the need for HIV testing: A comparison of three teaching methods.
      ;
      • Thrun M.
      • Cook P.F.
      • Bradley-Springer L.A.
      • Gardner L.
      • Marks G.
      • Wright J.
      • Wilson T.E.
      • Quinlivan E.B.
      • O'Daniels C.
      • Raffanti S.
      • Thompson M.
      • Golin C.
      Improved prevention counseling by HIV care providers in a multi-site clinic-based intervention: Positive steps.
      ), greater confidence in one's own ability to effectively carry out prevention strategies (
      • Bluespruce J.
      • Dodge W.T.
      • Grothhaus L.
      • Wheeler K.
      • Rebolledo V.
      • Carey J.W.
      • McAfee T.A.
      • Thompson R.S.
      HIV prevention in primary care: Impact of a clinical intervention.
      ;
      • Dreisbach S.
      • Burnside H.
      • Hsu K.
      • Smock L.
      • Coury-Doniger P.
      • Hall C.
      • Marrazzo J.
      • Nagendra G.
      • Rietmeijer C.
      • Rompalo A.
      • Thrun M.
      Improving HIV/STD prevention in the care of persons living with HIV through a national training program.
      ;
      • Thrun M.
      • Cook P.F.
      • Bradley-Springer L.A.
      • Gardner L.
      • Marks G.
      • Wright J.
      • Wilson T.E.
      • Quinlivan E.B.
      • O'Daniels C.
      • Raffanti S.
      • Thompson M.
      • Golin C.
      Improved prevention counseling by HIV care providers in a multi-site clinic-based intervention: Positive steps.
      ), greater comfort delivering interventions (
      • Thrun M.
      • Cook P.F.
      • Bradley-Springer L.A.
      • Gardner L.
      • Marks G.
      • Wright J.
      • Wilson T.E.
      • Quinlivan E.B.
      • O'Daniels C.
      • Raffanti S.
      • Thompson M.
      • Golin C.
      Improved prevention counseling by HIV care providers in a multi-site clinic-based intervention: Positive steps.
      ), and greater frequency delivering prevention interventions (
      • Bradley-Springer L.A.
      • Everett M.R.
      • Rotach E.G.
      • Vojir C.P.
      Changes in clinician ability to assess risk and help patients determine the need for HIV testing: A comparison of three teaching methods.
      ;
      • Dreisbach S.
      • Burnside H.
      • Hsu K.
      • Smock L.
      • Coury-Doniger P.
      • Hall C.
      • Marrazzo J.
      • Nagendra G.
      • Rietmeijer C.
      • Rompalo A.
      • Thrun M.
      Improving HIV/STD prevention in the care of persons living with HIV through a national training program.
      ;
      • Thrun M.
      • Cook P.F.
      • Bradley-Springer L.A.
      • Gardner L.
      • Marks G.
      • Wright J.
      • Wilson T.E.
      • Quinlivan E.B.
      • O'Daniels C.
      • Raffanti S.
      • Thompson M.
      • Golin C.
      Improved prevention counseling by HIV care providers in a multi-site clinic-based intervention: Positive steps.
      ). Patient-reported outcomes included increases in discussions about sex (
      • Dodge W.T.
      • BlueSpruce J.
      • Grothaus L.
      • Rebolledo V.
      • McAfee T.A.
      • Carey J.W.
      • Thompson R.S.
      Enhancing primary care HIV prevention: A comprehensive clinical intervention.
      ;
      • Rose C.D.
      • Courtenay-Quirk C.
      • Knight K.
      • Shade S.B.
      • Vittinghoff E.
      • Gomez C.
      • Lum P.J.
      • Bacon O.
      • Colfax G.
      HIV intervention for providers study: A randomized controlled trial of a clinician-delivered HIV risk-reduction intervention for HIV-positive people.
      ), receipt of prevention counseling (
      • Patel S.N.
      • Marks G.
      • Gardner L.
      • Golin C.E.
      • Shinde S.
      • O'Daniels C.
      • Wilson T.E.
      • Quinlivan E.B.
      • Banderas J.W.
      Brief training of HIV medical providers increases their frequency of delivering prevention counseling to patients at risk of transmitting HIV to others.
      ;
      • Rose C.D.
      • Courtenay-Quirk C.
      • Knight K.
      • Shade S.B.
      • Vittinghoff E.
      • Gomez C.
      • Lum P.J.
      • Bacon O.
      • Colfax G.
      HIV intervention for providers study: A randomized controlled trial of a clinician-delivered HIV risk-reduction intervention for HIV-positive people.
      ;
      • Thrun M.
      • Cook P.F.
      • Bradley-Springer L.A.
      • Gardner L.
      • Marks G.
      • Wright J.
      • Wilson T.E.
      • Quinlivan E.B.
      • O'Daniels C.
      • Raffanti S.
      • Thompson M.
      • Golin C.
      Improved prevention counseling by HIV care providers in a multi-site clinic-based intervention: Positive steps.
      ), and reductions in unsafe behavior (
      • Rose C.D.
      • Courtenay-Quirk C.
      • Knight K.
      • Shade S.B.
      • Vittinghoff E.
      • Gomez C.
      • Lum P.J.
      • Bacon O.
      • Colfax G.
      HIV intervention for providers study: A randomized controlled trial of a clinician-delivered HIV risk-reduction intervention for HIV-positive people.
      ). We do not know whether these findings translate to SUD treatment settings and SUD counselors.
      Regardless of professional discipline, maintaining provider behavior changes after training is challenging (
      • Cohen S.J.
      • Halvorson H.W.
      • Gosselink C.A.
      Changing physician behavior to improve disease prevention.
      ;
      • Gerbert B.
      • Berg-Smith S.
      • Mancuso M.
      • Caspers N.
      • McPhee S.
      • Null D.
      • Wofsy J.
      Using innovative video doctor technology in primary care to deliver brief smoking and alcohol intervention.
      ).
      • Walters S.T.
      • Matson S.A.
      • Baer J.S.
      • Ziedonis D.M.
      Effectiveness of workshop training for psychosocial addiction treatments: A systematic review.
      reviewed workshop training effectiveness for SUD treatment counselors for a range of intervention skills and found additional contact beyond workshops improved skill retention. In a review of training techniques for evidence-based treatments (EBTs),
      • Martino S.
      Strategies for training counselors in evidence-based treatments.
      concluded that clinical supervision (including observation, feedback, and coaching) is likely required for counselor proficiency in implementation of EBTs. Clinicians may need ongoing clinical supervision post-training to increase comfort and skill in discussing sex with patients.
      As part of the larger 2 × 2 factorial design reported herein, we previously reported training outcomes from two levels of SUD counselor training designed to reduce sexual risks among SUD patients (
      • Hatch-Maillette M.A.
      • Harwick R.
      • Baer J.S.
      • Wells E.A.
      • Masters T.
      • Robinson A.
      • Cloud K.
      • Peavy M.
      • Wiest K.
      • Wright L.
      • Dillon K.
      • Beadnell B.
      Increasing substance use disorder counselors' self-efficacy and skills in talking to patients about sex and HIV risk: A randomized training trial.
      ). Counselors receiving Enhanced training showed significant improvements compared to their Basic training counterparts in self-efficacy, use of reflections, and use of decision-making and communication strategies with standardized patients. These improvements were maintained from post-training to 3-month follow-up.

      1.5 Hypotheses

      We tested the effect on patients' sexual behavior of two different interventions, each aimed at increasing the likelihood of counselors and patients discussing, and patients reducing, sexual risks. The two interventions targeted the patient-counselor relationship in different ways: 1) counselor training about sexual risks and 2) a personalized feedback report (PFR), based on patient risk-behavior assessment, available to counselors and patients. The purpose of this study was to test three hypotheses: 1) counselor training + coaching, compared to minimal training, leads to increased discussion of sex with patients and reduced patient sexual risk behavior; 2) availability of personalized feedback to patients and counselors, compared to no feedback, increases counseling discussion of sex and reduces patient sexual risk behavior; and 3) combining training + coaching with feedback increases discussions of sex and reduces patient sexual risk behavior, compared with feedback only or counselor training + coaching only.

      2. Method

      2.1 Overview

      Patient outcomes are from a randomized nested 2 × 2 factorial repeated measures trial (see Fig. 1); we report counselor outcomes elsewhere (
      • Hatch-Maillette M.A.
      • Harwick R.
      • Baer J.S.
      • Wells E.A.
      • Masters T.
      • Robinson A.
      • Cloud K.
      • Peavy M.
      • Wiest K.
      • Wright L.
      • Dillon K.
      • Beadnell B.
      Increasing substance use disorder counselors' self-efficacy and skills in talking to patients about sex and HIV risk: A randomized training trial.
      ). We randomly assigned SUD counselors to receive Basic (2 h) or Enhanced (10 h plus monthly ongoing coaching) training in talking with patients about sex. Volunteer patients of participating counselors completed a web-based survey about their sexual behavior, then were randomly assigned to either not receive or receive a PFR. Counselors of patients receiving PFRs also received a copy. Patients completed assessments at baseline, and 3 and 6 months postbaseline. The paper reports recruitment, assignment, and follow-up numbers in the Consort Diagram (Fig. 2). This study received institutional Human Subjects approval.
      Fig. 2
      Fig. 2Consort diagram for counselor training and patient feedback.
      *Accurate records of numbers contacted about participation not available across sites.

      2.2 Sample

      Counselors and patients came from two MOUD treatment programs and one psychosocial outpatient program in the United States Pacific Northwest. MOUD programs served both patients who injected opiates and those using through another method. Primary drug and mode of use varied in psychosocial treatment patients. Patient recruitment began in May 2016, with the final follow-up completed in December 2017. Patient inclusion criteria were: 1) at least 18 years old; 2) able to speak and understand English; 3) enrolled in treatment within the last 45 days; 4) planning to remain in the local area for the next three months; and 5) assigned to a counselor enrolled in the study. Exclusion criteria were observable gross mental status impairment and not speaking English well enough to understand the informed consent document. Reading and writing English were not required, as study materials could be read to participants, and assessments were audio-assisted. We conducted a power analysis before the study, then re-calculated sample size needs midway once sufficient preliminary information (e.g., intraclass correlations, average cluster sizes) was available to produce more accurate estimates. We determined that the study needed an analysis sample of at least 472 to detect a small to medium effect between any two conditions (Cohen's f = 0.18, odds ratio = 2.20) with power of 0.80 (p < .05, two-tailed tests) in cross-sectional analysis.

      2.3 Patient procedures

      After consenting to participate, patients completed the baseline assessment and received random assignment to no-feedback or feedback conditions. Randomization to the two conditions was in a 1:1 ratio stratified by counselor's training assignment and gender of participant. The investigative team provided 4 large envelopes per site, each representing basic vs. enhanced counselor assignment crossed with patient male/female gender. Within these were sealed envelopes containing no-feedback or feedback assignments. Site research coordinators withdrew and opened one of these from the appropriate larger envelope at the end of each patient's baseline assessment. Because counselors had already been randomized to basic versus enhanced training, patient assignment was to one of four conditions: Basic + No Feedback; Basic + Feedback; Enhanced + No Feedback; or Enhanced + Feedback.
      Patients assigned to the feedback condition received a printed version of their PFR after randomization. The research coordinator did not review the report with the patient but told them their counselor would receive a copy. Patients could discuss the report with their counselor if they wished. The study did not require discussion of the feedback report between counselor and patient. After the 6-month follow-up assessment, no-feedback-condition patients received a PFR. Patients received $30 for baseline and $35 each for 3-month and 6-month assessments.

      2.4 Personalized feedback report (PFR)

      We designed the PFR to be easily understood. Stock photographs and brightly colored graphs enhanced patient interest. Text explanations were brief. The report covered five areas of risk (number of partners, riskiness of partners, riskiness of sexual behaviors, condom/barrier use, and sex under the influence of substances), comparing the patient's self-reported behavior to data from general population samples or other SUD treatment samples. The report also suggested ways to reduce risk.

      2.5 Counselor training

      2.5.1 Basic training

      All enrolled counselors received basic training and, immediately afterward, assignment to either basic training (no additional training or coaching) or enhanced training (additional 8-hour workshop, divided into 4 sessions, plus monthly coaching).
      Basic training was a single, two-hour session; trainers didactically presented study design and rationale for addressing sex in SUD treatment and introduced the PFR. They reminded counselors that sex is a personal and potentially sensitive topic. Trainers encouraged questions and discussion but offered no skill-training. Trainers oriented counselors to each PFR section but avoided teaching them how to present feedback to patients, instead emphasizing use of basic counseling skills.

      2.5.2 Enhanced training

      Four additional two-hour sessions occurred at weekly intervals. Session one, Discussing Sexuality-Related Issues with Patients: Why and How, trained counselors to provide specific information (e.g., HIV prevention strategies, substance use–risky sex link). This module drew from established work on teaching health care providers to talk to patients about sexual health (
      • Potter J.
      Talking to patients about sex curriculum.
      ) and presented the PLISSIT model (Permission, Limited Information, Specific Suggestions, Intensive Therapy;
      • Annon J.S.
      The PLISSIT model: A proposed conceptual scheme for the behavioral treatment of sexual problems.
      ) to determine the appropriate level of intervention. Counselors learned about associations between sexual behavior, substance use, relationships, and relapse, and skills to teach patients safer sex (condom use). The session also presented the possible impact of sexual diversity (e.g., sexual orientation, gender identity, partner choices) on discussions with patients. Session Two, Using Motivational Interviewing to Inform Conversations about Sexual Risk in SUD Treatment, reviewed the definition of MI and techniques for providing feedback using a PFR (
      • Drapkin M.L.
      • Wilbourne P.
      • Manuel J.K.
      • Baer J.
      • Karlin B.
      • Raffa S.
      National dissemination of Motivation Enhancement Therapy in the Veterans Health Administration: Training program design and initial outcomes.
      ) and gave counselors practice in MI skills by role-playing reviewing a PFR with a patient. Session Three, SODAS: Teaching your Patients Decision-Making Skills for Sexual Situations, taught a problem-solving technique (SODAS; Stop, Options, Decide, Action, Self-Praise) previously used in effective sexual risk reduction interventions in SUD treatment (e.g.,
      • Tross S.
      • Campbell A.N.C.
      • Cohen L.R.
      • Calsyn D.
      • Pavlicova M.
      • Miele G.
      • Hu M.
      • Haynes L.
      • Nugent N.
      • Gan W.
      • Hatch-Maillette M.
      • Mandler R.
      • McLaughlin P.
      • El-Bassel N.
      • Crits-Christoph P.
      • Nunes E.V.
      Effectiveness of HIV/STD sexual risk reduction groups for women in substance abuse treatment programs: Results of a NIDA Clinical Trials Network Trial.
      ). Session four, Talk Tools: Teaching your Patients Communication Skills for Sexual Situations, taught a communication technique, also used in previous effective sex-risk reduction interventions for SUD patients, to facilitate discussions of safer sexual practices (
      • Calsyn D.A.
      • Hatch-Maillette M.
      • Tross S.
      • Doyle S.R.
      • Crits-Christoph P.
      • Song Y.S.
      • Harrer J.M.
      • Lalos G.
      • Berns S.B.
      Motivational and skills training HIV/sexually transmitted infection sexual risk reduction groups for men.
      ;
      • Calsyn D.A.
      • Burlew A.K.
      • Hatch-Maillette M.A.
      • Beadnell B.
      • Wright L.
      • Wilson J.
      An HIV prevention intervention for ethnically diverse men in substance abuse treatment: Pilot study findings.
      ).

      2.6 Coaching

      Once patient recruitment began, counselors in the enhanced group received group coaching every other week for the first three months in which their patients were recruited and monthly thereafter until all patients recruited at the site had completed 6-month follow-up interviews. Counselor coaching groups usually met face-to-face for 30–60 min, though phone/video was an acceptable substitute. Coaches used a checklist for session structure: acknowledgment for participating; a “check-in” for talking with patients about sex; using study PFRs, SODAS, or TALK communications tools; and commitment to an action before the next meeting.

      2.7 Measures

      Patients provided self-reports via audio-computer-assisted-self-interview (ACASI). Patients completed the same set of assessments at baseline, 3, and 6 months. Assessments required approximately 1 to 1.5 h to complete.
      Participants answered demographic questions about age, ethnicity, race, education level, sex at birth, gender identity, and to whom they are sexually and romantically attracted. To help determine which sexual behavior questions they would see (and because transgender individuals may or may not have had sex reassignment surgery), participants responded to a question about having a penis or a vagina.
      We developed two primary outcomes among patients in this study, both computed from the Sexual Risk Assessment Questionnaire, adapted from Calysn et al. (2009) and
      • Tross S.
      • Campbell A.N.C.
      • Cohen L.R.
      • Calsyn D.
      • Pavlicova M.
      • Miele G.
      • Hu M.
      • Haynes L.
      • Nugent N.
      • Gan W.
      • Hatch-Maillette M.
      • Mandler R.
      • McLaughlin P.
      • El-Bassel N.
      • Crits-Christoph P.
      • Nunes E.V.
      Effectiveness of HIV/STD sexual risk reduction groups for women in substance abuse treatment programs: Results of a NIDA Clinical Trials Network Trial.
      . At 3- and 6-month assessments, patients responded to the question, “During the past 3 months, how many times did your counselor(s) discuss sexual issues with you during individual or group therapy sessions”? This count variable was extremely zero-inflated, making interpretation of mean values difficult. We chose transformation using dichotomous coding (zero = no times; 1 = 1 or more times) to derive one primary outcome measure, discussed sex in counseling.
      Participants could endorse having had a main male partner, a main female partner, other male partners (including how many), and other female partners (including how many). Questions about their behavior with each of these categories of partners followed and covered time periods of either the past 90 days or the past 12 months. Questions that we used in analyses reported here were (1) Frequency of vaginal and anal sex (insertive and receptive, as appropriate to participant's reported anatomy), and (2) Frequency of condom use during this vaginal or anal sex. These questions formed the basis of a second primary outcome, number of unsafe sex occasions (USO). To compute this variable, we first summed vaginal and anal sex occasions across all partner types. Next, we summed condom use occasions across all partner types, and subtracted the protected sex count from the total sex count. For women who had sex with both male and female partners, we used sex with male partners as their unsafe sex indicator. We coded women who had only female partners as having zero USO. Because USO may not adequately distinguish level of risk between monogamous and non-monogamous individuals, we calculated a secondary outcome, number of partners, from the same series of questions, by summing across all partner types.
      Sex under the influence of substances in the past 90 days was also a secondary outcome. The Sexual Risk Assessment Questionnaire asked, “Please estimate the percent of times in the past 3 months that you were under the influence of [Substance] when engaging in these sex acts”. Substances queried were heroin, cocaine, methamphetamine, marijuana, and alcohol. Response options were categorical: “Never,” “<25% of the time,” “Between 25 and 50% of the time,” “About half the time,” “50 to 75% of the time,” “Over 75% of the time,” and “Always.” We computed the sex under the influence variable to be the maximum percent of time sex was under the influence of any substance by making it equal to each patient's substance-specific variable with the highest value.
      Perceived condom barriers were another secondary outcome. The Condom Barriers Scale (CBS) (
      • St. Lawrence J.S.
      • Chapdelaine A.P.
      • Devieux J.G.
      • O'Bannon R.E.
      • Barsfield T.L.
      • Eldridge G.D.
      Measuring perceived barriers to condom use: Psychometric evaluation of the condom barriers scale.
      ) was a self-report instrument consisting of 29 short statements reflecting attitudes about condoms rated on a 5-point Likert-type scale from 1 (strongly agree) to 5 (strongly disagree). We compared two separate factor analyses for women (
      • St. Lawrence J.S.
      • Chapdelaine A.P.
      • Devieux J.G.
      • O'Bannon R.E.
      • Barsfield T.L.
      • Eldridge G.D.
      Measuring perceived barriers to condom use: Psychometric evaluation of the condom barriers scale.
      ) and for men (
      • Doyle S.R.
      • Calsyn D.A.
      • Ball S.A.
      Factor structure of the Condoms Barriers Scale with a sample of men at high risk for HIV.
      ). We removed items that did not load for either men or women or that loaded on different factors for men versus women, resulting in an 18-item measure. With this sample, alphas at baseline for the four subscales were: partner barriers (6 items, 0.91), effects on sexual experience (6 items, 0.84), access/availability (4 items, 0.81), and motivational barriers (2 items, 0.52).

      3. Results

      3.1 Analysis strategy

      In Mplus V7.31, we used logistic, ordinal, linear, or negative binomial regression, depending on the outcome distribution. Randomization was successful: preliminary analyses showed no between-condition differences in demographics or baseline scores on outcomes. To account for variance due to nesting in sites, we included site as a control predictor in all analyses. In terms of clustering of patients within counselors, design effects computed from intraclass correlations and group sizes were low (ranging from 1.3 to 1.5) with cluster sizes >10; hence clustering was unlikely to negatively influence findings (
      • Lai M.
      • Kwok O.
      Examining the rule of thumb of not using multilevel modeling: The “design effect smaller than two” rule.
      ).
      Given the complexity of the experimental design and number of potential variables in necessary statistical tests, we tested the effects of time separately from tests of between-condition effects. After first testing whether changes occurred over time for the overall sample, we conducted between-condition analysis separately at each of the two follow-up timepoints (controlling for the baseline value of the outcome). We conceptualized the primary analyses as consistent with a 2 (counselor training) × 2 (patient assignment) design. Accordingly, the regression analysis included as predictors—in addition to site and baseline score on the outcome—the two experimental conditions (basic vs. enhanced training, no feedback vs. feedback) and their interaction.
      Because we were concerned that implementation factors could influence findings, preliminary analyses tested potentially confounding effects of three key implementation factors by testing their interaction with counselor training, patient assignment, and counselor training × patient assignment. Our a priori plan was to control in subsequent analysis implementation factors with any effect having a p value ≤ .15. We identified being in treatment and having changed counselors as implementation factors to control for the outcome, discussing sex in counseling. We did not need to control for the other implementation factor—changing to a counselor in a different training to condition.
      We performed three sets of additional analyses to identify factors that moderate the effects of the experimental conditions. The first set tested moderating effects of gender and age by including their main effects plus interactions with counselor training, patient assignment, and interaction of counselor training and patient assignment. The second set—conducted only for the primary outcomes—used the same method but examined the moderating influence of whether participants were sexually active at study entry or monogamous during the time asked about at each follow-up point. These two moderators were intended to statistically address two study limitations (see Discussion): inclusion of patients without regard to their baseline level of sexual risk, and use of USO as a primary outcome. The latter variable may be more indicative of risk for patients reporting non-monogamy than those reporting monogamy. The final analysis tested the moderating influence of type of treatment (MOUD vs. psychosocial).

      3.2 Missing data

      The amount of missing data varied between outcomes (3 % to 12 %) and was consistent with the likelihood that it met assumptions of MCAR (missing completely at random) or MAR (missing at random). The Full Information Maximum Likelihood (FIML) procedure that we used in regression analyses where outcome data are missing is appropriate in either case (
      • Graham J.W.
      Missing data: Analysis and design.
      ).

      3.3 Sample demographics

      Our 476 participating patients were 52 % male and 48 % female; their average age was 38.53 (SD 11.46). The majority (76 %) identified their race as white, 12 % as Black, 6 % as multiracial, 5 % as Native American, and 1 % as Asian American or Pacific Islander. Ten percent identified their ethnicity as Latino/a. Thirty-four percent had not finished high school, 29 % were high school graduates, 31 % had some college, 3 % had a bachelor's degree, and 3 % had attended graduate or professional school. About one-third of patients (32 %) were not sexually active at baseline. Patients were enrolled in MOUD (78 %) or psychosocial treatment (22 %).

      3.4 Primary outcomes

      3.4.1 Discussed sex in counseling

      The study did not measure this outcome at baseline because counseling had not begun. In the overall sample, the percent of patients who discussed sex decreased slightly but significantly (p = .04) between 3- and 6-month follow-ups. Table 1 shows that in the between-condition analysis, both training and feedback (but not their interaction) were significant predictors of having discussed sex at the 6-month, but not the 3-month, follow-up. Accordingly, Fig. 3 shows that each of these two main effects increased the likelihood that patients discussed sex with their counselors. Subsequent moderator analyses found no significant interactions among experimental conditions and age, gender, being sexually active at study entry, being monogamous at follow-up, or treatment type.
      Table 1Between-condition tests of the effects of counselor and patient assignment (n = 476).
      Basic trainingEnhanced trainingz
      z = the regression coefficient ÷ standard error. This is statistically significant in two-tailed tests if >1.96 (p < .05), 2.58 (p < .01), and 3.29 (p < .001).
      Effect size
      OR = odds ratio; IRR = incidence rate ratio.
      No feedback report

      (n = 116)
      Feedback report

      (n = 116)
      No feedback report

      (n = 115)
      Feedback report

      (n = 129)
      TrainingFeedback reportTraining × feedback reportTrainingFeedback reportTraining × feedback report
      Discussed sex
      Because patients were newly enrolled in counseling at baseline, opportunities to discuss sex had not yet occurred.
      3-month fu23 %30 %32 %40 %1.601.040.11OR = 1.68OR = 1.39OR = 1.05
      6-month fu14 %23 %30 %37 %3.11
      p < .01.
      2.02
      p < .05.
      0.55OR = 3.23OR = 2.14OR = 0.77
      Number unsafe sex occasions
      Baseline16.5 (26.11)
      Numbers in parentheses are standard deviations.
      18.6 (25.69)16.8 (25.15)20.5 (31.07)
      3-month fu14.4 (25.40)14.6 (23.16)15.3 (24.57)19.4 (29.56)0.270.400.49IRR = 0.93IRR = 0.90IRR = 1.19
      6-month fu14.8 (24.96)11.7 (20.86)14.8 (25.32)17.7 (28.25)0.770.830.67IRR = 0.81IRR = 0.80IRR = 1.28
      a Because patients were newly enrolled in counseling at baseline, opportunities to discuss sex had not yet occurred.
      b z = the regression coefficient ÷ standard error. This is statistically significant in two-tailed tests if >1.96 (p < .05), 2.58 (p < .01), and 3.29 (p < .001).
      c OR = odds ratio; IRR = incidence rate ratio.
      d Numbers in parentheses are standard deviations.
      low asterisk p < .05.
      low asterisklow asterisk p < .01.
      Fig. 3
      Fig. 3Effects of enhanced training and receipt of a feedback report on discussing sex in counseling (6-month follow-up).

      3.4.2 Number of USO

      Number of USO's correlation with the other primary outcome, discussing sex in counseling, was not significant at either follow-up. Number of USO declined over time in the overall sample; ꭓ2 = 10.2(2), p < .01. Table 1 shows that intervention conditions and their interaction were not significantly associated with USO. Subsequent moderator analysis identified one predictor, having been sexually active at study entry, as having significant interaction with type of training at 3-month follow-up and being marginally significant at 6-month follow-up (z = −2.07 and −1.91, p < .05 and =.06, respectively). Having been sexually active also had a significant interaction at 3-month follow-up with feedback condition (z = −2.61, p < .01) (Fig. 4). For type of training, sexually active participants with basic and enhanced training counselors showed similar reductions. In contrast, those not sexually active at baseline showed increases only with an enhanced training counselor. A similar, but less striking, pattern occurred for receipt of a PFR from baseline to 3-month follow-up, though disappearing by 6-month follow-up. Note that neither self-reported monogamy nor type of treatment (MOUD vs. psychosocial) moderated the effect of training type or feedback receipt on USO.
      Fig. 4
      Fig. 4Moderating effects of having been sexually active at study entry with type of training and with receipt of a feedback report on number of unsafe sex occasions.

      3.5 Secondary outcomes

      Analysis of secondary outcomes focused solely on the effects of intervention condition. Preliminary analysis identified three implementation factors associated with outcomes at 6-month follow-up: being in treatment predicting number of partners, time since training predicting access/availability condom barriers, and being in treatment predicting having sex under the influence.
      In the overall sample, sex under the influence and number of partners decreased over time, and motivational barriers to condom use increased; ꭓ2 = 87.1(2), 14.2(2), and 13.9(2), respectively, all p < .001. We found no statistically significant change in the three other condom barriers subscales (partner barriers, effects on sexual experience, and access/availability).
      Intervention conditions were not associated with number of sexual partners or having sex under the influence. Similarly, intervention conditions were not associated with three of the condom barriers subscales: partner barriers, access/availability, and motivational barriers. We found a significant two-way interaction of the two intervention conditions (type of training by receiving feedback) predicting the effects on sexual experience subscale of the condom barriers measure at the 3-month follow-up; z = −2.36, p < .05. Fig. 5 shows that the two enhanced training groups showed no change from baseline to 3-month follow-up. The two basic training groups show diverging changes: the feedback group increased their perception of barriers while the no-feedback group decreased theirs.
      Fig. 5
      Fig. 5Interaction of the type of training × receiving feedback on the sexual experience subscale of the condom barriers measure.

      4. Discussion

      4.1 Summary and conclusions

      This study sought to determine whether interventions at the counselor level, patient level, or both, could affect SUD patient sexual risk behavior. This study sought to mimic usual, real-world, conditions for counselor continuing education in that counselor use of training content with their patients was not required, trained to criterion, or monitored for fidelity. Likewise, the study made personalized feedback available to patients in the feedback condition and their counselors, but they decided whether, and how extensively, to review the PFR. Aside from pilot work for this study, we believe this to be the first study that examined effects of feedback provision on SUD counseling interactions. The overall findings of the study suggest that both training and feedback can increase counselor-patient discussions of sex, but that these discussions do not necessarily translate into decreases in sexual risk behavior. A key implication of our findings is that counselor's abilities need to be strengthened to facilitate such discussions into behavior change.
      Both enhanced counselor training (vs. basic training) and providing participant feedback (vs. not) resulted in an increased likelihood of patients' talking about sex with their SUD treatment counselor in the previous 30 days. This finding supported main study hypotheses and is consistent with prior literature on training health professionals (
      • Dodge W.T.
      • BlueSpruce J.
      • Grothaus L.
      • Rebolledo V.
      • McAfee T.A.
      • Carey J.W.
      • Thompson R.S.
      Enhancing primary care HIV prevention: A comprehensive clinical intervention.
      ;
      • Patel S.N.
      • Marks G.
      • Gardner L.
      • Golin C.E.
      • Shinde S.
      • O'Daniels C.
      • Wilson T.E.
      • Quinlivan E.B.
      • Banderas J.W.
      Brief training of HIV medical providers increases their frequency of delivering prevention counseling to patients at risk of transmitting HIV to others.
      ;
      • Rose C.D.
      • Courtenay-Quirk C.
      • Knight K.
      • Shade S.B.
      • Vittinghoff E.
      • Gomez C.
      • Lum P.J.
      • Bacon O.
      • Colfax G.
      HIV intervention for providers study: A randomized controlled trial of a clinician-delivered HIV risk-reduction intervention for HIV-positive people.
      ;
      • Thrun M.
      • Cook P.F.
      • Bradley-Springer L.A.
      • Gardner L.
      • Marks G.
      • Wright J.
      • Wilson T.E.
      • Quinlivan E.B.
      • O'Daniels C.
      • Raffanti S.
      • Thompson M.
      • Golin C.
      Improved prevention counseling by HIV care providers in a multi-site clinic-based intervention: Positive steps.
      ). The increased likelihood of conversations may in part result from enhanced counselors having greater self-efficacy and skill. We reported in a prior paper that enhanced counselor training increased self-efficacy to discuss sex with patients, and it improved their skill in using reflections in discussions about sex (
      • Hatch-Maillette M.A.
      • Harwick R.
      • Baer J.S.
      • Wells E.A.
      • Masters T.
      • Robinson A.
      • Cloud K.
      • Peavy M.
      • Wiest K.
      • Wright L.
      • Dillon K.
      • Beadnell B.
      Increasing substance use disorder counselors' self-efficacy and skills in talking to patients about sex and HIV risk: A randomized training trial.
      ). It is important to note that while the increases in discussions were statistically significant, the effects were moderate; for example, only a minority (37 %) of patients with enhanced counselors and receiving a PFR reported having discussed sex in the 30 days prior to their 6-month assessment.
      Two questions exist about the effect of feedback. The first concerns how discussions were initiated given that providing a PFR to both patient and counselor may have resulted in either of them initiating conversations about sex or sexual risk. We reviewed counselor coaching sessions and found that counselors rarely reported initiating review of PFRs. Unfortunately, our measures did not assess details of the interaction(s), such as who initiated, specific content, or length of discussion(s). Patient self-reports included whether they had discussed sex but not whether they had discussed the PFR. It may be that simply receiving or viewing the PFR triggered consideration of the topic. The second question concerns the length of time for this effect to occur. We found it somewhat surprising that the effect occurred for the period between the 3-month and 6-month surveys, given that PFRs were given out after patient baseline surveys. We do not know why this did not occur earlier; perhaps the time between baseline and 3-month follow-up was too early in the counseling relationship for topics not specifically related to substance use or that were more sensitive.
      Our data did not support hypotheses about effects of training or feedback on patient USO, number of partners, perceived condom barriers, or sex under the influence. While it may be the case that training and feedback are not generally effective with these behaviors, a methodological explanation is possible. Specifically, because we sought to conduct a test of the intervention as it might be applied in future clinical settings, we purposely did not include having high sexual risk as an eligibility criterion. To address the potential impact of our heterogeneous sample, we included whether patients were sexually active at baseline as a moderator for primary outcome analyses. Results indicated neither training nor feedback condition assignment resulted in differential reductions in USO among sexually active participants. However, risk behavior decreased over time across conditions leaving rates possibly too low for a detectable condition effect to exist. This decrease in risk is consistent with prior research showing an association between treatment entry or retention and sexual risk reduction (
      • Kidorf M.
      • Brooner R.K.
      • Yan H.
      • Peirce J.
      Sexual-risk reduction following the referral of syringe exchange registrants to methadone maintenance: Impact of gender and drug use.
      ;
      • Martin G.S.
      • Serpelloni G.
      • Galvan U.
      • Rizzetto A.
      • Gomma M.
      • Morgante S.
      • Rezza G.
      Behavioural change in injecting drug users: Evaluation of an HIV/AIDS education programme.
      ;
      • Wells E.A.
      • Calsyn D.A.
      • Clark L.L.
      • Saxon A.J.
      • Jackson R.
      Retention in methadone maintenance is associated with reductions in different HIV risk behaviors for women and men.
      ;
      • Williams B.
      • McNelly E.A.
      • Williams A.E.
      • D'Aquila R.T.
      Methadone maintenance treatment and HIV type 1 seroconversion among injecting drug users.
      ). Researchers in future studies of this topic should bear in mind the statistical challenges when considering their eligibility criteria and sample sizes.
      Our assumption that increasing counselor-patient discussion would reduce patient risk behavior was unsupported in that we found patients' reports of discussing sex with their counselor were uncorrelated to their reports of unsafe sex. This finding may be due to the nature of the discussions, which might have been too limited to have much effect. We do not know how lengthy these discussions were or whether counselors, for example, supported patient change-talk related to risk reduction. Counselors' skills may have declined over time, reducing their impact on patient sexual risk behavior (see Limitations). Even studies with rigorous training and fidelity monitoring of brief feedback interventions sometimes fail to find feedback effects on risk behavior (
      • Brems C.
      • Dewane S.L.
      • Johnson M.E.
      • Eldridge G.D.
      Brief motivational interventions for HIV/STI risk reduction among individuals receiving alcohol detoxification.
      ). Perhaps seeing the PFR was enough to promote raising sex risk in counseling but was insufficient for behavior change.
      Unexpectedly, the interaction of whether someone was sexually active at baseline with experimental conditions indicated that patients not active at baseline who received PFRs or whose counselors received enhanced training increased unsafe sex more than those in the control conditions. The reason for this finding is unclear. For both enhanced training and PFR patients, the increased discussions about sex with counselors may have enhanced their willingness to self-report unsafe sex at follow-up. The PFR included messages for sexually abstinent patients, encouraging safe behavior if they had sex. However, hearing their risk was low, combined with normative feedback about sexual abstinence, could have motivated increased sexual activity. Statements urging patients to be safe if they became sexually active may need to be strengthened.
      Results regarding patient-perceived condom barriers related to the perceived effect on sexual experience also were unexpected: For patients of counselors in the basic condition, feedback appeared to increase perceived barriers at 3 months relative to no feedback. If replicated, this interaction could indicate that feedback, without added counselor training, heightens awareness about perceived downsides of condom use.

      4.2 Limitations

      Traditional, efficacy-based clinical trials examining the impact of training on counselors include mandated counselor procedures that reflect their training, and measures of fidelity and adherence. However, these elements are often dissimilar to counselors' everyday practices. This focus on internal validity sacrifices external validity in the clinical settings that these interventions are ultimately designed for, and which typically are under-funded, under-staffed, and busy (
      • Glasgow R.E.
      • Vogt T.M.
      • Boles S.M.
      Evaluating the public health impact of health promotion interventions: The RE-AIM framework.
      ). We sought to examine both counselor- and patient-level interventions as they might be delivered in real-world clinical settings, which may have weakened potential effects on behavior.
      As noted in
      • Hatch-Maillette M.A.
      • Harwick R.
      • Baer J.S.
      • Wells E.A.
      • Masters T.
      • Robinson A.
      • Cloud K.
      • Peavy M.
      • Wiest K.
      • Wright L.
      • Dillon K.
      • Beadnell B.
      Increasing substance use disorder counselors' self-efficacy and skills in talking to patients about sex and HIV risk: A randomized training trial.
      , having only a 3-month, and not a later, follow-up assessment evaluating counselor self-efficacy and skill was a limitation of our study. We know that skill gains often decay (e.g.,
      • Baer J.S.
      • Rosengren D.R.
      • Dunn C.
      • Wells E.A.
      • Ogle R.
      • Hartzler B.
      An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians.
      ;
      • Miller W.R.
      • Mount K.A.
      A small study of training in motivational interviewing: Does one workshop change clinician and client behavior?.
      ), and we do not know how such decay may have contributed to the lack of effect of counselor training on condomless sex.
      Although research hast noted high risk sexual behavior in SUD treatment populations (
      • Fairbairn N.
      • Hayashi K.
      • Milloy M.
      • Nolan S.
      • Nguyen P.
      • Wood E.
      • Kerr T.
      Hazardous alcohol use associated with increased sexual risk behaviors among people who inject drugs.
      ;
      • Kral A.H.
      • Bluthenthal R.N.
      • Lorvick J.
      • Gee L.
      • Bacchetti P.
      • Edlin B.R.
      Sexual transmission of HIV-1 among injection drug users in San Francisco, USA: Risk-factor analysis.
      ;
      • Shoptaw S.
      • Montgomery B.
      • Williams C.T.
      • El-Bassel N.
      • Aramrattana A.
      • Metsch L.
      • Metzger D.S.
      • Kuo I.
      • Bastos F.I.
      • Strathdee S.A.
      Not just the needle: The state of HIV-prevention science among substance users and future directions.
      ;
      • Volkow N.D.
      • Montaner J.
      The urgency of providing comprehensive and integrated treatment for substance abusers with HIV.
      ), ours was a varied risk sample, as it included people not sexually active at baseline. We tested the interventions with an externally valid representation of the in-treatment population and controlled for sexual activity at baseline. However, doing so, left less power to detect change.
      Using USO as a primary outcome has potential drawbacks. Although some patients reporting monogamy have risky partners (see e.g.,
      • Wells E.A.
      • Clark L.L.
      • Calsyn D.A.
      • Saxon A.J.
      • Jackson T.R.
      • Wrede A.F.
      Reporting of HIV risk behaviors by injection drug using heterosexual couples in methadone maintenance.
      ), patients reporting monogamy may be counted as at greater-than-warranted disease risk using this outcome. To address this limitation, we controlled for reported monogamy in primary outcome analyses. In addition, we included a secondary outcome variable, number of partners, to capture risk associated with non-monogamy.
      We did not collect data on actual counselor/patient interactions. Although discussions about sex were increased by both enhanced training and provision of feedback, their content remains unknown. In prior research (
      • Baer J.S.
      • Rosengren D.R.
      • Dunn C.
      • Wells E.A.
      • Ogle R.
      • Hartzler B.
      An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians.
      ), we found it difficult to obtain actual recorded samples of clinical work from community treatment clinicians; accordingly, this study did not include them.

      4.3 Future directions

      The results of this study raise questions for future research. For example, does content versus length of training account for changes in counselor-patient interactions? To what degree do counselor skills translate from training to actual (as opposed to standardized) patient interactions? And can feedback effects be improved by providing incentives to counselors or requiring them to review PFRs with patients, with fidelity monitoring?

      CRediT authorship contribution statement

      Mary Hatch: Conceptualization, Funding acquisition, Project administration, Supervision, Validation, Roles/Writing-original draft, Writing-review and editing. Betsy Wells: Conceptualization, Funding acquisition, Validation, Methodology, Roles/writing-original draft, Writing-review and editing. Tatiana Masters: Data curation, Formal analysis, Methodology, Validation, Visualization, Writing-review and editing. Blair Beadnell: Data curation, Formal analysis, Methodology, Validation, Visualization, Writing-review and editing. Robin Harwick: Project Administration, Supervision, Validation, Roles/Writing – original draft. Lynette Wright: Data curation, Project administration, Supervision, Validation, Writing-review and editing. Esther Ricardo-Bulis: Data curation, Investigation, Supervision, Validation, Writing-review and editing. Katherina Weist: Methodology, Resources, Supervision. Carrie Shriver: Data curation, Investigation, Validation. John Baer: Conceptualization, Funding Acquisition, Methodology, Visualization, Supervision, Resources, Roles/writing-original draft, Writing-review and editing.

      Declarations of competing interest

      None.

      Acknowledgements

      Clinical Trials.gov Identifier: NCT03575585
      This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (5R01HD078163).
      The authors would like to thank the staff and patients at participating sites and to acknowledge the inspiration and mentorship from the original Principal Investigator for this project, Donald A. Calsyn, Ph.D. (1950–2013).

      References

        • Albarracín D.
        • Gillette J.C.
        • Earl A.N.
        • Glasman L.R.
        • Durantini M.R.
        • Ho M.-H.
        A test of major assumptions about behavior change: A comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic.
        Psychological Bulletin. 2005; 131: 856-897
        • Annon J.S.
        The PLISSIT model: A proposed conceptual scheme for the behavioral treatment of sexual problems.
        Journal of Sex Education and Therapy. 1976; 2: 1-15https://doi.org/10.1080/01614576.1976.11074483
        • Baer J.S.
        • Rosengren D.R.
        • Dunn C.
        • Wells E.A.
        • Ogle R.
        • Hartzler B.
        An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians.
        Drug and Alcohol Dependence. 2004; 73: 99-106https://doi.org/10.1016/j.drugalcdep.2003.10.001
        • Bluespruce J.
        • Dodge W.T.
        • Grothhaus L.
        • Wheeler K.
        • Rebolledo V.
        • Carey J.W.
        • McAfee T.A.
        • Thompson R.S.
        HIV prevention in primary care: Impact of a clinical intervention.
        AIDS Patient Care and STDs. 2001; 15: 243-253https://doi.org/10.1089/10872910152050766
        • Bradley-Springer L.A.
        • Everett M.R.
        • Rotach E.G.
        • Vojir C.P.
        Changes in clinician ability to assess risk and help patients determine the need for HIV testing: A comparison of three teaching methods.
        Evaluation and the Health Professions. 2006; 29: 367-393https://doi.org/10.1177/0163278706293404
        • Brems C.
        • Dewane S.L.
        • Johnson M.E.
        • Eldridge G.D.
        Brief motivational interventions for HIV/STI risk reduction among individuals receiving alcohol detoxification.
        AIDS Education and Prevention. 2009; 21: 397-414https://doi.org/10.1521/aeap.2009.21.5.397
        • Broome K.M.
        • Joe G.W.
        • Simpson D.D.
        HIV risk reduction in outpatient drug abuse treatment: Individual and geographic differences.
        AIDS Education and Prevention. 1999; 11: 293-306
        • Brown N.K.
        Clinical judgments of high-risk behavior during recovery.
        Journal of Psychoactive Drugs. 2000; 32: 299-304https://doi.org/10.1080/02791072.2000.10400453
        • Calsyn D.A.
        • Hatch-Maillette M.
        • Tross S.
        • Doyle S.R.
        • Crits-Christoph P.
        • Song Y.S.
        • Harrer J.M.
        • Lalos G.
        • Berns S.B.
        Motivational and skills training HIV/sexually transmitted infection sexual risk reduction groups for men.
        Journal of Substance Abuse Treatment. 2009; 37: 138-150https://doi.org/10.1016/j.jsat.2008.11.008
        • Calsyn D.A.
        • Burlew A.K.
        • Hatch-Maillette M.A.
        • Beadnell B.
        • Wright L.
        • Wilson J.
        An HIV prevention intervention for ethnically diverse men in substance abuse treatment: Pilot study findings.
        American Journal of Public Health. 2013; 103: 896-902https://doi.org/10.2105/AJPH.2012.300970
        • Calsyn D.A.
        • Peavy K.M.
        • Wells E.A.
        • Campbell A.N.C.
        • Hatch-Maillette M.A.
        • Greenfield S.F.
        • Tross S.
        Differences between men and women in condom use, attitudes, and skills in substance abuse treatment seekers.
        The American Journal on Addictions. 2013; 22: 150-157https://doi.org/10.1111/j.1521-0391.2013.00312.x
        • Centers for Disease Control and Prevention
        Sexually transmitted diseases treatment guidelines, 2015.
        Morbidity and Mortality Weekly Report. 2015; 64 (Retrieved February 8, 2018, from https://www.cdc.gov/std/tg2015/emerging.htm)
        • Centers for Disease Control and Prevention
        CDC fact sheet: Today's HIV/AIDS epidemic.
        (Retrieved June 5, 2017, from)
        • Centers for Disease Control and Prevention
        Statistics overview.
        (Retrieved June 5, 2017, from)
        • Cohen S.J.
        • Halvorson H.W.
        • Gosselink C.A.
        Changing physician behavior to improve disease prevention.
        Preventive Medicine. 1994; 23: 284-291https://doi.org/10.1006/pmed.1994.1040
        • Covey J.
        • Rosenthal-Stott H.E.S.
        • Howell S.J.
        A synthesis of meta-analytic evidence of behavioral interventions to reduce HIV/STIs.
        Journal of Behavioral Medicine. 2016; 39: 371-385https://doi.org/10.1007/s10865-016-9714-1
        • Dodge W.T.
        • BlueSpruce J.
        • Grothaus L.
        • Rebolledo V.
        • McAfee T.A.
        • Carey J.W.
        • Thompson R.S.
        Enhancing primary care HIV prevention: A comprehensive clinical intervention.
        American Journal of Preventive Medicine. 2001; 20: 177-183https://doi.org/10.1016/s0749-3797(00)00308-1
        • Doyle S.R.
        • Calsyn D.A.
        • Ball S.A.
        Factor structure of the Condoms Barriers Scale with a sample of men at high risk for HIV.
        Assessment. 2009; 16: 3-15https://doi.org/10.1177/1073191108322259
        • Dreisbach S.
        • Burnside H.
        • Hsu K.
        • Smock L.
        • Coury-Doniger P.
        • Hall C.
        • Marrazzo J.
        • Nagendra G.
        • Rietmeijer C.
        • Rompalo A.
        • Thrun M.
        Improving HIV/STD prevention in the care of persons living with HIV through a national training program.
        AIDS Patient Care and STDs. 2014; 28: 15-21https://doi.org/10.1089/apc.2013.0094
        • Drapkin M.L.
        • Wilbourne P.
        • Manuel J.K.
        • Baer J.
        • Karlin B.
        • Raffa S.
        National dissemination of Motivation Enhancement Therapy in the Veterans Health Administration: Training program design and initial outcomes.
        Journal of Substance Abuse Treatment. 2016; 65: 83-87https://doi.org/10.1016/j.jsat.2016.02.002
        • Engstrom M.
        • Shibusawa T.
        • El-Bassel N.
        • Gilbert L.
        Age and HIV sexual risk among women in methadone treatment.
        AIDS and Behavior. 2011; 15: 103-113https://doi.org/10.1007/s10461-009-9625-z
        • Fairbairn N.
        • Hayashi K.
        • Milloy M.
        • Nolan S.
        • Nguyen P.
        • Wood E.
        • Kerr T.
        Hazardous alcohol use associated with increased sexual risk behaviors among people who inject drugs.
        Alcoholism: Clinical and Experimental Research. 2016; 40: 2394-2400https://doi.org/10.1111/acer.13216
        • Frost H.
        • Campbell P.
        • Maxwell M.
        • O’Carroll R.E.
        • Dombrowski S.U.
        • Williams B.
        • Cheyne H.
        • Coles E.
        • Pollock A.
        Effectiveness of motivational interviewing on adult behaviour change in health and social care settings: A systematic review of reviews.
        PLoS ONE. 2018; 13e0204890https://doi.org/10.1371/journal.pone.0204890
        • Gerbert B.
        • Berg-Smith S.
        • Mancuso M.
        • Caspers N.
        • McPhee S.
        • Null D.
        • Wofsy J.
        Using innovative video doctor technology in primary care to deliver brief smoking and alcohol intervention.
        Health Promotion Practice. 2003; 4: 249-261https://doi.org/10.1177/1524839903004003009
        • Gibson D.R.
        • Flynn N.M.
        • McCarthy J.J.
        Effectiveness of methadone treatment in reducing HIV risk behavior and seroconversion among injecting drug users.
        AIDS. 1999; 13: 1807-1818https://doi.org/10.1097/00002030-199910010-00002
        • Glasgow R.E.
        • Vogt T.M.
        • Boles S.M.
        Evaluating the public health impact of health promotion interventions: The RE-AIM framework.
        American Journal of Public Health. 1999; 89: 1322-1327https://doi.org/10.2105/AJPH.89.9.1322
        • Gottheil E.
        • Lundy A.
        • Weinstein S.P.
        • Sterling R.C.
        Does intensive outpatient cocaine treatment reduce AIDS risky behaviors?.
        Journal of Addictive Diseases. 1998; 17: 61-69https://doi.org/10.1300/J069v17n04_06
        • Graham J.W.
        Missing data: Analysis and design.
        Springer, New York, NY2012
        • Hatch-Maillette M.A.
        • Harwick R.
        • Baer J.S.
        • Wells E.A.
        • Masters T.
        • Robinson A.
        • Cloud K.
        • Peavy M.
        • Wiest K.
        • Wright L.
        • Dillon K.
        • Beadnell B.
        Increasing substance use disorder counselors' self-efficacy and skills in talking to patients about sex and HIV risk: A randomized training trial.
        Drug and Alcohol Dependence. 2019; 199: 76-84https://doi.org/10.1016/j.drugalcdep.2019.02.023
        • Haynes L.
        • Calsyn D.
        • Tross S.
        Addressing sexual issues in addictions treatment.
        Counselor (Deerfield Beach). 2008; 9: 28-36
        • Hoffman J.A.
        • Klein H.
        • Clark D.C.
        • Boyd F.T.
        The effect of entering drug treatment on involvement in HIV-related risk behaviors. (1998).
        The American Journal of Drug and Alcohol Abuse. 1998; 24: 259-284https://doi.org/10.3109/00952999809001712
        • Kidorf M.
        • Brooner R.K.
        • Yan H.
        • Peirce J.
        Sexual-risk reduction following the referral of syringe exchange registrants to methadone maintenance: Impact of gender and drug use.
        Journal of Substance Abuse Treatment. 2021; 124108286https://doi.org/10.1016/j.jsat.2021.108286
        • Kral A.H.
        • Bluthenthal R.N.
        • Lorvick J.
        • Gee L.
        • Bacchetti P.
        • Edlin B.R.
        Sexual transmission of HIV-1 among injection drug users in San Francisco, USA: Risk-factor analysis.
        Lancet. 2001; 357: 1397-1401https://doi.org/10.1016/S0140-6736(00)04562-1
        • Lai M.
        • Kwok O.
        Examining the rule of thumb of not using multilevel modeling: The “design effect smaller than two” rule.
        The Journal of Experimental Education. 2014; 83: 423-438
        • Latka M.H.
        • Wilson T.E.
        • Cook J.A.
        • Bacon M.C.
        • Richardson J.L.
        • Sohler N.
        • Cohen M.H.
        • Greenblatt R.M.
        • Andreopoulis E.
        • Vlahov D.
        Impact of treatment on subsequent sexual risk behavior in a multisite cohort of drug using women: A report from the women's interagency HIV study.
        Journal of Substance Abuse Treatment. 2005; 29: 329-337https://doi.org/10.1016/j.jsat.2005.08.008
        • Lewis M.A.
        • Patrick M.E.
        • Litt D.M.
        • Atkins D.C.
        • Kim T.
        • Blayney J.A.
        • Norris J.
        • George W.H.
        • Larimer M.E.
        Randomized controlled trial of a web-delivered personalized normative feedback intervention to reduce alcohol-related risky sexual behavior among college students.
        Journal of Consulting and Clinical Psychology. 2014; 82: 429-440https://doi.org/10.1037/a0035550
        • Lundahl B.W.
        • Kunz C.
        • Brownell C.
        • Tollefson D.
        • Burke B.L.
        A meta-analysis of motivational interviewing: Twenty-five years of empirical studies.
        Research on Social Work Practice. 2010; 20: 137-160https://doi.org/10.1177/1049731509347850
        • Martin G.S.
        • Serpelloni G.
        • Galvan U.
        • Rizzetto A.
        • Gomma M.
        • Morgante S.
        • Rezza G.
        Behavioural change in injecting drug users: Evaluation of an HIV/AIDS education programme.
        AIDS Care. 1990; 2: 275-279https://doi.org/10.1080/09540129008257740
        • Martino S.
        Strategies for training counselors in evidence-based treatments.
        Addiction Science and ClinicalPractice. 2010; 5 (www.ncbi.nlm.nih.gov/pmc/articles/PMC3120122/): 30-39
        • Meader N.
        • Li R.
        • Des Jarlais D.C.
        • Pilling S.
        Psychosocial interventions for reducing injection and sexual risk behavior for preventing HIV in drug users.
        Cochrane Database of Systematic Reviews. 2010; 2010CD007192https://doi.org/10.1002/14651858.CD007192.pub2
        • Metzger D.S.
        • Woody G.E.
        • O'Brien C.P.
        Drug treatment as HIV prevention: A research update.
        Journal of Acquired Immune Deficiency Syndrome. 2010; 55: S32-S36https://doi.org/10.1097/QAI.0b013e3181f9c10b
        • Miller W.R.
        • Mount K.A.
        A small study of training in motivational interviewing: Does one workshop change clinician and client behavior?.
        Behavioural and Cognitive Psychotherapy. 2001; 29: 457-471https://doi.org/10.1017/S1352465801004064
        • Mitchell C.G.
        • Oltean A.
        Integrating HIV prevention into substance user treatment: Current practices and challenges.
        Substance Use & Misuse. 2007; 42: 2173-2182https://doi.org/10.1080/10826080701663190
        • Murphy D.A.
        • Brecht M.-L.
        • Herbeck D.
        • Evans E.
        • Huang D.
        • Hser Y.-I.
        Longitudinal HIV risk behavior among the Drug Abuse Treatment Outcome Studies (DATOS) adult sample.
        Evaluation Review. 2008; 32: 83-112https://doi.org/10.1177/0193841X07307411
        • Patel S.N.
        • Marks G.
        • Gardner L.
        • Golin C.E.
        • Shinde S.
        • O'Daniels C.
        • Wilson T.E.
        • Quinlivan E.B.
        • Banderas J.W.
        Brief training of HIV medical providers increases their frequency of delivering prevention counseling to patients at risk of transmitting HIV to others.
        Health Education Journal. 2012; 72: 431-442
        • Potter J.
        Talking to patients about sex curriculum.
        (Retrieved from)
        • Rose C.D.
        • Courtenay-Quirk C.
        • Knight K.
        • Shade S.B.
        • Vittinghoff E.
        • Gomez C.
        • Lum P.J.
        • Bacon O.
        • Colfax G.
        HIV intervention for providers study: A randomized controlled trial of a clinician-delivered HIV risk-reduction intervention for HIV-positive people.
        Journal of Acquired Immune Deficiency Syndrome. 2010; 55: 572-581https://doi.org/10.1097/QAI.0b013e3181ee4c62
        • Satterwhite C.L.
        • Torrone E.
        • Meites E.
        • Dunne E.F.
        • Mahajan R.
        • Ocfemia M.C.B.
        • Su J.
        • Xu F.
        • Weinstock H.
        Sexually transmitted infections among US women and men.
        Sexually Transmitted Diseases. 2013; 40: 187-193https://doi.org/10.1097/olq.0b013e318286bb53
        • Shoptaw S.
        • Montgomery B.
        • Williams C.T.
        • El-Bassel N.
        • Aramrattana A.
        • Metsch L.
        • Metzger D.S.
        • Kuo I.
        • Bastos F.I.
        • Strathdee S.A.
        Not just the needle: The state of HIV-prevention science among substance users and future directions.
        Journal of Acquired Immune Deficiency Syndrome. 2013; 63: S174-S178https://doi.org/10.1097/QAI.0b013e3182987028
        • Springer S.A.
        • Larney S.
        • Alam-mehrjerdi Z.
        • Altice F.L.
        • Metzger D.
        • Shoptaw S.
        Drug treatment as HIV prevention among women and girls who inject drugs from a global perspective: Progress, gaps, and future directions.
        Journal of Acquired Immune Deficiency Syndrome. 2015; 69: S155-S161https://doi.org/10.1097/QAI.0000000000000637
        • St. Lawrence J.S.
        • Chapdelaine A.P.
        • Devieux J.G.
        • O'Bannon R.E.
        • Barsfield T.L.
        • Eldridge G.D.
        Measuring perceived barriers to condom use: Psychometric evaluation of the condom barriers scale.
        Assessment. 1999; 6: 391-400https://doi.org/10.1177/107319119900600409
        • Thrun M.
        • Cook P.F.
        • Bradley-Springer L.A.
        • Gardner L.
        • Marks G.
        • Wright J.
        • Wilson T.E.
        • Quinlivan E.B.
        • O'Daniels C.
        • Raffanti S.
        • Thompson M.
        • Golin C.
        Improved prevention counseling by HIV care providers in a multi-site clinic-based intervention: Positive steps.
        AIDS Education and Prevention. 2009; 21: 55-66https://doi.org/10.1521/aeap.2009.21.1.55
        • Tross S.
        • Campbell A.N.C.
        • Cohen L.R.
        • Calsyn D.
        • Pavlicova M.
        • Miele G.
        • Hu M.
        • Haynes L.
        • Nugent N.
        • Gan W.
        • Hatch-Maillette M.
        • Mandler R.
        • McLaughlin P.
        • El-Bassel N.
        • Crits-Christoph P.
        • Nunes E.V.
        Effectiveness of HIV/STD sexual risk reduction groups for women in substance abuse treatment programs: Results of a NIDA Clinical Trials Network Trial.
        Journal of Acquired Immune Deficiency Syndrome. 2008; 48: 581-589https://doi.org/10.1097/QAI.0b013e31817efb6e
        • Tucker T.
        • Fry C.L.
        • Lintzeris N.
        • Baldwin S.
        • Ritter A.
        • Donath S.
        • Whelan G.
        Randomized controlled trial of a brief behavioural intervention for reducing hepatitis C virus risk practices among injecting drug users.
        Addiction. 2004; 99: 1157-1166https://doi.org/10.1111/j.1360-0443.2004.00809.x
        • Tucker J.S.
        • D'Amico E.J.
        • Ewing B.A.
        • Miles J.N.V.
        • Pedersen E.R.
        A group-based motivational interviewing brief intervention to reduce substance use and sexual risk behavior among homeless young adults.
        Journal of Substance Abuse Treatment. 2017; 76: 20-27https://doi.org/10.1016/j.jsat.2017.02.008
        • Volkow N.D.
        • Montaner J.
        The urgency of providing comprehensive and integrated treatment for substance abusers with HIV.
        Health Affairs. 2011; 30 (Retrieved October 4, 2019, from https://www.healthaffairs.org/doi/10.1377/hlthaff.2011.0663)
        • Walters S.T.
        • Matson S.A.
        • Baer J.S.
        • Ziedonis D.M.
        Effectiveness of workshop training for psychosocial addiction treatments: A systematic review.
        Journal of Substance Abuse Treatment. 2005; 29: 283-293https://doi.org/10.1016/j.jsat.2005.08.006
        • Weissman J.
        • Kanamori M.
        • Devieux J.G.
        • Trepka M.J.
        • De La Rosa M.
        HIV risk reduction interventions among substance-abusing reproductive-age women: A systematic review.
        AIDS Education and Prevention. 2017; 29: 121-140https://doi.org/10.1521/aeap.2017.29.2.121
        • Wells E.A.
        • Calsyn D.A.
        • Clark L.L.
        • Saxon A.J.
        • Jackson R.
        Retention in methadone maintenance is associated with reductions in different HIV risk behaviors for women and men.
        American Journal of Drug and Alcohol Abuse. 1996; 22: 509-521https://doi.org/10.3109/00952999609001677
        • Wells E.A.
        • Clark L.L.
        • Calsyn D.A.
        • Saxon A.J.
        • Jackson T.R.
        • Wrede A.F.
        Reporting of HIV risk behaviors by injection drug using heterosexual couples in methadone maintenance.
        Drug and Alcohol Dependence. 1994; 36: 33-38https://doi.org/10.1016/0376-8716(94)90007-8
        • Williams B.
        • McNelly E.A.
        • Williams A.E.
        • D'Aquila R.T.
        Methadone maintenance treatment and HIV type 1 seroconversion among injecting drug users.
        AIDS Care. 1992; 4: 35-41https://doi.org/10.1080/09540129208251618