Training and Education in Professional Psychology

© 2020 American Psychological Association 2022, Vol. 16, No. 3, 253– 262
ISSN: 1931-3918 http://dx.doi.org/10.1037/tep0000320

Examining the Effects of the Supervisory Relationship and Therapeutic

Alliance on Client Outcomes in Novice Therapists
M. Kristina DePue, Ren Liu, Glenn W. Lambie, and Jessica Gonzalez
The University of Central Florida

The supervisory working alliance (SWA) is an element of the supervisory relationship (SR) and has also
been found to be related to the therapeutic alliance (TA; DePue, Lambie, Liu, & Gonzalez, 2016). As the
TA has a well-established relationship with client outcomes (Leibert, Smith, & Agaskar, 2011), the SWA
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

should also be related to client outcomes as it works through the TA (Bambling, King, Raue, Schweitzer,
This document is copyrighted by the American Psychological Association or one of its allied publishers.

& Lambert, 2006). No researchers have examined how the SWA and TA between therapists, supervisors,
and clients may work together to predict client outcomes using dyadic data analysis, strong client
outcome measures, and client perspectives of the TA. The authors examined the contribution of the TA
between novice therapists (n ⫽ 155) and their clients (n ⫽ 193) on client outcomes, as well as the
contribution of the supervisees’ SWA scores on their client outcomes. Data was matched between
therapist/supervisees and clients, and 2 structural equation models were developed to investigate the
hypothesized contribution of the TA and SWA on client outcome. Results identified that the strength of
the SWA and TA have direct effects on client outcomes, and the effect of the SWA on client outcomes
is not mediated by the TA when clients’ ratings of TA are used. The SWA is both directly and indirectly
related to client outcome, when considering client and therapist ratings of the TA.

Public Significance Statement

Therapists should continue to foster the TA and consider it as a fundamental component of effective
therapy that impacts client outcome. Clinical supervisors can foster the SWA to help strengthen
novice trainees’ confidence in their perceptions of the TA. As the strength of the SWA is directly
related to client outcome, clinical supervisors should monitor the SWA and aim to repair any ruptures
in the relationship if they occur, as this can directly impact both the supervisee and clients. In
addition, we encourage supervisees to advocate for their needs within the supervision process.

Keywords: client outcome, clinical supervision, therapist training, therapeutic process

This article was published Online First April 30, 2020. served as the chair for the Department of Child, Family, and Community
M. KRISTINA DEPUE, PhD, NCC, is an assistant professor of counselor Sciences (2014 –2018). He is a National Certified Counselor, a National
education at the University of Florida (UF) and received her doctoral Certified School Counselor, and a Certified Clinical Mental Health Coun-
degree in counselor education from the University of Central Florida. She selor. He has practiced in the field of counseling and counselor education
also graduated from Vanderbilt University for both her master’s and for over 20 years. He is an American Counseling Association Fellow and
bachelor’s degrees. Dr. DePue serves on the board of directors for the UF conducts empirical investigations examining constructs in the areas of (a)
Collegiate Recovery Community and holds affiliate faculty status with UF counselor development and supervision, (b) counseling children and ado-
College of Medicine. Her two core research areas are addictions, specifi- lescents, and (c) professional school counseling.
cally concentrating on gender disparities and stigma, and counselor devel- JESSICA GONZALEZ, PhD, LPC, NCC, is an assistant professor at
opment and supervision, focusing on how both counseling trainees and Colorado State University. She received her doctorate in education with
clients change. an emphasis in counselor education from the University of Central
REN LIU, PhD, is an assistant professor of quantitative psychology at Florida and a master’s degree in clinical mental health counseling from
the University of California, Merced. He received his doctoral degree in Nova Southeastern University. She is now at the School of Education,
research and evaluation methodology from the University of Florida. His Colorado State University. She specializes in working with adults in
research is focused on statistical and measurement methods as applied in outpatient/inpatient counseling settings, as well as working with family
psychology, education, and social sciences. His methodological work is caregivers and persons with health disparities. She is also the past
centered on advancing item response theory models and their applications, president of the Florida Counseling Association for Multicultural Coun-
specifically in the area of diagnostic classification models and rating scale seling and Development.
design and analysis. CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to
GLENN W. LAMBIE, PhD, serves as the associate dean for graduate M. Kristina DePue, School of Human Development and Organizational
affairs and faculty excellence and the Robert N. Heintzelman Eminent Studies in Education, The University of Florida, 1506 East Ridgewood
Scholar Endowed Chair. He is a professor of counselor education and Street, Orlando, FL 32803. E-mail: kristinadepue@gmail.com

253
 254 DEPUE, LIU, LAMBIE, AND GONZALEZ

The therapeutic alliance (TA) is an essential factor within ef- ined the relationship between the TA and symptomatic change in
fective therapy, and is a collaborative approach between clients clients diagnosed with depression (Zilcha-Mano, Dinger, McCar-
and therapists that is defined as the agreement between the tasks thy, & Barber, 2014). Modeling over four time points revealed that
and goals of therapy, and the emotional bond between the therapist stronger TA scores predicted lower levels of depression (and vice
and client throughout the therapeutic process (Tracey & Koko- versa); however, outcome scores did not predict the TA over time.
tovic, 1989). Common factors in therapy (e.g., empathy, warmth, These results help justify the directionality for the current model,
congruence, and the TA; Lambert & Barley, 2001) have consis- which is that the TA directly impacts client outcome. As the
tently been found to have the most influence on client outcome, current study focuses on novice trainees, it is necessary to consider
regardless of treatment modality (e.g., Lei & Duan, 2014; Malin & the role of supervision in the formulation of the TA. As the
Pos, 2015; Norcross, 2011), with therapist effects (i.e., the effect presence of the TA is a factor influencing client change, the
the individual therapist has on client outcome) representing a supervisory working alliance (SWA) has also been found to impact
moderate effect size on treatment outcome (r ⫽ .275; Horvath, Del supervisee change factors (Inman & Ladany, 2008); therefore,
Re, Flückiger, & Symonds, 2011). The TA has been repeatedly additional research is needed to explore factors that influence the
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

found to predict client outcomes (Leibert, Smith, & Agaskar, strength of the TA, specifically including the supervisory relation-
This document is copyrighted by the American Psychological Association or one of its allied publishers.

2011), being noted as a precondition for treatment effectiveness, as ship (SR).

well as a contributor to treatment failure (Weck, Grikscheit, Jakob,
Höfling, & Stangier, 2015). However, the exact nature of why the
Supervisory Relationship (SR)
TA is effective is not fully understood, partially due to the complex
nature of factors influencing the TA. For example, the strength of The SR may be the most well-researched and efficacious com-
the TA is influenced by client expectations of the expertise of ponent of the supervision process (Callahan & Watkins, 2018a,
therapists (Patterson, Anderson, & Wei, 2014). As such, variability 2018c; Inman et al., 2014; Watkins & Callahan, 2019). To
in therapists, rather than the clients, might contribute to the TA and strengthen research on supervision, Watkins (2018) offered the
client outcomes (e.g., Baldwin, Wampold, & Imel, 2007; Del Re, generic model of psychotherapy supervision (GMPS) as a tran-
Flückiger, Horvath, Symonds, & Wampold, 2012). Moreover, stheoretical model of the commonalities within supervision expe-
some therapists’ personal characteristics contribute to a strong TA, riences that stemmed from decades of work on the common factors
including trustworthiness, warmth, confidence, flexibility, hon- of the therapeutic experience. The GMPS includes inputs (i.e.,
esty, and respect for the client (Ackerman & Hilsenroth, 2003). client and therapist factors), processes (i.e., the SR), and outputs
Yet, in one study, therapists’ theoretical orientation, gender, level (i.e., client and supervisee outcomes) as common factors in super-
of training, or years of experience did not contribute to client vision; thus, providing a theoretical lens for the current study. The
change (Okiishi et al., 2006). It is noteworthy that although the SWA is a component of the SR, and is defined as including a client
therapists in the Okiishi and colleagues’ study varied in experi- focus with a relational component (Bernard & Goodyear, 2019;
ence level, they had a higher caseload and more experience than Watkins, 2011, 2014, 2016). The SWA has been associated with
those measured in the current study. In fact, the authors reported various supervision outcomes for supervisees, such as lower levels
the model therapist was a male, doctoral level psychologist. As of burnout, higher satisfaction with supervision, and higher levels
such, a focus on novice therapists might have yielded different of self-efficacy (Livni, Crowe, & Gonsalvez, 2012; Watkins,
results. To elucidate this point, in a study by Mallinckrodt and 2014).
Nelson (1991), therapist training levels were found as significant Although the research findings examining the impact of super-
predictors in the ability to form agreement with the client on the vision on supervisee development is both extensive and continu-
tasks and goals in treatment, although no differences on bond were ously growing, the relationship between supervision and client
found. In fact, novice therapists may have a natural ability to form outcome is not well-understood (Callahan & Watkins, 2018c).
a bond with clients, but experience working with clients is needed Researchers have found moderate effects on client outcome as
to form the skills necessary to create goal and task agreement (Hill, related to supervisor qualities (Callahan, Almstrom, Swift, Borja,
Spiegel, Hoffman, Kivlighan, & Gelso, 2017). Huppert and col- & Heath, 2009; Wrape, Callahan, Ruggero, & Watkins, 2015).
leagues (2001) argued that mixed findings with regards to therapist Specifically focusing on the SWA, the correlation between the
effect may be due to the nature of standardized therapy (e.g., strength of the SWA and client outcomes (client meeting voca-
Cognitive Behavioral Therapy for anxiety), which has been found tional goals) was examined; however, no relationship was identi-
to decrease the effect of the therapist in comparison to nonstan- fied (McCarthy, 2013). Nevertheless, McCarthy’s data was not
dardized therapy on treatment outcome. A factor that might be matched between SWA scores and client outcomes, thus limiting
influencing these seemingly mixed results is the level of supervi- the study due to the lack of dyadic data analysis. In another study,
sion that novice therapists receive, which is a primary focus within Grossl, Reese, Norsworthy, and Hopkins (2014) found that the
the current study. In addition to these factors, clients and therapists SWA did not correlate with client outcomes (Outcome Rating
often rate the quality of the TA differently (DePue, Lambie, Liu, Scale; Miller & Duncan, 2000), yet the employed methods for the
& Gonzalez, 2016), and although both may predict client outcome, study (e.g., sampling, instrumentation) had significant limitations.
the differences between the client and therapist ratings indicate the On the other hand, Bambling, King, Raue, Schweitzer, and Lam-
need for additional research that includes both perspectives on bert (2006) conducted the first randomized control trial on the
outcomes (Bachelor, 2013). impact of clinical supervision with supervised or unsupervised
Significant debate exists over the directionality of the TA— therapists on the TA and client symptom reduction of clients
does TA predict client outcome or does client outcome predict diagnosed with major depression and found (a) clients with ther-
TA? To help answer this significant question, researchers exam- apists who were supervised rated the TA higher, (b) a reduction in
 SUPERVISORY RELATIONSHIP AND CLIENT OUTCOMES 255

symptoms of depression occurred, and (c) clients had higher treat- University-Based Community Counseling and Research Center
ment satisfaction than those who had therapists who were not (UBCCRC), which provides free counseling services to over 1,400
supervised. Hence, although the SR is important in supervision clients per year within a large, metropolitan area, while also
outcomes (i.e., promoting supervisee self-efficacy, clinical skill engaging in research. Although presenting concerns were not
development, coping skills; Watkins, 2014), the specific role of the systematically reported in the database at that time, the UBCCRC
SWA within the promoting therapeutic outcomes remains unclear. does not treat severe mental illness, clients in current domestic
We encourage readers to review Callahan and Watkins (2018a, abuse or crisis situations, or clients seeking addiction services.
2018b, and 2018c) works for thorough reviews of the current status From Spring 2015–2016, additional data was collected at the
of supervision theory and research. The authors call for additional UBCCRC from one of the primary investigators for a separate
research in various disciplines, as supervision has not been found project, and although these participants were not a part of our
effective in varying helping professions. study, we have included some of this data to be considered as a
Supervision may be most effective for novice therapists due to reference point on common clinical presentations and demograph-
factors such as high anxiety and a lack of foundational knowledge ics of clients and supervisors within the clinic. In that timeframe,
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

(Callahan & Watkins, 2018a). Due to the developmental nature of clients seen at the UBCCRC were primarily treated for the fol-
This document is copyrighted by the American Psychological Association or one of its allied publishers.

the novice therapists in our study, we postulate that the SWA is lowing four categories: mood disorders (n ⫽ 107, 43.5%), rela-
fundamental in helping them grow. Specifically, when therapists tionship issues (n ⫽ 37, 15%), transition/adjustment concerns (n ⫽
have a negative experience in therapy, supervision offers a space 61, 24.8%), and prior trauma/abuse (n ⫽ 17, 6.9%). For this study,
where novice therapists may process their clinical experiences to only practicum classes and clients from those classes were re-
improve the TA (Nissen-Lie, Havik, Høglend, Rønnestad, & Mon- cruited to control for level of therapist preparation. Novice thera-
sen, 2015). The SWA may also provide the opportunity to develop pists completed 36 graduate hours of counseling coursework prior
through constructive criticism, which may not be as well received to beginning their practicum experience (e.g., Theories of Coun-
for those with a negative SR (Rieck, Callahan, & Watkins, 2015). seling and Personality, Treatment and Diagnosis of Psychiatric
As such, we aimed to answer the following two primary research Disorders, and Techniques of Counseling). Practicum was the first
questions: (a) Does the TA predict client outcome change scores in time that the novice trainees had worked with real clients in a
novice therapists? (b) Does the TA mediate the relationship be- clinical setting. Practicum is scheduled as a weekly class lasting
tween the SWA and client outcome change scores in novice four hours in the UBCCRC and includes (a) group supervision, (b)
therapists? To answer the research questions, we hypothesized a three therapy appointments scheduled for each student by clinic
conceptual model depicted by the path diagrams presented in staff, and (c) weekly individual/triadic supervision. As such, over
Figure 1. In the model, we hypothesized that (a) both TA and SWA the academic semester, supervisees would have appropriately 15 hr
would have a significant direct effect on client outcome change of triadic supervision and 22.5 hr of group supervision with their
scores, and (b) TA would be a significant mediator between SWA supervisors. A faculty member accompanied by a doctoral student
and client outcome change scores. Because the SWA is a tran- leads the class with live supervision, and either the faculty member
stheoretical concept, supervisors and researchers may utilize the or the doctoral student provides additional individual supervision
results of this study to inform how elements of supervision impact outside of class time, which is not manualized. As such, faculty
both trainee and client outcomes, regardless of the specific tech- members and doctoral students vary in education, training, and
niques being used. backgrounds, as well as supervision approaches.
After receiving institutional review board (IRB) approval, clinic
Method staff introduced the study to the novice therapists in the first
practicum class and explained the two aspects of the study. It is
Procedure noteworthy that a larger IRB was in place for clinic data collection
procedures (i.e., client outcome data). As such, clients agreed to
This study employed a correlational research design as part of a participate in research as part of regular clinic procedures. How-
larger research study in a therapist preparation program accredited ever, we created a separate IRB, as the protocol for this study was
by the Council for Accreditation of Counseling and Related Edu- outside of typical clinic data collection procedures. First, the study
cational Programs at a large, southeastern university in the United included collecting data on the TA between the client and thera-
States. This study responds to Callahan and Watkins (2018c) call pist. All adult clients receiving individual counseling were re-
for discipline-specific supervision research. The therapist trainees’ cruited for the study by the trained novice therapists, and clients
graduate level practicum experience is housed within the were referred to the research team for any questions. To avoid
clients feeling pressured to take the survey, the following measures
were taken: (a) clients were presented with the questionnaire along
with an envelope that could be sealed for privacy, (b) clients were
informed their participation was voluntary and had no impact on
counseling services, (c) clients were informed their therapist would
never know whether the assessment was completed because ther-
apists were not study investigators, (d) clients were informed to
drop their sealed envelope into a locked box in the clinic that only
the investigators could retrieve, (e) clients left the clinic without
Figure 1. Conceptual model of the relationship among client outcome, the therapist present to maintain privacy during the time they
therapeutic alliance, and supervisory working alliance. would drop the envelope in the box, (f) clients were given the
 256 DEPUE, LIU, LAMBIE, AND GONZALEZ

informed consent document in their first session to review, which 31, 20.0%). The client sample included 123 women (63.7%) and
included the IRB information and primary investigator’s informa- 70 men (36.3%). The mean age of the client participants was 33.6
tion, (g) clients were referred to the investigators with any ques- years (SD ⫽ 11.97).
tions about the study, and (h) clients were also told they could Although demographics were not systematically reported in the
withdraw from the study at any point directly through the inves- clinic at the time of this study, demographics were collected from
tigators, so the clinician would remain blind to client participation. one of the primary investigators for a separate project for adult
In addition, the therapist left the room while clients took the clients between spring of 2015–2016 and are considered to be
survey. Two investigators were clinic staff members, and their representative of general trends within the clinic, as the gender
relationships with students were in the context of clinic procedures categories strongly match our data: male (n ⫽ 88, 35.8%); female
(i.e., scheduling). As such, these investigators were present during (n ⫽ 157, 63.8%); White (n ⫽ 116, 47.2%); African American
recruitment procedures and available for student and client ques- (n ⫽ 32, 13%), Hispanic (n ⫽ 38, 15.4%), Asian (n ⫽ 8, 3.3%),
tions. One investigator was a faculty member at the respective multiracial (n ⫽ 15, 6.1%), and other (n ⫽ 14, 5.7%), and no racial
institution but did not have interactions with practicum students. data (n ⫽ 23, 9.3%). In addition, data from supervisors was not
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Lastly, the final investigator had no affiliation with the institution collected for the present study because no data was collected
This document is copyrighted by the American Psychological Association or one of its allied publishers.

and did not have relationships with the students. If clients agreed directly from supervisors; however, we can use the same dataset
to participate, they were given a Working Alliance Inventory-Short from 2015–2016 as a reference point. In that time, there were a
Form (WAI-S) to fill out on the TA after the third therapy session total of 16 supervisors that self-reported as male (n ⫽ 5, 31.25%)
without the counselor in the room. We chose the third session to or female (n ⫽ 11, 68.75%) and were ages 20 –29 (n ⫽ 1, 6.25%),
provide information on early TA formation; the third session is 30 –39 (n ⫽ 6, 37.5%), 40 – 49 (n ⫽ 2, 12.5%), 50 –59 (n ⫽ 4,
also a frequently used data collection point for the TA (e.g., 25%), 60 – 69 (n ⫽ 1, 6.25%), or 70 –79 (n ⫽ 2, 12.5%). Also,
Hersoug, Monsen, Havik, & Høglend, 2002; Smits, Luyckx, supervisors self-reported as White (n ⫽ 8, 50%), African Ameri-
Smits, Stinckens, & Claes, 2015). If therapists agreed to partici- can (n ⫽ 3, 18.75%), Hispanic (n ⫽ 2, 12.5%), Asian (n ⫽ 1,
pate, they simultaneously filled out a WAI-S, and both the client 6.25%), and other (n ⫽ 2, 12.5%).
and therapist dropped completed forms into sealed envelopes in a
locked box within the UBCCRC. Only the primary investigators
had the key to the mailbox. Second, the study included a SWA Instrumentation
assessment that was completed by the supervisees on their indi-
vidual supervision experience. Participants were made aware that SWAI-T. The SWA was assessed using the Supervisory
the investigators were not involved in grading for the course, and Working Alliance Inventory-Trainee Version (SWAI-T) (Efsta-
instructors were not aware of whether supervisees participated in tion, Patton, & Kardash, 1990), which measures the supervisees’
the investigation or not. Lastly, clinic staff retrieved archival data perspective of the supervisor-supervisee relationship based on two
from the UBCCRC, and the research team identified and matched domains: (a) Rapport and (b) Client Focus. Created from the
clients’ Outcome Questionnaire 45.2 (OQ-45.2) scores (Lambert et Supervisory Styles Inventory (Friedlander & Ward, 1984), the
al., 2004) with data collection instruments. The UBCCRC does not SWAI-T rates 19 items (e.g., “My supervisor makes the effort to
distinguish clients’ OQ-45.2 subscale scores; therefore, we used understand me”) on a Likert scale ranging from 1 (almost never) to
clients’ OQ-45.2 total scores. At that time, the UBCCRC did not 7 (almost always), with higher scores indicating a stronger SWA.
systematically collect demographic data; therefore, client demo- The SWAI-T was tested with a diverse sample of 178 supervisees
graphic information is limited. with mean scores as follows: (a) Rapport (M ⫽ 5.85, SD ⫽ .83)
and (b) Client Focus (M ⫽ 5.44, SD ⫽ .44). Further, the SWAI-T
yielded acceptable Cronbach’s alpha scores: (a) Rapport (.90), and
Participants (b) Client Focus (.77). These SWAI-T scores were consistent with
Responses were collected from 155 novice therapist supervisees the data from this study, where Cronbach’s alphas for our data
and 193 clients from the UBCCRC. Although clients could be were as follows: (a) Client (.95), and (b) Client Focus (.89).
theoretically cross-classified with supervisees using hierarchical WAI-S. The WAI-S (Tracey & Kokotovic, 1989) consists of
linear modeling techniques, we only had on average of one or two both a client and therapist version, measuring the strength of the
clients at the first level of the model cross-classified with each TA on three related therapy domains: (a) Tasks, four items mea-
therapist supervisee, which hindered us from performing hierar- suring the relevancy and effectiveness of the therapists’ tasks; (b)
chical linear modeling analysis (Snijders, 2005). Therefore, we Bond, four items measuring the affective bond between client and
continued with typical structural equation model analysis in the therapist; and (c) Goals, four items measuring the endorsement of
rest of the study where 193 therapist– client dyads were matched. the outcome. The WAI-S includes 12 questions (four per domain)
The novice therapist sample included 138 women (89.0%) and rated on a Likert scale ranging from 1 (seldom) to 7 (always)
17 men (11.0%). The mean age of therapist participants was 27.3 without a neutral option for both the client and therapist versions
years (SD ⫽ 6.8). Much of the therapist sample self-identified as (e.g., Client: “I feel that my therapist appreciates me”; Therapist “I
White (103 participants, 66.5%); additional participants identified appreciate my client as a person”). Higher WAI-S scores indicate
as African American (16 participants, 10.3%), Hispanic (14 par- a stronger TA. The Cronbach’s alpha scores for the current study
ticipants, 9.0%), Asian American (11 participants, 7.1%), and for the client version were as follows: Task (.91), Bond (.89), Goal
other (11 participants, 7.1%). The therapists reported their track as (.83), and Total (.95). The Cronbach’s alpha scores for the current
clinical mental health therapy (n ⫽ 84, 54.2%); marriage, couple, study for the therapist version were as follows: Task (.89), Bond
and family therapy (n ⫽ 40, 25.8%); and school counseling (n ⫽ (.82), Goal (.90), and Total (.92).
 SUPERVISORY RELATIONSHIP AND CLIENT OUTCOMES 257

OQ-45.2. Client outcomes were assessed using the self-report Table 1

OQ-45.2 (Lambert et al., 2004), which was designed to track client Univariate Descriptive Statistics
progress in therapy, measuring the level of psychological distur-
bance. The OQ-45.2 contains 45 questions (e.g., “I have difficulty Measures M SD
concentrating”) rated on a Likert scale ranging from 1 (never) to 5 OQ.45.2 4.622 14.525
(almost always) under three domains: (a) Symptom Distress, (B) WAI-S
Interpersonal Relationships, and (C) Social Role. Higher scores Clients’ responses
indicate greater levels of client distress; as such, higher levels of Task 24.513 3.130
Bond 24.601 3.192
the OQ-45.2 relate to worsening symptomology. For this study, Goal 24.606 2.619
change scores were calculated by subtracting initial OQ-45.2 total Therapists’ responses
scale scores with final OQ-45.2 total scale scores, like previous Task 21.285 3.962
research (Okiishi et al., 2006). The OQ-45.2 has evidence of Bond 22.142 3.231
Goal 21.833 3.447
reliability as a total scale score (.96; Lambert et al., 2004). In the
SWAI-T 11.621 1.813
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

current study, Cronbach alphas were not calculated, due to the lack
Note. OQ.45.2 ⫽ Outcome Questionnaire 45.2; WAI-S ⫽ Working Al-
This document is copyrighted by the American Psychological Association or one of its allied publishers.

of availability of client individual responses on each item in the

liance Inventory-Short; SWAI-T ⫽ Supervisory Working Alliance-Trainee
archival data from the UBCCRC.
Version.

Statistical Analysis
Prior to data analysis, we checked multivariate normality and Measurement Models
relationships among variables. No case was identified as a univar-
iate and multivariate outlier. The two subscales of SWAI-T were Confirmatory factory analysis was conducted to examine and
found to be highly correlated, r ⫽ .92, p ⬍ .01; therefore, we test the measurement models for WAI-S scores for the client
combined the two scales, aligning with previous studies (DePue et and therapist responses. The models were just-identified after
al., 2016). The Cronbach’s alpha for the total SWAI-T was .92. we fixed the variance of the latent variable at one because there
Next, two structural equation models were developed to investi- were only three observed variables for the latent variable,
gate the hypothesized contribution of the TA and SWA on client meaning that all the parameters are uniquely solved because
outcome. Both models are realizations of the conceptual model there is just enough information. The fit indices were not
presented in Figure 1. The difference between the two models is applicable for just-identified models because those indices were
that Model 1 used clients’ responses to the WAI-S, whereas Model used to determine the distance between the proposed model and
2 used the therapists’ WAI-S responses to represent the TA. For the data (Schumacker & Lomax, 2004). Given that the mea-
the procedure of modeling, we followed the two-step approach surement models were identified, and the purpose of this study
suggested by Anderson and Gerbing (1988). The first step is to fit is to examine the structural component of the models, we
the measurement model to the observed variables, and the second continue to present the results for the structural models.
step is to fit the structural model to the relationship among latent
variables. The measurement component of both Model 1 and The Effects of TA and SWA on Client Outcomes
Model 2 was represented by using observed indicators: task, goal,
Fit indices for both structural models indicated adequate fit. In
and bond to measure TA. In the structural component, SWA was
Model 1, ␹2(4, 193) ⫽ 8.912, p ⫽ .063; RMSEA ⫽ .080, 90%
represented by the combined unidimensional supervisees’ re-
confidence interval [0.000, 0.151]; TLI ⫽ .953; CFI ⫽ .981,
sponses to the SWAI-T. Client outcome was represented by the
SRMR ⫽ .038, AIC ⫽ 4,235, BIC ⫽ 4,267. In Model 2, ␹2(4,
change in OQ45 scores between the first and last time of measure-
193) ⫽ 8.814, p ⫽ .066; RMSEA ⫽ .079; 90% confidence interval
ment. For the significance testing of TA’s indirect effects, the
[0.000, 0.151]; TLI ⫽ .962; CFI ⫽ .985, SRMR ⫽ .032, AIC ⫽
bootstrapping method was used because this approach does not
4,383, BIC ⫽ 4,415.
require a symmetric distribution and is recommended (Preacher &
The standardized loadings for Models 1 and 2 are presented in
Hayes, 2004). Both Model 1 and Model 2 used the maximum
Figure 2. In Model 1 (i.e., the model using clients’ responses for
likelihood (ML) estimator. Seven statistical fit indices were used to
the TA), the SWA did not have a direct effect on the TA (ß ⫽ .02,
evaluate the model-data fit: (a) the chi-square test (␹2), (b) the root
p ⫽ .59), but TA had a significant effect on client outcomes (ß ⫽
mean square error of approximation (RMSEA), (c) the compara-
.31, p ⫽ .00). The SWA also had a significant effect on client
tive fit index (CFI), (d) the Tucker–Lewis index (TLI), (e) the
outcomes (ß ⫽ .07, p ⫽ .05). In Model 2 (i.e., the model using
standardized root mean residual (SRMR), (f) Akaike information
therapists’ responses for the TA), the SWA had a significant effect
criterion (AIC), and (g) the Bayesian information criterion (BIC).
on the TA (ß ⫽ .15, p ⫽ .03), but it did not have a significant effect
on client outcomes (ß ⫽ .11, p ⫽ .12). The TA had a significant
Results
effect on client outcomes (ß ⫽ .17, p ⫽ .02). To summarize,
We displayed the univariate descriptive statistics in Table 1 and regardless of using clients’ or therapists’ responses to represent the
the correlation matrices of the two sets of variables in Table 2, so TA, the TA always had a significant direct effect on client out-
that readers can replicate our study if interested. The values for comes. However, the TA’s meditation effects in the two models
skewness and kurtosis are all between ⫺2 and ⫹2, which are were different. When clients’ responses were used to represent the
considered acceptable to satisfy the assumption of normality TA, the TA was not a significant mediator between the effects of
(George & Mallery, 2010). SWA on client outcomes. When therapists’ responses were used to
 258 DEPUE, LIU, LAMBIE, AND GONZALEZ

Table 2 TA. In other words, there was a linear relationship between the
Correlation Matrices of the Variables in Models 1 and 2 SWA, TA (therapist version), and client outcome, whereby ther-
apists/supervisees who viewed the SWA positively, also viewed
Model OQ.45.2 Task Bond Goal SWAI-T the TA positively, which in turn, predicted stronger client change
1. Client responses scores. Previous research has identified that therapists’ perception
OQ 45.2 1.000 of the SWA relates to their perception of the TA (DePue et al.,
WAI-S 2016); therefore, this mediating effect within the therapist model is
Task 0.262 1.000 logical. Differences in client and therapist perception models
Bond 0.280 0.737 1.000
Goal 0.309 0.611 0.506 1.000 might also be attributed to correlation effects. For example, Model
SWAI-T 0.136 0.012 0.058 0.110 1.000 1 measured the TA and SWA based on the clients’ and therapists’
2. Therapist responses perspectives on different variables, both having effects on client
OQ 45.2 1.000 outcomes, but given that the measures are from different perspec-
WAI-S
tives, the mediation effects might not have occurred as a result of
Task 0.162 1.000
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Bond 0.109 0.504 1.000 the varying perspectives in measurement. However, in Model 2,
This document is copyrighted by the American Psychological Association or one of its allied publishers.

Goal 0.181 0.821 0.607 1.000 both the TA and the SWA represented the therapists’ perspectives
SWAI-T 0.136 0.233 0.042 0.141 1.000 on different constructs; therefore, it was more likely that the
Note. OQ 45.2 ⫽ Outcome Questionnaire 45.2; SWAI-T ⫽ Supervisory measures were correlated, which is what our results indicated.
Working Alliance-Trainee Version.
Implications for Clinical Supervision

represent the TA, it fully mediated the effect of SWA on client Although it is unknown whether client symptomology improves
outcomes, meaning that the zero-order correlation between the as a result of the TA or if the TA is seen as positive due to client
SWA and client outcomes was completely explained by the indi- change (Webb, Beard, Auerbach, Menninger, & Björgvinsson,
rect effect through therapists’ responses for the strength of the TA. 2014), the TA is important in the therapeutic process. As such, we
encourage therapists to continue to foster the TA and consider it as
a fundamental component of effective therapy. Still, the effect
Discussion
sizes were small in our study; therefore, we argue there are
In our study, we identified a relationship between the SWA and additional, unknown variables that are also promoting client
client outcomes, although the nature of that relationship changed change scores over time. For example, attention toward repairing
based on whether the TA was measured using the client or ther- TA ruptures, may increase the TA and overall effectiveness of
apist perspectives. As clients and therapists often rate the quality of therapy (Stiles et al., 2004). The influence of the TA varied based
the TA differently (Bachelor, 2013; DePue et al., 2016), our results on whether the client or therapist was measuring the quality of the
indicate these differences in perception (i.e., clients often rate the TA. Clinicians can be mindful that ratings of clients differ from
TA higher than therapists) may be related to outcomes. Specifi- their own ratings of the TA, yet both predicted client outcomes.
cally, Model 1 included the client perspective of the TA, and there We consider the SR as a space to provide support and promote the
was a direct relationship between the SWA and client outcomes, as self-efficacy of novice counselors in trusting themselves and the
well as the TA and client outcomes. However, the TA (client relationships they are building with clients. Self-critiquing as a
version) did not mediate the relationship between the SWA and novice therapist is developmentally appropriate, thus adding to the
client outcome, as hypothesized. Rather, our results were congru- importance of a strong SWA to normalize the developmental
ent with previous studies that indicated the strength of the TA from aspects of becoming a therapist. Because therapist ratings of the
clients’ perception predicted positive client change scores (Ur- TA were related to their perceptions of the SWA in our sample, we
banoski, Kelly, Hoeppner, & Slaymaker, 2012). In addition, our encourage therapists to continue to foster positive alliances with
investigation was the first study to identify that a strong SWA, both their supervisors and clients. In fact, therapists may need to be
based on therapists’ viewpoint, was positively related to client diligent to monitor transference, countertransference, and the par-
change scores based on the clients’ viewpoint. Supervisor factors, allel processes that may be occurring within the TA and the SWA
such as length of time since obtaining a graduate degree, positively to help contribute to the quality of how these relationships work
correlate with client outcomes (Callahan et al., 2009; Wrape et al., together (Pearson, 2000). Yet, we highlight the power differential
2015); thus, it is logical that there is a relationship with client and hierarchical nature that exists within the SR; as such, the onus
outcomes based on the strength of the SWA. The SWA may be a is on the supervisor to create a safe space for supervisees (Kangos
function of individual therapist and supervisor factors, and addi- et al., 2018). Moreover, Kangos and colleagues (2018) offered
tional research is needed to study this relationship and its influence guidelines for supervisees to provide a sense of agency within the
on client change. One consideration is that the effect sizes were process. Some of their suggestions include the following: (a) a
small in our study, indicating there are additional factors impacting supervision contract, where expectations within supervision and
client outcomes that we did not measure in this study. evaluation methods are clearly delineated, (b) be active partici-
In Model 2, we tested therapists’ perspective of the TA within pants in the SWA by talking about the quality of supervision and
the identical models used in Model 1. Using therapists’ viewpoint the relationship throughout the process, and (c) educate themselves
of the TA, we found that the direct relationship of the SR on client on effective supervision, as this can be a point of reference for
change disappeared. Rather, the relationship between the SWA supervisees to discuss the quality of supervision with their super-
and client change was mediated by therapists’ perception of the visors. Although supervisors should initiate conversations about
 SUPERVISORY RELATIONSHIP AND CLIENT OUTCOMES 259
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
This document is copyrighted by the American Psychological Association or one of its allied publishers.

Figure 2. Standardized factor loadings for Models 1 and 2. SWAI-T ⫽ Supervisory Working Alliance
Inventory-Trainee; WAI-S ⫽ Working Alliance Inventory-Short; OQ.45.2 ⫽ Outcome Questionnaire 45.2. See
the online article for the color version of this figure.

the SR, they may not. As such, we encourage supervisees to be utilized in supervision and counseling. The importance of these
active participants within the SR, with well-informed expectations findings cannot be overstated: we must work to understand the
of supervision. commonalities within the SR that influence effective clinical prac-
The SWA and the TA were both related to client change scores tice and supervision across theories and disciplines.
in similar manners, in that the SWA measured the therapist/ Disruptions of the SWA are of importance to practicing super-
supervisee perception and the TA measured the client perspective. visors and therapists, as the SWA directly impacted client change
Clients’ perceptions of the TA are a more accurate measure of in our sample. The SWA is influenced by and related to a multi-
client outcome than therapists’ perceptions of the TA (Gelso et al., tude of factors including (a) supervisee self-efficacy and therapist
2012). In addition, our results identified that the SWA may have a maladaptive perfectionism (Ganske, Gnilka, Ashby, & Rice,
similar and parallel pattern of influence on client change based 2015), (b) supervisor multicultural competency (Crockett & Hays,
on supervisees’ viewpoints. These results may support Watkins 2015), (c) a negative or abrupt ending to the SR (Dawson &
(2018) common factors approach to supervision (i.e., GMPS). Akhurst, 2015), (d) supervisory styles (Fernando & Hulse-
Specifically, GMPS processes (i.e., the SWA) predicted outputs Killacky, 2005), and (e) stress of the supervisee (Gnilka, Chang, &
(i.e., client outcomes); supporting the integration of supervision Dew, 2012). Thus, we encourage supervisors to consider the noted
theory into common factors of supervision. As predictors of client influences of the quality of the SWA, and target supervision
change are at the forefront of helping professions, the utility in the strategies to meet the developmental needs of supervisees (Bernard
GMPS framework allows supervisors to focus on the commonal- & Goodyear, 2019; Inman et al., 2014). In addition, we suggest
ities in supervision (i.e., the SWA) that impact client outcomes, that supervisors be transparent with supervisees as ruptures occur
regardless of the specific techniques and theories utilized. As such, within the SWA. A negative supervision relationship might be
the mechanisms of change for both novice supervisees and clients considered as unhelpful, judgmental, and unaffirming, to name a
could lie somewhere within the commonalities of supervision few (Watkins, 2011, 2016), but can also be can be harmful (Ellis
inputs, processes, and outputs, rather than the specific techniques et al., 2014), thus resulting in negative supervisee emotional states
 260 DEPUE, LIU, LAMBIE, AND GONZALEZ

(Watkins, 2014). Although not studied here, these ruptures could was also only collected at a single time point, and because the
result in a cyclical effect: the supervisee is adversely impacted, the alliance can change over time, we encourage future research to
client is adversely impacted, and the supervisor is adversely im- consider using multiple time points for a more comprehensive
pacted, and so on. We argue it is supervisors’ responsibility to understanding of these relationships with client outcome.
work to discuss and repair ruptures as they occur, due to the Third, the models used in this study can be modified in at least
influence of the SWA on client outcomes. two ways. One way is to combine the two models as one, where
The common factors of counseling (Lambert & Barley, 2001) both the therapists’ rating and clients’ rating on the WAI-S are in
and supervision (Watkins, 2018) served as the theoretical link the same model. Unfortunately, this model is underidentified, and
between the TA and SR and the impact on client change scores in its parameter cannot be obtained in the current study. The other
the current study. Although similarities exist between the SR and way to modify the model is to introduce the initial OQ-45.2 score
the TA, the evaluative and liability issues surrounding the SR as a covariate; yet, the initial and last scores on OQ-45.2 are highly
make it unique from the TA and more challenging in some respects correlated (r ⫽ .85). To avoid collinearity issues, we subtracted the
(Bernard & Goodyear, 2019; Pearson, 2000). Yet, the qualities that scores to represent the change in one variable instead of two.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

continue to contribute to a strong TA can also contribute to a Lastly, social desirability and rater bias of both clients and thera-
This document is copyrighted by the American Psychological Association or one of its allied publishers.

strong SWA. These alliances are influenced by empathy, therapist pists is a limitation. Although, Reese and colleagues (2013) found
and supervisor congruence, and unconditional positive regard that client scores on the WAI-SR were not influenced by the
(Rogers, 1958). Although considering the notion of gatekeeping knowledge that the therapist would see the client’s ratings of the
within supervision, we encourage these same qualities within the TA, additional research is warranted to replicate these findings.
SR, while also maintaining transparency about the evaluative
methods with supervisees. In doing so, the supervisor can help
promote and facilitate the comfort of the supervisee to open up to References
the supervisor (Ladany, Hill, Corbett, & Nutt, 1996). Ackerman, S. J., & Hilsenroth, M. J. (2003). A review of therapist char-
acteristics and techniques positively impacting the therapeutic alliance.
Limitations and Future Directions Clinical Psychology Review, 23, 1–33. http://dx.doi.org/10.1016/S0272-
7358(02)00146-0
Our study had notable limitations, including the following: (a) Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in
sample, (b) instrumentation, (c) model, and (d) social desirability. practice: A review and recommended two-step approach. Psychological
First, the sample of this study only included one UBCCRC site; Bulletin, 103, 411– 423. http://dx.doi.org/10.1037/0033-2909.103.3.411
therefore, results should be interpreted with caution. Future re- Bachelor, A. (2013). Clients’ and therapists’ views of the therapeutic
search should expand the study to include UBCCRCs from across alliance: Similarities, differences and relationship to therapy outcome.
Clinical Psychology & Psychotherapy, 20, 118 –135. http://dx.doi.org/
the United States to verify and provide generalizable results. In
10.1002/cpp.792
addition, our sample only included novice therapists; therefore, not
Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling the
only were we unable to compare experience levels, but this also alliance-outcome correlation: Exploring the relative importance of ther-
limits generalizability. We encourage future research to expand apist and patient variability in the alliance. Journal of Consulting and
this study to include more experienced clinicians for points of Clinical Psychology, 75, 842– 852. http://dx.doi.org/10.1037/0022-006X
comparison on all measures. In addition, the sample consisted of .75.6.842
predominately White women, which also limits external validity. Bambling, M., King, R., Raue, P., Schweitzer, R., & Lambert, W. (2006).
The client sample was also a limitation due to the lack of demo- Clinical supervision: Its influence on client-rated working alliance and
graphical data collected by the UBCCRC. In addition, a limitation client symptom reduction in the brief treatment of major depression.
on controls was the variation in supervisor experience and modal- Psychotherapy Research, 16, 317–331. http://dx.doi.org/10.1080/
ity. As research is growing in the area of exceptional supervisors 10503300500268524
Bernard, J. M., & Goodyear, R. K. (2019). Fundamentals of clinical
(Callahan & Watkins, 2018c), we suggest that future research
supervision (6th ed.). New York, NY: Pearson.
include data from both supervisors and trainees to measure how
Callahan, J. L., Almstrom, C. M., Swift, J. K., Borja, S. E., & Heath, C.
these differences might impact results, as well as factors that may (2009). Exploring the contribution of supervisors to intervention out-
contribute to exceptional supervision. Second, this study utilized comes. Training and Education in Professional Psychology, 3, 72–77.
the short version of the WAI. There is a newer version of the WAI http://dx.doi.org/10.1037/a0014294
with greater evidence of construct validity (WAI-SR; Hatcher & Callahan, J. L., & Watkins, C. E., Jr. (2018a). Evidence-based training: The
Gillaspy, 2006); however, research testing the psychometric fea- time has come. Training and Education in Professional Psychology, 12,
tures of the short versions of the WAI is limited. We suggest future 211–218. http://dx.doi.org/10.1037/tep0000204
research to focus on both the full and short versions of the WAI to Callahan, J. L., & Watkins, C. E., Jr. (2018b). The science of training II:
verify the soundness of the short version scores. In addition, the Prepracticum and practicum training. Training and Education in Pro-
study did not use the subscale scores of the OQ-45.2; therefore, fessional Psychology, 12, 231–244. http://dx.doi.org/10.1037/
tep0000209
limiting more descriptive knowledge on the nature of the client
Callahan, J. L., & Watkins, C. E., Jr. (2018c). The science of training III:
changes. It is noteworthy that Callahan and Watkins (2018c)
Supervision, competency, and internship training. Training and Educa-
warned researchers about the risk of Type II errors due to a lack of tion in Professional Psychology, 12, 245–261. http://dx.doi.org/10.1037/
statistical client change scores. Our sample included significant tep0000208
client change scores measured by the total scores of the OQ-45.2, Crockett, S., & Hays, D. G. (2015). The influence of supervisor multicul-
and we encourage future researchers to consider these Type II tural competence on the supervisory working alliance, supervisee coun-
errors as they conduct and evaluate supervision research. The WAI seling self-efficacy, and supervisee satisfaction with supervision: A
 SUPERVISORY RELATIONSHIP AND CLIENT OUTCOMES 261

mediation model. Counselor Education and Supervision, 54, 258 – Huppert, J. D., Bufka, L. F., Barlow, D. H., Gorman, J. M., Shear, M. K.,
273. http://dx.doi.org/10.1002/ceas.12025 & Woods, S. W. (2001). Therapists, therapist variables, and cognitive-
Dawson, D., & Akhurst, J. (2015). “I wouldn’t dream of ending with a behavioral therapy outcome in a multicenter trial for panic disorder.
client in the way he did to me”: An exploration of supervisees’ experi- Journal of Consulting and Clinical Psychology, 69, 747–755. http://dx
ences of an unplanned ending to the supervisory relationship. Counsel- .doi.org/10.1037/0022-006X.69.5.747
ling & Psychotherapy Research, 15, 21–30. http://dx.doi.org/10.1080/ Inman, A. G., Hutman, H., Pendse, A., Devdas, L., Luu, L., & Ellis,
14733145.2013.845235 M. V. (2014). Current trends concerning supervisors, supervisees,
Del Re, A. C., Flückiger, C., Horvath, A. O., Symonds, D., & Wampold, and clients in clinical supervision. In C. E. Watkins, Jr., & D. L. Milne
B. E. (2012). Therapist effects in the therapeutic alliance-outcome rela- (Eds.), The Wiley international handbook of clinical supervision (pp.
tionship: A restricted-maximum likelihood meta-analysis. Clinical Psy- 61–102). West Sussex, United Kingdom: Wiley. http://dx.doi.org/10
chology Review, 32, 642– 649. http://dx.doi.org/10.1016/j.cpr.2012.07 .1002/9781118846360.ch4
.002 Inman, A. G., & Ladany, N. (2008). Research: The state of the field. In
DePue, M. K., Lambie, G. W., Liu, R., & Gonzalez, J. (2016). Investigat- A. K. Hess, K. D. Hess, & T. H. Hess (Eds.), Psychotherapy supervi-
ing supervisory relationships and therapeutic alliance using structural sion: Theory, research, and practice (2nd ed., pp. 500 –517). Hoboken,
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

equation modeling. Counselor Education and Supervision, 55, 263–277. NJ: Wiley. Retrieved from http://search.ebscohost.com.lp.hscl.ufl.edu/
This document is copyrighted by the American Psychological Association or one of its allied publishers.

http://dx.doi.org/10.1002/ceas.12053 login.aspx?direct⫽true&AuthType⫽ip,uid&db⫽psyh&AN⫽2008-
Efstation, J. F., Patton, M. J., & Kardash, C. M. (1990). Measuring the 12978-026&site⫽ehost-live
working alliance in counselor supervision. Journal of Counseling Psy- Kangos, K. A., Ellis, M. V., Berger, L., Corp, D. A., Hutman, H., Gibson,
chology, 37, 322–329. http://dx.doi.org/10.1037/0022-0167.37.3.322 A., & Nicolas, A. I. (2018). American Psychological Association guide-
Ellis, M. V., Berger, L., Hanus, A. E., Ayala, E. E., Swords, B. A., & lines for clinical supervision: Competency-based implications for super-
Siembor, M. (2014). Inadequate and harmful clinical supervision: visees. The Counseling Psychologist, 46, 821– 845. http://dx.doi.org/10
Testing a revised framework and assessing occurrence. The Coun- .1177/0011000018807128
seling Psychologist, 42, 434 – 472. http://dx.doi.org/10.1177/ Ladany, N., Hill, C. E., Corbett, N. N., & Nutt, E. A. (1996). Nature,
0011000013508656 extent, and importance of what psychotherapy trainees do not disclose to
Fernando, D. M., & Hulse-Killacky, D. (2005). The relationship of super- their supervisors. Journal of Counseling Psychology, 43, 10 –24. http://
visory styles to satisfaction with supervision and the perceived self-
dx.doi.org/10.1037/0022-0167.43.1.10
efficacy of master’s-level counseling students. Counselor Education and
Lambert, M. J., & Barley, D. E. (2001). Research summary on the thera-
Supervision, 44, 293–304. http://dx.doi.org/10.1002/j.1556-6978.2005
peutic relationship and psychotherapy outcome. Psychotherapy: Theory,
.tb01757.x
Research, & Practice, 38, 357–361. http://dx.doi.org/10.1037/0033-
Friedlander, M. L., & Ward, L. G. (1984). Development and validation of
3204.38.4.357
the supervisory styles inventory. Journal of Counseling Psychology, 31,
Lambert, M. J., Morton, J. S., Hatfield, D., Harmon, C., Hamilton, S., Reid,
541–557. http://dx.doi.org/10.1037/0022-0167.31.4.541
R. C., . . . Burlingame, G. M. (2004). Administration and scoring manual
Ganske, K. H., Gnilka, P. B., Ashby, J. S., & Rice, K. G. (2015). The
or the OQ-45.2 (Outcome Questionnaire; 3rd ed.). Wilmington, DE:
relationship between counseling trainee perfectionism and the working
American Professional Credentialing Services.
alliance with supervisor and client. Journal of Counseling and Devel-
Lei, Y., & Duan, C. (2014). Relationship between therapist empathy and
opment, 93, 14 –24. http://dx.doi.org/10.1002/j.1556-6676.2015.00177.x
client-perceived working alliance in China: A multilevel modeling anal-
Gelso, C. J., Kivlighan, D. M., Jr., Busa-Knepp, J., Spiegel, E. B., Ain, S.,
ysis. Counselling Psychology Quarterly, 27, 200 –215. http://dx.doi.org/
Hummel, A. M., . . . Markin, R. D. (2012). The unfolding of the real
10.1080/09515070.2014.882812
relationship and the outcome of brief psychotherapy. Journal of Coun-
seling Psychology, 59, 495–506. http://dx.doi.org/10.1037/a0029838 Leibert, T. W., Smith, J. B., & Agaskar, V. R. (2011). Relationship
George, D., & Mallery, M. (2010). SPSS for Windows step by step: A between the working alliance and social support on counseling outcome.
simple guide and reference. Boston, MA: Pearson. Journal of Clinical Psychology, 67, 709 –719. http://dx.doi.org/10.1002/
Gnilka, P. B., Chang, C. V., & Dew, B. J. (2012). The relationship between jclp.20800
supervisee stress, coping resources, the working alliance, and the super- Livni, D., Crowe, T. P., & Gonsalvez, C. J. (2012). Effects of supervision
visory working alliance. Journal of Counseling and Development, 90, modality and intensity on alliance and outcomes for the supervisee.
63–70. http://dx.doi.org/10.1111/j.1556-6676.2012.00009.x Rehabilitation Psychology, 57, 178 –186. http://dx.doi.org/10.1037/
Grossl, A. B., Reese, R. J., Norsworthy, L. A., & Hopkins, N. B. (2014). a0027452
Client feedback data in supervision: Effects on supervision and outcome. Malin, A. J., & Pos, A. E. (2015). The impact of early empathy on alliance
Training and Education in Professional Psychology, 8, 182–188. http:// building, emotional processing, and outcome during experiential treat-
dx.doi.org/10.1037/tep0000025 ment of depression. Psychotherapy Research, 25, 445– 459. http://dx.doi
Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a .org/10.1080/10503307.2014.901572
revised short version of the Working Alliance Inventory. Psychotherapy Mallinckrodt, B., & Nelson, M. L. (1991). Counselor training level and the
Research, 16, 12–25. http://dx.doi.org/10.1080/10503300500352500 formation of the psychotherapeutic working alliance. Journal of Coun-
Hersoug, A. G., Monsen, J. T., Havik, O. E., & Høglend, P. (2002). Quality seling Psychology, 38, 133–138. http://dx.doi.org/10.1037/0022-0167
of early working alliance in psychotherapy: Diagnoses, relationship and .38.2.133
intrapsychic variables as predictors. Psychotherapy and Psychosomatics, McCarthy, A. K. (2013). Relationship between supervisory working alli-
71, 18 –27. http://dx.doi.org/10.1159/000049340 ance and client outcomes in state vocational rehabilitation counseling.
Hill, C. E., Spiegel, S. B., Hoffman, M. A., Kivlighan, D. M., Jr., & Rehabilitation Counseling Bulletin, 57, 23–30. http://dx.doi.org/10
Gelso, C. J. (2017). Therapist expertise in psychotherapy revisited. .1177/0034355213484177
The Counseling Psychologist, 45, 7–53. http://dx.doi.org/10.1177/ Miller, S. D., & Duncan, B. L. (2000). The Outcome Rating Scale.
0011000016641192 Chicago, IL: Author.
Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011). Nissen-Lie, H. A., Havik, O. E., Høglend, P. A., Rønnestad, M. H., &
Alliance in individual psychotherapy. Psychotherapy: Theory, Research, Monsen, J. T. (2015). Patient and therapist perspectives on alliance
& Practice, 48, 9 –16. http://dx.doi.org/10.1037/a0022186 development: Therapists’ practice experiences as predictors. Clinical
 262 DEPUE, LIU, LAMBIE, AND GONZALEZ

Psychology & Psychotherapy, 22, 317–327. http://dx.doi.org/10.1002/ Tracey, T. J., & Kokotovic, A. M. (1989). Factor structure of the Working
cpp.1891 Alliance Inventory. Journal of Consulting and Clinical Psychology, 1,
Norcross, J. C. (Ed.). (2011). Psychotherapy relationships that work: 207–210. http://dx.doi.org/10.1037/1040-3590.1.3.207
Evidence-based responsiveness. New York, NY: Oxford University Urbanoski, K. A., Kelly, J. F., Hoeppner, B. B., & Slaymaker, V. (2012).
Press. http://dx.doi.org/10.1093/acprof:oso/9780199737208.001.0001 The role of therapeutic alliance in substance use disorder treatment for
Okiishi, J. C., Lambert, M. J., Eggett, D., Nielsen, L., Dayton, D. D., & young adults. Journal of Substance Abuse Treatment, 43, 344 –351.
Vermeersch, D. A. (2006). An analysis of therapist treatment effects: http://dx.doi.org/10.1016/j.jsat.2011.12.013
Toward providing feedback to individual therapists on their clients’ Watkins, C. E., Jr. (2011). Does psychotherapy supervision contribute to
psychotherapy outcome. Journal of Clinical Psychology, 62, 1157– patient outcomes? Considering thirty years of research. The Clinical
1172. http://dx.doi.org/10.1002/jclp.20272 Supervisor, 30, 235–256. http://dx.doi.org/10.1080/07325223.2011
Patterson, C. L., Anderson, T., & Wei, C. (2014). Clients’ pretreatment role .619417
expectations, the therapeutic alliance, and clinical outcomes in outpa- Watkins, C. E., Jr. (2014). The supervisory alliance: A half century of
tient therapy. Journal of Clinical Psychology, 70, 673– 680. http://dx.doi theory, practice, and research in critical perspective. American Jour-
.org/10.1002/jclp.22054 nal of Psychotherapy, 68, 19 –55. http://dx.doi.org/10.1176/appi
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Pearson, Q. M. (2000). Opportunities and challenges in the supervisory .psychotherapy.2014.68.1.19

Watkins, C. E., Jr. (2016). A unifying vision of psychotherapy supervision:
This document is copyrighted by the American Psychological Association or one of its allied publishers.

relationship: Implications for counselor supervision. Journal of Mental

Health Counseling, 22, 283–294. Part I. Productive and unproductive supervision relationships. Journal of
Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for Unified Psychotherapy and Clinical Science, 4, 21–35.
estimating indirect effects in simple mediation models. Behavior Re- Watkins, C. E., Jr. (2018). The generic model of psychotherapy supervi-
search Methods, Instruments, & Computers, 36, 717–731. http://dx.doi sion: An analogized research-informing meta-theory. Journal of Psycho-
.org/10.3758/BF03206553 therapy Integration, 28, 521–536. http://dx.doi.org/10.1037/int0000114
Reese, R. J., Gillaspy, J. A., Jr., Owen, J. J., Flora, K. L., Cunningham, Watkins, C. E., Jr., & Callahan, J. L. (2019). Psychotherapy supervision
L. C., Archie, D., & Marsden, T. (2013). The influence of demand research: A status report and proposed model. In S. G. De Golia & K. M.
characteristics and social desirability on clients’ ratings of the therapeu- Corcoran (Eds.), Supervision in psychiatric practice: Practical ap-
tic alliance. Journal of Clinical Psychology, 69, 696 –709. http://dx.doi proaches across venues and providers (pp. 25–34). Washington, DC:
.org/10.1002/jclp.21946 American Psychiatric Association.
Rieck, T., Callahan, J. L., & Watkins, C. E., Jr. (2015). Clinical supervi- Webb, C. A., Beard, C., Auerbach, R. P., Menninger, E., & Björgvinsson,
sion: An exploration of possible mechanisms of action. Training and T. (2014). The therapeutic alliance in a naturalistic psychiatric setting:
Education in Professional Psychology, 9, 187–194. http://dx.doi.org/10 Temporal relations with depressive symptom change. Behaviour Re-
.1037/tep0000080 search and Therapy, 61, 70 –77. http://dx.doi.org/10.1016/j.brat.2014.07
Rogers, C. R. (1958). The characteristics of a helping relationship. The .015
Personnel and Guidance Journal, 37, 6 –16. http://dx.doi.org/10.1002/j Weck, F., Grikscheit, F., Jakob, M., Höfling, V., & Stangier, U. (2015).
.2164-4918.1958.tb01147.x Treatment failure in cognitive-behavioural therapy: Therapeutic alliance
Schumacker, R., & Lomax, R. (2004). A beginner’s guide to structural as a precondition for an adherent and competent implementation of
equation modeling (2nd ed.). Mahwah, NJ: Erlbaum. http://dx.doi.org/ techniques. British Journal of Clinical Psychology, 54, 91–108. http://
10.4324/9781410610904 dx.doi.org/10.1111/bjc.12063
Smits, D., Luyckx, K., Smits, D., Stinckens, N., & Claes, L. (2015). Wrape, E. R., Callahan, J. L., Ruggero, C. J., & Watkins, C. J. (2015). An
Structural characteristics and external correlates of the Working Alliance exploration of faculty supervisor variables and their impact on client
Inventory–Short Form. Psychological Assessment, 27, 545–551. http:// outcomes. Training and Education in Professional Psychology, 9, 35–
dx.doi.org/10.1037/pas0000066 43. http://dx.doi.org/10.1037/tep0000014
Snijders, T. A. (2005). Encyclopedia of statistics in behavioral science: Zilcha-Mano, S., Dinger, U., McCarthy, K. S., & Barber, J. P. (2014). Does
Vol. 3. Power and sample size in multilevel modeling (pp. 1570 –1573). alliance predict symptoms throughout treatment, or is it the other way
Chichester, UK: Wiley. around? Journal of Consulting and Clinical Psychology, 82, 931–935.
Stiles, W. B., Glick, M. J., Osatuke, K., Hardy, G. B., Shapiro, D. A., http://dx.doi.org/10.1037/a0035141
Agnew-Davies, R., . . . Barkham, M. (2004). Patterns of alliance devel-
opment and the Rupture—Repair Hypothesis: Are productive relation- Received June 10, 2019
ships u-shaped or v-shaped? Journal of Counseling Psychology, 51, Revision received February 24, 2020
81–92. http://dx.doi.org/10.1037/0022-0167.51.1.81 Accepted March 15, 2020 䡲