Objective: The novel coronavirus disease (COVID-19) has drastically impacted the provision of mental health services. Changes required of providers were substantial and could lead to increased burnout and, subsequently, increased turnover intentions. This study examined burnout experienced by mental health services providers in the context of COVID-19 and through the lens of the job demands-resources (JD-R) model. We examined the effects of work changes on burnout and subsequent turnover intentions, and how job and personal resources may have buffered the extent to which work changes due to COVID-19 impacted burnout. Methods: Service providers (n = 93) from six community mental health centers (CMHCs) in one Midwestern state in the United States completed surveys as part of service contracts to implement evidence-based practices. Path analysis tested the unconditional indirect relations between work changes and turnover intentions through burnout. Moderated mediation determined whether the indirect effect of work changes on turnover intentions via burnout varied in strength by job and personal resources. Results: Work changes had a significant indirect effect on turnover intentions through burnout ( β ^ = .140, 95% CI = .072, .217). This indirect effect varied as a function of two job resources, organizational trust and perceived organizational support. Conclusions and Implications for Practice: Burnout was relatively low only when work changes were low and job resources levels high. When work changes were high, burnout was similarly high across levels of job resources. To minimize burnout, organizations should limit task, setting, and team-related work changes to the extent possible. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

BACKGROUND: Implementation is proposed to be a multiphase, multilevel process. After a period of exploration, an adoption decision is made, typically at the upper management or policy level. Nevertheless, movement through each of the subsequent phases of the implementation process involves clinicians or providers at the individual level to adopt the innovation and then change their behavior to use/deliver the innovation. Multiple behavioral change theories propose that intentions are a critical determinant of implementation behavior. However, there is a need for the development and testing of pragmatic measures of providers intentions to use a specific innovation or evidence-based practice (EBP). METHODS: Nine items were developed to assess providers intentions to use a specific innovation or EBP. Motivational interviewing was the EBP in the study. Items were administered, as part of larger survey, to 179 providers across 38 substance use disorder treatment (SUDT) programs within five agencies in California, USA. Rasch analysis was conducted using RUMM2030 software to assess the items, their overall fit to the Rasch model, the response scale used, individual item fit, differential item functioning (DIF), and person separation. RESULTS: Following a stepwise process, the scale was reduced from nine items to three items to increase the feasibility and acceptability of the scale while maintaining suitable psychometric properties. The three-item unidimensional scale showed good person separation (PSI = .872), no disordering of thresholds, and no evidence of uniform or non-uniform DIF. Rasch analysis supported the viability of the scale as a measure of implementation intentions. CONCLUSIONS: The Measure of Innovation-Specific Implementation Intentions (MISII) is a sound measure of providers intentions to use a specific innovation or EBP. Future evaluation of convergent, divergent, and predictive validity are needed. The study also demonstrates the value of Rasch analysis for testing the psychometric properties of pragmatic implementation measures.

Background

In line with recent research on the role of the inner context of organizations in implementation effectiveness, this study extends research on organizational citizenship behavior (OCB) to the domain of evidence-based practice (EBP) implementation. OCB encompasses those behaviors that go beyond what is required for a given job that contribute to greater organizational effectiveness. The goal of this study was to develop and test a measure of implementation citizenship behavior (ICB) or those behaviors that employees perform that go above and beyond what is required in order to support EBP implementation.

Methods

The primary participants were 68 supervisors from ten mental health agencies throughout California. Items measuring ICB were developed based on past research on OCB and in consultation with experts on EBP implementation in mental health settings. Supervisors rated 357 of their subordinates on ICB and implementation success. In addition, 292 of the subordinates provided data on self-rated performance, attitudes towards EBPs, work experience, and full-time status. The supervisor sample was randomly split, with half used for exploratory factor analyses and the other half for confirmatory factor analyses. The entire sample of supervisors and subordinates was utilized for analyses assessing the reliability and construct validity of the measure.

Results

Exploratory factor analyses supported the proposed two-factor structure of the Implementation Citizenship Behavior Scale (ICBS): (1) Helping Others and (2) Keeping Informed. Confirmatory factor analyses with the other half of the sample supported the factor structure. Additional analyses supported the reliability and construct validity for the ICBS.

Conclusions

The ICBS is a pragmatic brief measure (six items) that captures critical behaviors employees perform to go above and beyond the call of duty to support EBP implementation, including helping their fellow employees on implementation-related activities and keeping informed about issues related to EBP and implementation efforts. The ICBS can be used by researchers to better understand the outcomes of improved organizational support for implementation (i.e., implementation climate) and the proximal predictors of implementation effectiveness. The ICBS can also provide insight for organizations, practitioners, and managers by focusing on key employee behaviors that should increase the probability of implementation success.

Background

Although the importance of the organizational environment for implementing evidence-based practices (EBP) has been widely recognized, there are limited options for measuring implementation climate in public sector health settings. The goal of this research was to develop and test a measure of EBP implementation climate that would both capture a broad range of issues important for effective EBP implementation and be of practical use to researchers and managers seeking to understand and improve the implementation of EBPs.

Methods

Participants were 630 clinicians working in 128 work groups in 32 US-based mental health agencies. Items to measure climate for EBP implementation were developed based on past literature on implementation climate and other strategic climates and in consultation with experts on the implementation of EBPs in mental health settings. The sample was randomly split at the work group level of analysis; half of the sample was used for exploratory factor analysis (EFA), and the other half was used for confirmatory factor analysis (CFA). The entire sample was utilized for additional analyses assessing the reliability, support for level of aggregation, and construct-based evidence of validity.

Results

The EFA resulted in a final factor structure of six dimensions for the Implementation Climate Scale (ICS): 1) focus on EBP, 2) educational support for EBP, 3) recognition for EBP, 4) rewards for EBP, 5) selection for EBP, and 6) selection for openness. This structure was supported in the other half of the sample using CFA. Additional analyses supported the reliability and construct-based evidence of validity for the ICS, as well as the aggregation of the measure to the work group level.

Conclusions

The ICS is a very brief (18 item) and pragmatic measure of a strategic climate for EBP implementation. It captures six dimensions of the organizational context that indicate to employees the extent to which their organization prioritizes and values the successful implementation of EBPs. The ICS can be used by researchers to better understand the role of the organizational context on implementation outcomes and by organizations to evaluate their current climate as they consider how to improve the likelihood of implementation success.

The organizational context in which substance use disorder treatment (SUDT) evidence-based practices (EBPs) are implemented plays a critical role in successful implementation. Employee behaviors that go above and beyond typical job requirements to support EBP implementation have been suggested to facilitate the likelihood of overall implementation success. The current study explored the psychometric properties of the Implementation Citizenship Behavior Scale (ICBS) within SUDT settings. Utilizing a sample of 322 direct service providers and 60 of their respective supervisors from three SUDT agencies, results from a confirmatory factor analysis and construct validity analysis support the use of the ICBS in the SUDT context. Validation of the ICBS provides a useful, pragmatic tool for both researchers and practitioners to assess employee citizenship behavior to support EBP implementation. The ICBS can provide critical insights into how providers respond to organizational context that may facilitate EBP implementation.

Background

In healthcare and allied healthcare settings, leadership that supports effective implementation of evidenced-based practices (EBPs) is a critical concern. However, there are no empirically validated measures to assess implementation leadership. This paper describes the development, factor structure, and initial reliability and convergent and discriminant validity of a very brief measure of implementation leadership: the Implementation Leadership Scale (ILS).

Methods

Participants were 459 mental health clinicians working in 93 different outpatient mental health programs in Southern California, USA. Initial item development was supported as part of a two United States National Institutes of Health (NIH) studies focused on developing implementation leadership training and implementation measure development. Clinician work group/team-level data were randomly assigned to be utilized for an exploratory factor analysis (n = 229; k = 46 teams) or for a confirmatory factor analysis (n = 230; k = 47 teams). The confirmatory factor analysis controlled for the multilevel, nested data structure. Reliability and validity analyses were then conducted with the full sample.

Results

The exploratory factor analysis resulted in a 12-item scale with four subscales representing proactive leadership, knowledgeable leadership, supportive leadership, and perseverant leadership. Confirmatory factor analysis supported an a priori higher order factor structure with subscales contributing to a single higher order implementation leadership factor. The scale demonstrated excellent internal consistency reliability as well as convergent and discriminant validity.

Conclusions

The ILS is a brief and efficient measure of unit level leadership for EBP implementation. The availability of the ILS will allow researchers to assess strategic leadership for implementation in order to advance understanding of leadership as a predictor of organizational context for implementation. The ILS also holds promise as a tool for leader and organizational development to improve EBP implementation.

PURPOSE: Leadership in health care is instrumental to creating a supportive organizational environment and positive staff attitudes for implementing evidence-based practices to improve patient care and outcomes. The purpose of this study is to demonstrate the alignment of the Ottawa Model of Implementation Leadership (O-MILe), a theoretical model for developing implementation leadership, with the Implementation Leadership Scale (ILS), an empirically validated tool for measuring implementation leadership. A secondary objective is to describe the methodological process for aligning concepts of a theoretical model with an independently established measurement tool for evaluating theory-based interventions. METHODS: Modified template analysis was conducted to deductively map items of the ILS onto concepts of the O-MILe. An iterative process was used in which the model and scale developers (n=5) appraised the relevance, conceptual clarity, and fit of each ILS items with the O-MILe concepts through individual feedback and group discussions until consensus was reached. RESULTS: All 12 items of the ILS correspond to at least one O-MILe concept, demonstrating compatibility of the ILS as a measurement tool for the O-MILe theoretical constructs. CONCLUSION: The O-MILe provides a theoretical basis for developing implementation leadership, and the ILS is a compatible tool for measuring leadership based on the O-MILe. Used together, the O-MILe and ILS provide an evidence- and theory-based approach for developing and measuring leadership for implementing evidence-based practices in health care. Template analysis offers a convenient approach for determining the compatibility of independently developed evaluation tools to test theoretical models.

BACKGROUND: Behavioral health organizations are characterized by multiple organizational climates, including molar climate, which encompasses clinicians shared perceptions of how the work environment impacts their personal well-being, and strategic implementation climate, which includes clinicians shared perceptions of the extent to which evidence-based practice implementation is expected, supported, and rewarded by the organization. Theory suggests these climates have joint, cross-level effects on clinicians implementation of evidence-based practice and that these effects may be long term (i.e., up to 2 years); however, no empirical studies have tested these relationships. We hypothesize that molar climate moderates implementation climates concurrent and long-term relationships with clinicians use of evidence-based practice such that strategic implementation climate will have its most positive effects when it is accompanied by a positive molar climate. METHODS: Hypotheses were tested using data collected from 235 clinicians in 20 behavioral health organizations. At baseline, clinicians reported on molar climate and implementation climate. At baseline and at a 2-year follow-up, all clinicians who were present in the organizations reported on their use of cognitive-behavioral psychotherapy techniques, an evidence-based practice for youth psychiatric disorders. Two-level mixed-effects regression models tested whether baseline molar climate and implementation climate interacted in predicting clinicians evidence-based practice use at baseline and at 2-year follow-up. RESULTS: In organizations with more positive molar climates at baseline, higher levels of implementation climate predicted increased evidence-based practice use among clinicians who were present at baseline and among clinicians who were present in the organizations at 2-year follow-up; however, in organizations with less positive molar climates, implementation climate was not related to clinicians use of evidence-based practice at either time point. CONCLUSIONS: Optimizing clinicians implementation of evidence-based practice in behavioral health requires attention to both molar climate and strategic implementation climate. Strategies that focus exclusively on implementation climate may not be effective levers for behavior change if the organization does not also engender a positive molar climate. These findings have implications for the development of implementation theory and effective implementation strategies.

There has been a growing impetus to bridge the gap between basic science discovery, development of evidence-based practices (EBPs), and the availability and delivery of EBPs in order to improve the public health impact of such practices. To capitalize on factors that support implementation and sustainment of EBPs, it is important to consider that health care is delivered within the outer context of public health systems and the inner context of health care organizations and work groups. Leaders play a key role in determining the nature of system and organizational contexts. This article addresses the role of leadership and actions that leaders can take at and across levels in developing a strategic climate for EBP implementation within the outer (i.e., system) and inner (i.e., organization, work group) contexts of health care. Within the framework of Edgar Schein's "climate embedding mechanisms," we describe strategies that leaders at the system, organization, and work group levels can consider and apply to develop strategic climates that support the implementation and sustainment of EBP in health care and allied health care settings.