From Rhetoric To Routine: Assessing Perceptions of Recovery-Oriented Practices in A State Mental Health and Addiction System
From Rhetoric To Routine: Assessing Perceptions of Recovery-Oriented Practices in A State Mental Health and Addiction System
Maria O’Connell, Janis Tondora, The Recovery Self Assessment (RSA) was developed to gauge perceptions of the
Gerald Croog, Arthur Evans degree to which programs implement recovery-oriented practices. Nine hundred
& Larry Davidson and sixty-seven directors, providers, persons in recovery, and significant others
from 78 mental health and addiction programs completed the instrument. Factor
articles
378
spring 2005—Volume 28 Numb er 4
quality of life (Drake et al., 2001). Hatry, Newcomer & Wholey, 1994; citizenship activities (i.e., voting, pay-
Unfortunately, most people receiving Rouse, Toprac & MacCabe, 1998) if they ing taxes); managing symptoms; and
publicly delivered mental health servic- are to be successful in translating the being supported by others. Recovery-
es do not yet receive treatment that is science of recovery-oriented care to Oriented practices associated with
evidence-based (U.S. Department of routine psychiatric practice. each of these recovery principles were
Health and Human Services, 1999. then explored and articulated as a sec-
From Rhetoric to Routine
Drake et al., 2001; Lehman & ond step in this line of research. In
Many state mental health systems
Steinwachs, 1998; Torrey et al., 2001). summary, a recovery-oriented environ-
have responded to the challenge of de-
This failure to transfer knowledge from ment is one that encourages individu-
veloping more recovery-oriented sys-
the research realm to routine practice ality; promotes accurate and positive
tems of care, only to be faced with
settings is well documented in the “call portrayals of psychiatric disability
tremendous uncertainties about how to
to action” issued by the Institute of while fighting discrimination; focuses
proceed. By integrating more recovery-
Medicine in its report “Bridging the on strengths; uses a language of hope
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Gap Between Research and Practice” and possibility; offers a variety of op-
hope, optimism, and high expectations
(1998). tions for treatment, rehabilitation, and
into mission statements, policies, and
support; supports risk-taking, even
In responding to the challenges put procedures, it may appear as if states
when failure is a possibility; actively in-
forth by the Surgeon General and the have met the challenge. Because there
volves service users, family members,
Institute of Medicine, the mental health are few, if any, models of care that
and other natural supports in the de-
field must incorporate the science of have operationalized the principles of
velopment and implementation of pro-
technology transfer to stimulate the recovery into objective practices that
grams and services; encourages user
widespread organizational change that can be used to guide, monitor, and
participation in advocacy activities;
is necessary for effective and recovery- evaluate mental healthcare (Anthony,
helps develop connections with com-
oriented systems to evolve. What we 2000), the extent to which a change in
munities; and helps people develop
learn from the science of technology rhetoric is accompanied by a similar
valued social roles, interests and hob-
transfer is that the existing gap be- change in practice is unclear. For exam-
bies, and other meaningful activities.
tween research and practice can only ple, one recent review suggested that
Moving beyond the rhetoric of recovery
be bridged by employing a combina- there has been much old wine poured
requires operationalizing these princi-
tion of multifaceted knowledge dissem- into the new bottles of recovery lan-
ples into standards and practices that
ination tools (Bero et al., 1998; Brown guage (Jacobson & Greenley, 2001).
can be observed, measured, and then
& Flynn, 2002).
In order to begin to address this lack of fed back to mental health organiza-
Among the variety of tools available to clarity in the application of recovery tions in a manner that encourages the
facilitate the evolution of care is the ca- principles to reforms in practice, we use of data to inform program improve-
pacity of an organization to make use conducted an extensive review of liter- ment and organizational change
of ongoing technical assistance and ature produced by users and providers efforts.
consultation. It is critical that such as- of mental health and addiction servic-
The Present Study
sistance be extended in the context of es, researchers, and advocates on the
The purpose of the present study was
a collaborative partnership that has topics of recovery from mental illness-
to assess the degree to which recovery-
been developed with stakeholders at es and addictions. We then identified
oriented practices were perceived to be
all levels in the organization. This rela- several common principles of recovery
implemented in mental health and ad-
tionship is the vehicle through which and recovery-oriented systems of care
diction agencies funded by the
information can be gathered, evaluat- (Davidson, O’Connell, Sells & Staeheli,
Connecticut Department of Mental
ed, and fed back to the organization for 2003). In this review, nine principles of
Health and Addiction Services using a
the purpose of program adjustment recovery were identified: renewing
newly developed tool—the Recovery
and improvement. Researchers and hope and commitment; redefining self;
Self Assessment (RSA). Based on the
consultants must therefore prioritize incorporating illness; being involved in
recovery principles of empowerment
the cultivation of collaborative evalua- meaningful activities; overcoming the
and stakeholder involvement in the as-
tion-stakeholder feedback loops effects of discrimination; assuming
sessment and development of servic-
(Borich & Jemelka, 1982; Chinman, control; becoming empowered and
es, perceptions of recovery-oriented
Weingarten, Stayner & Davidson, 2001; more involved in one’s community and
practices were assessed from multiple
articles
379
P s y c h i at r i c R e h a b i l i tat i o n J o u r n a l From Rhetoric to Routine: Assessing Perceptions of Recovery-Oriented Practices
perspectives including those of agency version. A total of 3,328 surveys were with a six-category Likert response for-
directors, direct care providers, per- mailed to agency directors across the mat from 1 (strongly disagree) to 5
sons in recovery, family members, sig- state. Participation was voluntary and (strongly agree) or N/A (not applica-
nificant others, and advocates. completion of the questionnaire was ble). Persons in recovery, providers of
Agency-specific “recovery profiles” considered written informed consent. mental health and addiction services,
were then made available to all partici- family members, and researchers with
Sample
pants as part of an ongoing consulta- expertise in measurement develop-
Participants included 974 individuals
tion and technical assistance initiative. ment and clinical and community psy-
(individual response rate of 29%) from
This initiative assists agencies to build chology reviewed all items for content
82 of the facilities (39% agency re-
upon their existing strengths and to and comprehension. Items were then
sponse rate). Four agencies and corre-
focus upon areas in need of improve- edited, balanced with regard to con-
sponding respondents (n=7) were
ment as they strive to offer recovery- ceptual domain, and eliminated until
eliminated from the analysis due to the
oriented care to the people they treat. 36 items remained. The items on the
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
articles
380
spring 2005—Volume 28 Numb er 4
and provision of new programs and services. A third factor, “Diversity of Treatment
27 People in recovery are regular members of agency advisory boards .74 Options,” contains 6 items that indi-
and management meetings. cate perceptions of the extent to which
15 Persons in recovery are involved with facilitating staff trainings and .71 an agency provides linkages to peer
education programs at this agency. mentors and support, a variety of treat-
Factor 3: Diversity of Treatment Options ment options, and assistance with be-
34 Criteria for exiting or completing the agency are clearly defined .54 coming involved in non-mental health
and discussed with participants upon entry to the agency. activities. This factor accounted for
18 This agency actively attempts to link people in recovery with other persons .53 9.8% of the total variance in the sam-
in recovery who can serve as role models or mentors by making referrals ple. The internal consistency estimate
to self-help, peer support, or consumer advocacy groups or programs. for this factor was .83.
19 This agency provides a variety of treatment options (i.e., individual, .52
A fourth factor, “Choice,” contains 6
group, peer support, holistic healing, alternative treatments, medical)
from which agency participants may choose. items measuring perceptions of the ex-
tent to which service users have access
Factor 4: Choice
3 People in recovery have access to all their treatment records. .62 to their treatment records, staff refrain
from using coercive measures to influ-
13 Agency staff do not use threats, bribes, or other forms of coercion .59
to influence a person’s behavior or choices. ence choice, and the choices of service
users are respected by staff. This factor
10 Staff at this agency listen to and follow the choices and preferences .56
of participants. accounted for 8.9% of the total vari-
ance in the sample. The internal consis-
Factor 5: Individually-Tailored Services
2 This agency offers specific services and programs for individuals .65 tency estimate for this factor was .76.
with different cultures, life experiences, interests, and needs.
The final factor, “Individually-Tailored
9 All staff at this agency regularly attend trainings on cultural competency. .58 Services,” contains 5 items that reflect
1 Helping people build connections with their neighborhoods and .55 perceptions of the extent to which serv-
communities is one of the primary activities in which staff at this ices are tailored to individual needs,
agency are involved.
cultures, and interests, and focus on
building community connections. This
Higher factor scores reflect greater factor accounted for 8% of the total
Results agreement with the items in that factor. variance and had an internal consisten-
RSA Factors A first factor, “Life Goals” contains 11
cy estimate of .76.
The 36 items on the RSA fell into five items that reflect perceptions of the ex- Differences in Perceived Recovery-
components and accounted for 53.8% tent to which staff help with the devel- Oriented Practices by Respondent
of the total variance in the sample. opment and pursuit of individually Category
Table 1 contains sample items and item defined life goals such as employment A series of Analysis of Variance
loadings for each of the five factors. and education. This factor accounted (ANOVA) techniques were used to ex-
for 13.7% of the total variance in the amine scores on the five factors and
articles
381
P s y c h i at r i c R e h a b i l i tat i o n J o u r n a l From Rhetoric to Routine: Assessing Perceptions of Recovery-Oriented Practices
Table 2—Means and Standard Deviations for RSA Summary Score and Factor Scores
Family/
Persons in SO/ Total
Directors Providers Recovery Advocates Sample
(n=68) (n=344) (n=326) (n=229) (N=967)
Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD)
Summary Score 4.09 (.42) 3.87 (.62)a 4.06 (.69)b** 4.00 (.77) 3.98 (.67)
Life Goals 4.36 (.44)b* 4.10 (.66)a 4.21 (.70) 4.16 (.78) 4.17 (.70)
Involvement 3.56 (.74) 3.39 (.80)a 3.79 (.91)b** 3.79 (.93)c** 3.62 (.88)
Diversity of Treatment Options 4.23 (.51) 3.95 (.78) 4.02 (.91) 4.00 (.87) 4.00 (.83)
Choice 4.29 (.53) 4.08 (.69) 4.14 (.82) 4.06 (.80) 4.11 (.76)
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Individually-Tailored Services 3.91 (.61) 3.82 (.79)a 4.01 (.81)b* 4.05 (.83)c** 3.94 (.80)
This document is copyrighted by the American Psychological Association or one of its allied publishers.
Notes.
* p<.05, ** p<.01
the RSA summary score as a function the normal curve and lines demarcat- ual item analysis of the five highest
of respondent category (director, ing standard deviations. The his- rated recovery-oriented practices at the
provider, person in recovery, and fami- tograms showed a relatively normal agency and the five lowest rated items
ly/significant other/advocate). Results distribution of average scores on the (based on the average scores of all re-
indicate that service providers had sig- RSA summary score and factor scores spondents).
nificantly lower ratings than persons in with approximately 65% of the agen-
recovery on the RSA summary score, cies falling within one standard devia-
Discussion and Conclusions
F (3,912) =5.03, p=.00. On the individ- tion above or below the mean and
ual factors, ANOVAs revealed that serv- approximately 95% falling within two As state mental health systems begin
ice providers had significantly lower standard deviations above or below to shift from traditional mental health
scores than directors on Life Goals, the mean. treatment to models of care that focus
F (3,934) =2.97, p=.03. Providers also on supporting an individual’s recovery,
An individual, confidential Recovery
had significantly lower scores than it is imperative that the redefinition
Profile was created for each agency
persons in recovery and family mem- goes beyond rhetoric into actual
that desired such a report. The
bers/significant others/advocates on changes in practice. A common chal-
Recovery Profiles contained the his-
Involvement, F (3,894) =14.36, p=.00, lenge emerges in this translation—
tograms described above with the re-
and Individually-Tailored Services, there are few, if any, models of care
spective agency highlighted in yellow.
F (3,934) =4.70, p =.00. There were no that have operationalized the princi-
These histograms allowed individual
significant differences between the ples of recovery into objective prac-
agencies to view their relative standing
various categories of respondents tices. The present study contributes to
in comparison to other participating
on Choice or Diversity of Treatment this effort in several ways: a) it is the
agencies (see Appendix A for a sample
Options. See Table 2 for the means and first known statewide assessment of
Agency Recovery Profile). Each
standard deviations of factor scores for perceptions of recovery-oriented prac-
Recovery Profile also contained a brief
each of the respondent categories. tices; b) it involved the development of
description of the agency’s sample and
an inventory that can be used by serv-
Recovery Profiles a description of relative strengths (fac-
ice systems to assess perceptions of
The individual-level data were aggre- tors and items more than 1 standard
practices that are considered to be con-
gated to the level of the agency (N=78). deviation above the state average) and
sistent with a recovery-orientation from
The average factor scores and the aver- areas for improvement (factors and
multiple perspectives; and c) it pro-
age RSA summary score were comput- items more than 1 standard deviation
vides an illustration of how research
ed for each of the agencies and were below the state average). Finally, each
can be translated into everyday prac-
displayed in histograms that contained Recovery Profile contained an individ-
articles
382
spring 2005—Volume 28 Numb er 4
tice through the use of self-assessment service user surveys is that people only siasm. Much of the feedback received
and structured feedback. know that to which they have been ex- indicated that providers were hungry
posed (Davidson, Hoge, Godleski, for tools that would enable them to en-
Results indicate mental health profes-
Rakfeldt & Griffith, 1996). If someone is hance the recovery orientation of the
sionals, persons in recovery, and fami-
not familiar with alternatives, it is easi- care they provided. Since the study,
ly members generally agreed that their
er to be satisfied with what he/she the items in the RSA have served as the
agencies were providing services that
has. It should be noted that the ideal impetus for much discussion among
are consistent with a recovery orienta-
administration of the RSA is one that a) stakeholders in mental health and ad-
tion. It is encouraging that the highest
is administered consistently across diction programs about the meaning of
rated items were those related to serv-
sites, program participants, and staff; recovery and recognizable practices.
ices focusing on helping people ex-
b) maintains anonymity of respon- Based on these discussions, separate
plore their own goals and interests
dents; and c) assesses the majority of models of recovery pertaining to men-
beyond symptom management.
the program participants/staff. This tal health and/or addictions and stan-
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
would help to reduce some of the self- dards of care have been developed at
on items regarding involvement of
selection bias as well as allowing for the level of the state to help practition-
service users in service design, man-
the most accurate description of per- ers learn to differentiate recovery-ori-
agement, and provision. This finding is
ceptions of recovery-oriented practices ented practices from non-recovery
consistent with the literature indicating
at the each agency. oriented practices. The dissemination
that one of the most difficult barriers
of findings within agencies has also
for practitioners in recovery is being In addition to the limitation of the lack
helped to identify discrepancies in per-
accepted as equal members of agency of randomized selection of respon-
ceptions within organizations that may
staff (Mowbray, Moxley, Jasper & dents within agencies, data on con-
be attributed to a lack of communica-
Howell, 1997). struct and external validity of the RSA
tion or misunderstanding (i.e., persons
is not available at this time because it
It is important to note that, despite in recovery often indicated that they
was administered as a single inventory.
overall favorable responses, when ex- did not have access to their treatment
In the future, research should be con-
amining agency ratings as a function of records, whereas providers and direc-
ducted to examine scores on the RSA in
role in the mental health system, tors often indicated the opposite).
relation to other recovery-oriented con-
providers gave significantly lower rat- Thus, the Recovery Self Assessment is
structs such as quality of life, satisfac-
ings on three of the five factors. This not only a useful, self-reflective tool for
tion with services, and empowerment
finding indicates a degree of discon- agencies to begin to identify strengths
to determine if persons in recovery re-
nect between the various stakeholders and target areas for improvement as
ceiving services at agencies that score
in the field about what is actually oc- they strive to offer recovery-oriented
higher on the RSA in fact have better
curring in everyday practice. The con- care to the people they treat, but it is
individual outcomes. Furthermore, be-
sistently higher ratings by persons in an effective tool to help strengthen col-
cause the items are written to assess
recovery may indicate that they per- laborative evaluation-stakeholder feed-
perceptions of measurable indices of a
ceive recovery-oriented practices to be back loops.
recovery-oriented system of care, fu-
occurring in their agency. However,
ture research should also examine the
these higher service user ratings may
degree to which the subjective percep-
also reflect common difficulties that
tions of recovery-oriented practices are
occur in conducting service user satis-
consistent with the actual implementa-
faction surveys. For example, there
tion of the practices and more objec-
may have been a selection bias due to
tive measures of fidelity (i.e., data
the fact that agency directors and
available in chart reviews, policies, and
providers hand-selected the respon-
procedures).
dents. Providers could have chosen
persons in recovery who were actively Despite these limitations, the attempt
engaged in treatment and other activi- to operationalize and measure prac-
ties outside of the system, and thus tices that could be supportive of a per-
were more likely to rate these services son’s recovery was embraced by the
highly. Another problem that occurs in provider community with much enthu-
articles
383
P s y c h i at r i c R e h a b i l i tat i o n J o u r n a l From Rhetoric to Routine: Assessing Perceptions of Recovery-Oriented Practices
articles
384
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Appendix B
385
articles
spring 2005—Volume 28 Numb er 4
P s y c h i at r i c R e h a b i l i tat i o n J o u r n a l
New York, NY: Academic Press. Research Team (PORT) treatment recom-
This document is copyrighted by the American Psychological Association or one of its allied publishers.
articles
386