EVALUATION HSRI Measuring the Promise:

A Compendium of Recovery
Measures
@
Volume II
C E N T E R

September 2005
Prepared by:

Theodora Campbell-Orde, Judi Chamberlin Jenneth Carpenter, M.S.W.

M.P.A. National (Corresponding Author)
the Empowerment
Center
H. Stephen Leff, Ph.D.
The Evaluation Center@HSRI

Human Services Research Institute U.S. Department of Health and Human Services
2269 Massachusetts Avenue Substance Abuse and Mental Health Services Administration
Cambridge, MA 02140 Center for Mental Health Services
www.tecathsri.org www.samhsa.gov
Measuring the Promise: A Compendium of Recovery Measures, Volume II is one of a series of materials produced by the
Evaluation Center@HSRI. The center is supported by a cooperative agreement with the Center for Mental Health
Services, Substance Abuse and Mental Health Services Administration. The mission of the Evaluation Center is to
provide technical assistance related to the evaluation of adult mental health systems change.

The Evaluation Center offers seven programs all of which are designed to enhance evaluation capacity. The pro-
grams are: the Consultation Program, which provides consultation tailored to the needs of individual projects;
the e-Community Program, which provides a forum for ongoing dialogue via electronic conferencing; the Tool-
kits & Materials Program, which provides evaluators with tested methodologies, instruments, and original pa-
pers on selected topics and identifies relevant literature in the field; the e-Learning Program, which supplies on-
line courses and in-person training; the Multicultural Program, which provides technical assistance with respect
to evaluation of mental health services and systems for racially, ethnically, and culturally diverse persons; the
Conferences Program designed to inform our audience of events in which issues related to evaluation research are
discussed; and the Evidence-Based Practices Program, which assists in identifying evidence-based practices and
moving promising interventions to evidence-based service.

The Evaluation Center creates and disseminates toolkits and materials designed to provide evaluators with com-
plete descriptions of methodologies and instruments for use in evaluating specific topics. Based on information
from a needs assessment study conducted by the Evaluation Center and on feedback from evaluators in the field,
we have identified a number of important topics that evaluators are frequently interested in examining. Expert
consultants have been engaged to review the background of these topics and to compile toolkits that provide
evaluators with state-of-the-art evaluation techniques to use in their own work.

The Evaluation Center has also established bulletin boards for discussing issues surrounding its toolkits as well as
other matters related to mental health service evaluation. This bulletin board will provide an electronic forum for
toolkit users to share their expertise and experiences using the toolkits. If you would like to participate in a user
group, please visit the e-Community area of our Web site, www.tecathsri.org.

We hope that this publication will be helpful to those evaluators who are interested in measuring recovery among
individuals and the recovery orientation of systems. Additionally, we encourage readers to visit our Web site,
www.tecathsri.org, where a regularly updated Web version of this document will be available.

H. Stephen Leff, Ph.D. Virginia Mulkern, Ph.D.

Director Associate Director
Table of Contents

EXECUTIVE SUMMARY ........................................................................ 7

ACKNOWLEDGEMENTS ........................................................................ 9
CHAPTER 1. INTRODUCTION ............................................................. 10
Aims of Volume II .................................................................................................................. 11
Development of Volume II ...................................................................................................... 11

Identification of Potential Instruments .................................................................................. 11

Recovery Compendium Volume I .................................................................................... 11

Literature Review........................................................................................................... 12
Invitational Conference .................................................................................................. 13

Instrument Selection ......................................................................................................... 13

Inclusion and Exclusion Criteria....................................................................................... 13

Author Contact Process ................................................................................................. 14
Categorization Criteria ................................................................................................... 14

Report Completion ........................................................................................................... 14

Internal Review Process .................................................................................................. 14

Author Review .............................................................................................................. 14

Dissemination and Ongoing Development ........................................................................... 14

Organization of the Second Volume ......................................................................................... 15

References ............................................................................................................................ 16

CHAPTER 2. INVITATIONAL CONFERENCE.......................................... 17

Conference Overview and Summary ........................................................................................ 17

Measuring Recovery: Obstacles and Concerns ...................................................................... 17

General Concerns with Current Instruments ......................................................................... 17
Ideas for Developing and/or Testing Instruments ................................................................... 18
Promoting a Recovery Agenda ............................................................................................ 18

Conference Reflections and Remarks........................................................................................ 18

The Current Mental Health System and EBP ......................................................................... 19

Shifting the Paradigm ......................................................................................................... 19
CHAPTER 3. RECOVERY MEASURES .................................................... 20
Using the Instrument Descriptions........................................................................................... 20

A Note on Validity Testing .................................................................................................. 20

Instruments at a Glance: Measures of Individual Recovery ...................................................... 21
Instruments at a Glance: Measures of Recovery Promoting Environments ................................. 23

Measures of Individual Recovery ............................................................................................. 24

1. Consumer Recovery Outcomes System Version 3(CROS) ................................................... 25

2. Illness Management and Recovery ................................................................................... 32
3. Mental Health Recovery Measure (MHRM) ....................................................................... 36
4. Ohio Outcomes System .................................................................................................. 42
5. Peer Outcomes Protocol (POP) ....................................................................................... 50
6. Reciprocal Support Scale ................................................................................................ 57
7. Recovery Assessment Scale (RAS) ................................................................................... 60
8. Recovery Measurement Tool Version 4 (RMT) .................................................................. 65
9. Relationships and Activities that Facilitate Recovery (RAFRS) .............................................. 68

Measures of Recovery Promoting Environments ........................................................................ 71

1. AACP Recovery Oriented Service Evaluation (AACP-ROSE) ............................................... 72

2. Recovery Enhancing Environment Measures (REE) ............................................................. 75
3. Recovery Oriented Systems Indicators Measure (ROSI) ...................................................... 81
4. Recovery Self-Assessment .............................................................................................. 91

CHAPTER 4. CONCLUDING REMARKS ................................................ 97

APPENDIXES .................................................................................... 100
Appendix A: Conference Participant List ................................................................................ 101

Appendix B: Instrument Description Form .............................................................................. 102

Appendix C: Notes for Non-Researchers ................................................................................ 109

Appendix D: Instruments and Materials.................................................................................. 111

Measures of Individual Recovery ....................................................................................... 111

1. Consumer Recovery Outcomes System Version 3(CROS) ................................................. 112

2. Illness Management and Recovery ................................................................................. 124
3. Mental Health Recovery Measure (MHRM) ..................................................................... 133
 The Evaluation Center @ HSRI

4. Ohio Outcomes System ............................................................................................... 136

5. Peer Outcomes Protocol (POP) ..................................................................................... 150
6. Reciprocal Support Scale .............................................................................................. 192
7. Recovery Assessment Scale (RAS) ................................................................................. 194
8. Recovery Measurement Tool Version 4 (RMT) ............................................................... 198
9. Relationships and Activities that Facilitate Recovery (RAFRS) ............................................ 205

Measures of Recovery Promoting Environments .................................................................. 209

1. AACP Recovery Oriented Services Evaluation (AACP-ROSE) ........................................... 209

2. Recovery Enhancing Environment Scale (REE) ................................................................. 213
3. Recovery Oriented Systems Indicators (ROSI) ................................................................. 229
4. Recovery Self-Assessment (RSA) ................................................................................... 244

Measuring the Promise: A Compendium of Recovery Measures, Volume II 6

 The Evaluation Center @ HSRI

EXECUTIVE SUMMARY
Measuring the Promise: A Compendium of Recovery Measures, Volume II has three specific aims:

• To provide the adult mental health field with research-and evaluation-based perspectives on the
nature of recovery.
• To provide a resource of current recovery and recovery-related instruments for adult mental health
system stakeholders to use in research and evaluation particularly towards the end of identifying
evidence-based practices.
• To provide stakeholders with a detailed summary of the key characteristics of each instrument in an
easy to follow format.

The instruments reviewed in this volume were drawn from a number of sources, most importantly from
the previous volume, Can We Measure Recovery? A Compendium of Recovery and Recovery-Related Instruments
(Ralph, Kidder, & Phillips, 2000), and from literature searches and professional networking. The instru-
ment descriptions were informed by discussions and presentations at the November 2004 invitational con-
ference, Measuring the Promise: Assessing Recovery and Self-Determination Instruments for Evidence-Based Practices.

This conference drew a total of 30 consumer- and nonconsumer-researchers for two days of discussion
on recovery measurement. Conference participants identified obstacles to measuring recovery, concerns
about current instruments, and ideas for further instrument testing and the promotion of a recovery
agenda.

The instruments themselves fall into one of two categories: measures of individual recovery and measures
of recovery-promoting environments. The instruments vary widely in their stages of development, ranging
from those that have not yet been pilot tested to those that have undergone considerable testing and have
some established psychometric properties. The instruments also differ considerably in length and their
content reflects a variety of domains.

Measures of Individual Recovery

1. Consumer Recovery Outcomes System (CROS 3.0)
The Colorado Health Networks Partnership (undated)
2. Illness Management and Recovery (IMR) Scales
Mueser, K.T., Gingerich, S., Salyers, M.P., McGuire, A.B., Reyes, R.U., & Cunningham, H. (2004)
3. Mental Health Recovery Measure (MHRM)
Young, S.L. & Bullock, W.A (2003)
4. Ohio Mental Health Consumer Outcomes System
Ohio Department of Mental Health Office of Program Evaluation and Research (2004)
5. Peer Outcomes Protocol (POP)
Campbell, J, Cook, J.A., Jonikas, J.A., & Einspahr, K. (2004)
6. Reciprocal Support Scale
Silver, T., Bricker, D., Pesta, Z., & Pugh., D. (2002)
7. Recovery Assessment Scale (RAS)
Giffort, D., Schmook A., Woody, C., Vollendorf, C., & Gervain, M. (1999)
8. Recovery Measurement Tool Version 4 (RMT)
Ralph, R. O. (2003)
9. Relationships and Activities that Facilitate Recovery Survey (RAFRS)
Leavy, R.L., McGuire, A.B., Rhoades, C., & McCool, R. (2002)

Measuring the Promise: A Compendium of Recovery Measures, Volume II 7

 The Evaluation Center @ HSRI

EXECUTIVE SUMMARY, con’t.

Measures of Recovery Promoting Environments
1. AACP ROSE - Recovery Oriented Service Evaluation
American Association of Community Psychiatrists (undated)
2. Recovery Enhancing Environment Measure (REE)
Ridgway, P.A. (2004)
3. Recovery Oriented Systems Indicators Measure (ROSI)
Dumont, J. M., Ridgway, P., Onken, S. J., Dornan, D. H., & Ralph, R. O. (2005)
4. Recovery Self-Assessment (RSA)
O’Connell, M., Tondora, J., Croog, G., Evans, A., & Davidson, L. (2005)

The relatively high degree of meaningful consumer involvement in instrument development is a strength
of these instruments as a group. Maintaining consumer involvement and authorship, drawing upon and
reflecting the recovery experiences of diverse populations, and designing and utilizing innovative measures
of instrument validity will be critical goals in the continued development of these instruments.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 8

 The Evaluation Center @ HSRI

ACKNOWLEDGMENTS

The Evaluation Center@HSRI (the Evaluation Center) thanks the many people who generously contrib-
uted to the development of Measuring the Promise: A Compendium of Recovery Instruments, Volume II. We are
especially grateful to the authors of the instruments for allowing the Evaluation Center to include their
instruments in this volume and to the instrument contacts who provided detailed information about each
instrument. Without these contributions, the development of this volume would not have been possible.

We also thank the individuals who participated in the invitational conference Measuring the Promise: Assess-
ing Recovery and Self-Determination Instruments for Evidence-Based Practices. The conference was convened by the
Evaluation Center, a mental health services research and evaluation technical assistance center, in collabo-
ration with the National Empowerment Center, a nationally prominent Consumer/Survivor Technical As-
sistance Center, and the National Technical Assistance Center for State Mental Health Planning, a center
associated with the National Association of State Mental Health Program Directors that provides planning
and technical assistance to states. As described later in this volume, 30 consumer/survivor and non-con-
sumer/survivor researchers participated in this conference. Their input greatly informed the development
of the form used to collect information about each instrument and further shaped the compendium by
bringing key issues concerning the measurement of recovery to the table.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 9

 The Evaluation Center @ HSRI

CHAPTER 1. INTRODUCTION

Individuals with psychiatric disabilities formally introduced the concept of recovery into the mental
health field in the 1980s through published accounts of their struggles with mental illness and their jour-
neys to wellness (Anonymous, 1989; Deegan, 1988; Leete, 1989, Unzicker, 1989). In its broadest sense, re-
covery can be characterized by the ability of individuals who have a psychiatric disability to live personally
meaningful and fulfilling lives (Anthony 1993; Deegan; Leete; Corrigan, Giffort, Rashid, Leary, Okeke, 1999;
New Freedom Commission on Mental Health, 2003). Grounded in consumers/survivors’ struggles with
mental illness and journeys to wellness, the concept of recovery has been defined in many ways. While no
widely accepted operational definition of recovery currently exists within the field of mental health, efforts
have been undertaken to identify common components of the recovery paradigm (Ridgway, 2001; Onken,
Craig, Ridgway, Ralph, & Cook, 2004; Young and Ensing, 1999) and, most recently, a meeting was held to
develop a consensus statement on the definition of recovery. The results of this meeting will appear on the
SAMHSA website: http://www.samhsa.gov.

Since its emergence, the notion of recovery has gained credence through numerous consumer/survivors’
first person recovery accounts and a growing number of empirical studies (Harding, Brooks, Ashikaga,
Strauss, & Breier, 1987; Desito, Harding, McCormick, Ashikaga, & Brooks, 1995). As evidence of recov-
ery begins to discount traditional beliefs that characterize mental illness as chronic and degenerative in
nature, stakeholders have begun to discuss a national recovery-oriented mental health system of care. In
1999, the U.S. Surgeon General’s highly influential report on mental health brought recovery to the fore-
front of the field by calling for mental health services that promote recovery (U.S. Department of Health
and Human Services, 1999). Most recently, recovery has been identified as the goal of a transformed
mental health system (New Freedom Commission on Mental Health, 2003). As the current mental health
system begins to undergo fundamental changes based on that goal, research and evaluation activities are
necessary to the successful development and continuous delivery of services that do indeed promote
recovery.

In 2000, Ruth Ralph and colleagues collaborated with the Evaluation Center to produce a compendium
of recovery and recovery-related measures entitled Can We Measure Recovery? A Compendium of Recovery and
Recovery-Related Instruments, Volume I (Ralph, Kidder, & Phillips). Since the development of Volume I, both
the study of recovery in the field of mental health and the development of instruments to measure recovery
have progressed. Some instruments that were in existence have been further developed and new instru-
ments have emerged. One of the most notable developments is a class of recovery instruments designed to
assess the recovery-orientation of services, practices, and systems. Such developments point to the need
for an updated recovery instruments compendium, particularly given the ever-increasing interest in recov-
ery among mental health service recipients, providers, funding agencies, and the public.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 10

 The Evaluation Center @ HSRI

AIMS OF VOLUME II

Volume II has three specific aims:

1. To provide the adult mental health field with research- and evaluation-based perspectives on the
nature of recovery.
2. To provide a resource of current recovery and recovery-related instruments for adult mental health
system stakeholders to use in research and evaluation particularly towards the end of identifying
evidence-based practices.
3. To provide stakeholders with a detailed summary of the key characteristics of each instrument in an
easy to follow format.

Our first aim is based on the belief that viewing recovery from research and evaluation-based perspectives
will deepen our understanding of this goal. Our second aim reflects our belief that, as services, programs,
and systems strive to become both recovery-oriented and evidence-based, quantitative measures of recov-
ery and its components can help identify practices that are effective in bringing about recovery and provide
tools for monitoring recovery for quality improvement purposes (for an alternative view on evidence-based
practice, see Judi Chamberlin’s comments in Chapter 2). Our third aim is based on the perception that
instruments that measure recovery vary in terms of their conceptual foundations, development processes,
domains of recovery measured, psychometric properties, and supporting materials. Volume II provides us-
ers with this information to assist them in choosing an instrument that is best suited for their research and
evaluation purposes.

DEVELOPMENT OF VOLUME II

Identification of Potential Instruments

Instruments presented in Volume II were identified through three formal processes: A review of the re-
covery instruments in Volume I, a review of the relevant literature, and discussions with leading recovery
researchers at the invitational conference Measuring the Promise: Assessing Recovery and Self-Determination Instru-
ments for Evidence-Based Practices. Informally, instruments were also identified through networking with
individuals involved in the measurement of recovery in the mental health field.

Recovery Compendium Volume I: All instruments reviewed in Volume 1 (Ralph et al., 2000) are listed in
Table 1.1. These instruments fell into two categories: instruments intended to measure one or more aspects
of recovery (recovery measures) and those intended to measure constructs thought to be associated with
recovery (recovery-related measures).

During the development of Volume II, the Evaluation Center attempted to contact the authors of those
instruments categorized as recovery measures in Volume I to find out if updated information was available.
Table 1.2 summarizes the development status, if known, of each of these instruments. If authors indicated
no further development had occurred in terms of either testing or actual instrument content, Table 1.2
shows “not updated.” If authors had not responded by the time of publication for Volume II, Table 1.2
indicates “no further information at this time.” As shown in Table 1.2, only two of the instruments - the
Recovery Assessment Scale (RAS) and the Mental Health Recovery Measure (MHRM) - had been further

Measuring the Promise: A Compendium of Recovery Measures, Volume II 11

 The Evaluation Center @ HSRI

developed since the publication of Volume I. Both of these instruments are included in Volume II; the reader
is directed to the first volume for information on any of the other earlier instruments.

Literature Review: The literature review was based primarily on electronic searches of the PubMed and
PsycINFO databases, using terms like “recovery,” “empowerment,” and “self-determination.” Addition-
ally, articles collected by the Evaluation Center staff in the course of their work were included. Finally, as
word of the project spread, some articles and reports were acquired from colleagues.
Table 1.1

Instruments Included in Volume I

Recovery Measures: Recovery-Related Measures:

1. Crisis Hostel Healing Scale 1. Leadership Education and Training Assessment

New York Crisis Hostel Project, 1998 Bullock, W.A. , Ensing, D.S., Alloy, V., & Weddle G., 2000

2. Recovery Assessment Scale 2. Well-Being Scale

Giffort D., Schmook, A., Woody C., Vollendorf, C., Campbell, J. & Schraiber, R., 1989
&Gervain, M., 1995
3. Mental Health Confidence Scale
3. Rochester Recovery Inquiry Carpinello, S.E., Knight, E.L., Markowitz, F.E., & Pease, E.A.,
Hopper, K., Blanch, A., Carpinello, S., Johnson, S., 2000
Knight, E., Kovasznay, B., & Krauss, A., 1996
4. Herth Hope Index
4. Recovery Interview Herth, K., 1992
Heil, J. & Johnson, L.K., 1998
5. Hope Scale
5. Recovery Attitudes Questionnaire Snyder, C.R., Harris, C., Anderson, J.R., Holleran, S.A.,
RAQ-7: Borkin, J.R., Steffen, J.J., Ensfield, L.B. , Irving, L.M., Sigmon, S.T., Yoshinobu, L., Gibb, J., Langelle,
Krzton, K., Wishnick, H., Wilder, K.E., & Yangarber C., & Harney, P., 1991
N., 1998
RAQ-16: Steffen, J.J., Borkin, J.R., Krzton, K., 6. Staff Relationships Scale
Wishnick, H., & Wilder, K.E., 1998 Hornik, J., Ralph, R.O., & Salmons ,T., 1999

6. Personal Vision of Recovery Questionnaire 7. Making Decisions Empowerment Scale

Ensfield, L.B., Steffen, J.J., Borkin, J.R., & Schafer, Rogers, E.S., Chamberlin, J., Ellison, M.L., & Crean, T., 1997
J.C., 1998
8. UCLA Loneliness Scale, Version 3
7. Agreement with Recovery Attitudes Scale Russell, D.W., 1996
Murnen, S.K. & Smolak, L., 1996
9. Personal/Organizational /Extra-Organizational
8. Mental Health Recovery Measure Empowerment Scales
Young, S.L., Ensing, D.S., & Bullock, W.A., 1999 Segal, S.P., Silverman, C., & Temkin, T., 1995

10. Community Living Skills Scale

Smith, M.K. & Ford, J., 1990

Measuring the Promise: A Compendium of Recovery Measures, Volume II 12

 The Evaluation Center @ HSRI

Table 1.2

Development Status of Recovery Measures

Included in Version I
Instrument Testing and Use Instrument Content
Crisis Hostel Healing Scale No further information at No further information at
New York Crisis Hostel Project, 1998 this time this time

Recovery Assessment Scale UPDATED NOT UPDATED

Giffort, D., Schmook, A., Woody, C., Vollendorf, C., & Gervain,
M., 1995
Rochester Recovery Inquiry No further information at No further information at
Hopper, K., Blanch, A., Carpinello, S., Johnson, S., Knight, E., this time this time
Kovasznay, B., & Krauss ,A., 1996
Recovery Interview NOT UPDATED NOT UPDATED
Heil, J. & Johnson, L.K., 1998
Recovery Attitudes Questionnaire (RAQ-7) No further information at No further information at
Borkin, J.R., Steffen, J.J., Ensfield, L.B. , Krzton, K., Wishnick, H., this time this time
Wilder, K.E. & Yangarber, N., 1998
Recovery Attitudes Questionnaire (RAQ-16) No further information at No further information at
Steffen, J.J., Borkin, J.R., Krzton, K., Wishnick, H. ,& Wilder, K.E., this time this time
1998
Personal Vision of Recovery Questionnaire (PRVQ) No further information at No further information at
Ensfield, L.B., Steffen, J.J., Borkin, J.R., & Schafer, J.C., 1998 this time this time
Agreement with Recovery Attitudes Scale NOT UPDATED NOT UPDATED
Murnen, S.K. & Smolak, L., 1996
Mental Health Recovery Measure (MHRM) UPDATED UPDATED
Young ,S.L., Ensing, D.S,. & Bullock, W.A., 1999

Invitational Conference: National experts in mental health recovery and its measurement met in Boston,
Massachusetts, on November 3-4, 2004 (see Appendix A for a complete list of participants). The meet-
ing was convened by the Evaluation Center in collaboration with the National Empowerment Center and
the National Technical Assistance Center for State Mental Health Planning. It brought together both
consumer/survivor researchers and non-consumer/survivor researchers to discuss measuring recovery for
evidence-based practices and systems improvement.

The conference provided an opportunity for consumer/survivor researchers and non-consumer/survivor

researchers to discuss an array of issues around the measurement of recovery. A conference summary and
reflections on the conference are included in Chapter 2 of this volume. The conference also informed the
development of the form used to systematically collect critical information about the recovery instruments
included in Volume II. A copy of this form may be found in Appendix B.

Instrument Selection

Inclusion and Exclusion Criteria: Instruments were considered for inclusion in Volume II if they were
identified as recovery measures by their developers. The Evaluation Center decided not to “screen” mea-
sures for Volume II based on their content. Instead, we decided to let users have the widest possible choice
of measures.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 13

 The Evaluation Center @ HSRI

Author Contact Process: Once the instruments had been identified, the Evaluation Center staff com-
pleted the instrument description forms to the best degree possible using information collected during the
literature search. Authors or contacts for each instrument were identified, and each was sent the partially
completed description form for their instrument, along with details on the second volume and its purpose
and a request for the remaining instrument information. The Evaluation Center staff attempted to reach
authors multiple times, as necessary.

Categorization Criteria: As this information was being gathered, the instruments were sorted into cate-
gories. Volume I consisted of two categories of instruments: those that were intended to measure individu-
als’ recovery specifically, and those measuring constructs considered closely related to recovery. Although
these categories were considered for Volume II, the distinction between recovery and recovery-related
measures in the newer body of instruments proved to be less distinct. Additionally, the newer systems-
level measures did not fit into the original categorization scheme. Therefore, the measures in Volume II are
divided into measures of individual recovery (e.g., The Mental Health Recovery Measure) and measures of
recovery promoting systems or environments (e.g., The Recovery Oriented Service Evaluation).

It should be noted that this categorization refers only to the phenomena measured by the instruments and
not to their intended use: the individual recovery measures can be, and in many cases have been, used to
measure program or system impact. When used in this way, the individual measures examine consumer
recovery as the marker of program success. Similarly, systems instruments may also contain components
that measure consumer level of recovery alongside such components measuring constructs as agency orien-
tation and practice. Readers interested primarily in individual measures may want to review the Recovery
Enhancing Environment Measure (REE), a systems measure containing a subscale that has been used on
its own to measure individual-level recovery.

Report Completion

Internal Review Process: Evaluation Center staff implemented the literature review, author contact
process and initial draft writing of Volume II. Once the draft was complete, key personnel from the Na-
tional Empowerment Center (NEC), the Substance Abuse and Mental Health Services Administration
(SAMHSA), and the National Association of State Mental Health Program Directors National Technical
Assistance Center (NASMHPD NTAC) were asked to review the draft and offer feedback.

Author Review: Wherever possible, the Evaluation Center staff retained the instruments’ authors’ lan-
guage in the instrument reviews. Some editing was required for the purpose of maintaining consistency
across reviews. Following this final editing, the reviews were sent back to the appropriate authors for
final approval. Again, multiple attempts were made to contact authors, as necessary.

Dissemination and Ongoing Development

The instruments in this volume are at various stages of development with many not being in a final form:
some instruments have not been tested; others have limited testing; and others have established psycho-
metric properties. As testing and data analyses continue to inform the development of many of the instru-
ments presented here, it is likely that the current information describing the instruments and the instru-
ments themselves will continue to evolve. To keep up with these changes, as well as with the development
of new instruments in the field, the development of Volume II will be an ongoing process.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 14

 The Evaluation Center @ HSRI

The Evaluation Center will update the Web version of Volume II as information is submitted to us and
inform readers that they can check the Evaluation Center Web site for this information or sign up to be
notified when new information is added to the online volume. To submit an instrument or to provide up-
dated information on an instrument currently included in either compendium, Volume I or Volume II, please
contact the manager of the Toolkits & Materials Program at the Evaluation Center. The current manager
can be found on the Toolkits and Materials Web page: http://tecathsri.org/materials.asp.

ORGANIZATION OF THE SECOND VOLUME

Following this introduction, Volume II is organized into the chapters listed below:

Chapter 2: Invitational Conference Summary and Reflections

This section includes two pieces: 1) an overview and summary of conference discussions and 2) a reflective
piece on the conference by consumer/survivor advocate Judi Chamberlin.

Chapter 3: Instrument Tables and Descriptions

As indicated previously, the recovery instruments included in Volume II are organized into two categories 1)
Measures of individual recovery, and 2) Service, program, and system level measures of recovery promoting
environments. Information on each instrument is presented in two formats: 1) An “instruments-at-a-glance”
table covering all the instruments in the category and 2) An individual, detailed, instrument description. To
view the instruments themselves, refer to Appendix D.

Chapter 4: Concluding Remarks

The compendium concludes with some brief reflections on the instruments and next steps for the
development of a quantitative recovery knowledge base.

Appendixes
The second volume includes four appendixes: A) a list of meeting participants; B) the form the Evaluation
Center used to collect information about each instrument; C) notes for non-researchers, which includes
definitions and discussions of research terms (this document was taken from Volume I); and D) the
instruments themselves.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 15

 The Evaluation Center @ HSRI

References and Suggested Readings

Anonymous. (1989). First person account: How I’ve managed my chronic mental illness. Schizophrenia
Bulletin, 15, 635-640.

Anthony, W.A. (1993). Recovery from mental illness: The guiding vision of the mental health service system
in the 1990’s. Psychosocial Rehabilitation Journal, 16(4), 11-23.

Corrigan P.W., Giffort D., Rashid F., Leary, M., & Okeke, I. (1999). Recovery as a psychological construct.
Community Mental Health Journal, 35(3), 231-239.

Deegan, P.E. (1988). Recovery: The lived experience of rehabilitation. Psychosocial Rehabilitation Journal, 11(4),
11-19.

DeSisto, M.J., Harding, C.M., McCormick, R.V., Ashikaga, T., & Brooks, G.W. (1995). The Maine and
Vermont three-decade studies of serious mental illness. British Journal of Psychiatry, 167, 331-341.

Harding, C.M., Brooks, G.W., Ashikaga, T., Strauss, J.S., & Breier, A. (1987). The Vermont longitudinal
study of persons with severe mental illness: I. Methodology, study sample and overall status 32 years
later. American Journal of Psychiatry, 144, 718-726.

Leete, E. (1989). How I perceive and manage my illness. Schizophrenia Bulletin, 8, 605-609.

New Freedom Commission on Mental Health, Achieving the promise: Transforming mental health care in America.
Final report. (DHHS Pub. No. SMA-03-3832). Rockville, MD: 2003.

Onken, S.J., Craig, C.M., Ridgway. P., Ralph, R.O., & Cook, J.A. (2004, December). An analysis of the definitions
and elements of recovery: A review of the literature. Pre-Conference Paper prepared for the National
Consensus Conference on Mental Health Recovery and Systems Transformation. Rockville, MD.

Ralph, R.O., & Corrigan, P.W. (Eds.). (2005). Recovery in mental illness: Broadening our understanding of wellness.
Washington, DC: American Psychological Association.

Ralph, R.O., Kidder, K., & Phillips, D. (2000). Can we measure recovery? A compendium of recovery and recovery-
related instruments, Volume I. Cambridge, MA: The Evaluation Center @ Human Services Research
Institute.

Ridgway, P.A. (2001). Restorying psychiatric disability: Learning from first person recovery narratives.
Psychiatric Rehabilitation Journal, 24(4), 335-343.

U.S. Department of Health and Human Services. (1999). Mental health: A report of the surgeon general. Rockville,
MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services
Administration, Center for Mental Health Services, National Institutes of Health, National Institute
of Mental Health.

Unzicker, R. (1989). On my own: A personal journey through madness and re-emergence. Psychosocial
Rehabilitation Journal, 13(1), 71-77.

Young, S.L. & Ensing, D.S. (1999). Exploring recovery from the perspective of people with psychiatric
disabilities. Psychiatric Rehabilitation Journal, 22(3), 219-231.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 16

 The Evaluation Center @ HSRI

CHAPTER 2. INVITATIONAL CONFERENCE

CONFERENCE OVERVIEW AND SUMMARY

On November 3-4, 2004, a total of thirty consumer/survivor and non-consumer/survivor researchers were
convened to attend the invitational conference Measuring the Promise: Assessing Recovery and Self-Deter-
mination Instruments for Evidence-Based Practices. Conference participants reviewed instruments that
measure 1) the degree to which consumers achieve recovery and components of recovery and 2) the degree
to which programs, services and/or systems include processes thought to bring about recovery. The con-
ference aimed to discuss measuring recovery particularly for research and evaluation related to evidence-
based practices and systems improvement, and to identify a select number of instruments to be used for
these purposes.

The following are the main ideas expressed by conference participants. The ideas presented here do not
necessarily reflect the thoughts of all conference participants; rather, they are the ideas and themes that
emerged repeatedly during discussion.

Some conference participants expressed concern with recommending certain recovery instruments to the
field to be used to measure recovery for evidence-based practices or systems improvement. At this time,
some felt that the field is not ready to formally come to consensus on instruments to be used for this pur-
pose. A number of participants pointed out that many of the instruments are at different stages of devel-
opment, making a fair comparison of the instruments premature. Rather than rate the instruments at this
time, participants agreed that key characteristics of the instruments should be documented using a “Con-
sumer Reports” format. Such a document would provide potential users with critical information about
each instrument. To collect this information, the Evaluation Center developed the Instrument Description
Form, based substantially on input from conference participants (See Appendix B).

Measuring Recovery: Obstacles and Concerns

Participant discussion identified areas of concern and potential obstacles in measuring recovery. These
considerations were debated among individuals without conclusion:
• How do we measure recovery when there is not a common definition of recovery?
• Where do we measure recovery, i.e. in the person, community, or the program?
• How do we account for program and staff values when measuring recovery?
• How do we take into account stages of recovery?
• How do we guard against individual’s recovery scores being misused, e.g., the politics of scoring?
• What constitutes evidence? (qualitative versus quantitative)

General Concerns with Current Instruments

Participants noted several concerns about the use of recovery and recovery-related instruments:
• The current group of program/system measures focus on individuals within the mental health
system, whereas one goal of recovery is social integration, e.g., having friends and supports outside of
the system.
• Instruments have not been tested on individuals in different stages of recovery to see if they
discriminate between individuals in different stages.
• Some instruments do not take into account program and staff values.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 17

 The Evaluation Center @ HSRI

Ideas for Developing and/or Testing Instruments

Participants offered strategies for the further development and testing of existing instruments. These
included:
• Create “model programs” that promote recovery as a place to test instruments.
• Have recovered individuals who have left the mental health system inform instrument development.
• Put the instrument to use in the field and have information generated through the instrument’s
application inform its development.
• Test instruments against conceptual models of recovery (e.g., The Empowerment Model of Recovery
or The Recovery Advisory Group Recovery Model) to establish a better understanding of the
instruments in relation to the stages of recovery.
• Calibrate instruments (taking 2 or more instruments and seeing how they relate).

Promoting a Recovery Agenda

Several next steps were proposed to promote measurement that is recovery oriented and therefore
supportive of recovery oriented programs and systems.
• Develop an electronic mailing list for conference participants.
• Open up the mailing list discussion to a larger group of individuals (if desired by the majority of
conference participants).
• Contact collaborating organizations (e.g., consumer technical assistance centers, the National
Technical Assistance Center for State Mental Health Planning, the National Association of State
Mental Health Program Directors Research Institute, Inc., the Mental Health Statistics Improvement
Project) to discuss the possibility of reconvening a consumer advisory research work group.
• Send recovery instruments to evidence-based practice sites and grant recipients to be tested/used.
• Encourage evaluators to involve consumers (meaningful involvement) in all evaluation design and
implementation activities.
• Compare instruments using the “consumer reports” format identified by conference participants.
• Support testing of the discriminative validity of instruments, i.e. the ability of instruments to
discriminate among persons recovered to different degrees or in different stages of recovery.
• Provide education and raise consciousness about recovery and its measurement to stakeholders.
• Partner with conference participants and others (e.g., states) to help design and implement a multi-
site study to further test instruments, i.e. bottom up research approach.

CONFERENCE REFLECTIONS AND REMARKS

Judi Chamberlin, the National Empowerment Center

Helping to design and then participating in the Measuring the Promise: Assessing Recovery and Self-Determination
Instruments for Evidence-Based Practices conference was an exciting and exhilarating experience. As the men-
tal health field appears to be moving quickly toward embracing an “evidence-based practice” (EBP) model
that largely disregards the recent exciting work on recovery, this conference provided an opportunity for
researchers (both consumers and non-consumers) who embrace the recovery concept to discuss ways to
counterbalance this trend.

While there is no inherent reason why EBPs should be in conflict with a recovery-based mental health
system, the reality is that most EBPs were established in an environment in which the lived experiences of
consumers were not part of the process of evaluation. Because this input was lacking, the resulting EBPs
favor the status quo, while the recovery approach proposes drastic changes.
Measuring the Promise: A Compendium of Recovery Measures, Volume II 18
 The Evaluation Center @ HSRI

The Current Mental Health System and EBP

The current mental health system is an academic one, based on professional expertise, with limited roles
for consumers and family members (despite much rhetoric to the contrary). It is a medical model which
presents mental illness as a brain disease, biochemical or genetic in nature, for which treatments are large-
ly based on medication, and for which the prognosis is one of long-term maintenance.

The recovery model, on the other hand, is one that has grown out of the lived experiences of people who
have been diagnosed with mental illness, and who have learned, through trial and error, what helps and
what hinders. It is one that puts far more emphasis on peer support and on daily life than on medical
treatment per se, although such treatment is an important element for some people.

A key difference between these two models is the distribution of power. In the medical model, profes-
sionals direct the system and set its direction and tone: they decide what its most important elements are,
control funding, and determine research priorities. The primary role for consumers, in this model, is to be
compliant patients who accept the treatments that are offered.

It is not surprising that, given these realities and this distribution of power, the EBPs that have gained
acceptance are both medical and professionally directed. Since recovery-based programs, such as peer
support and self-help, are barely researched, there is little opportunity to develop the evidence that they
work. Nonetheless, consumers and survivors who have lived their own recovery experiences know that
such supports as help from peers, permanent integrated housing, educational and work opportunities, and,
for some, medication and therapy, can lead to real changes that go far beyond maintenance. As people who
have recovered often say: “We are the evidence!”

The most complete research showing the value of self-help and peer support is the Consumer-Operated
Services Program study conducted by Jean Campbell and associates and supported by the Center for
Mental Health Services, Substance Abuse and Mental Health Services Administration (see http://www.
cstprogram.org/consumer%20op/ for further details on this study). However, rigorous research on these
approaches is rarely conducted because of the inherent bias of the research establishment toward more
academic and professionally-operated models.

Shifting the Paradigm

How then to move the mental health system to embrace the recovery approach and to recognize its
theoretical validity and practical applications? One small part of making such a change is to develop and
promote the research evidence that exists, and this was the focus of the invitational conference. Bringing
together a group of researchers with a genuine belief in recovery and experience in developing research
instruments designed to measure its various aspects provided a unique opportunity to energize one
another and to confirm that our work is real and valuable.

The attendees at the invitational conference shared a set of values, among them the recognition that the
lived experiences of people who have been diagnosed with mental illness must form the basis of a new and
redesigned mental health system that maximizes opportunities for recovery and for leading valued and
worthwhile lives. We know, both from our experiences and our work, that the research enterprise is not
the dry, static, and academic field that some people assume; rather, it is a way of translating those experi-
ences into credible evidence that can be replicated and that holds enormous potential for transformation,
both of the mental health field and of individual lives.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 19

 The Evaluation Center @ HSRI

CHAPTER 3. RECOVERY MEASURES

This section contains reviews of both the measures of individual recovery and the measures of recovery
promoting environments. The instruments themselves are located in Appendix D.

USING THE INSTRUMENT TABLES AND DESCRIPTIONS

Each instrument is reviewed in two formats: 1) an instruments-at-a-glance table and 2) a narrative review.
All instruments for a given section (individual measures or measures of recovery-promoting environments)
may be compared by reading the instruments-at-a-glance table for that section. Basic measure information
available in these tables includes title and author, name of domains, versions available, and total number of
items. Information on measure development includes the involvement of consumers and members of racial
or ethnic minority groups in the instrument development process, as well as an indication of whether or
not the measure has undergone some initial testing. For instruments that have undergone testing, the
tables offer information on the type of quantitative reliability and validity testing that has been performed.
Note that an indication that such testing has been performed does mean the testing has yielded results at
or above a designated cutoff; it simply indicates that the results of the given test may be found in the narra-
tive section. Finally, the tables offer information on the availability of each measure.

Narrative instrument descriptions follow the at-a-glance tables. The instrument descriptions are com-
prised of eight sections: Introduction, Instrument Description, Practical Issues, Testing and Psychometric
Properties, Utility, Summary, Permission to Use, and References and Suggested Readings. The extent a
section is addressed varies from instrument to instrument, typically with instruments at further stages
of development having a more complete description. The absence of a section or subsection for a given
instrument generally indicates that no relevant information was available. Readers with questions relat-
ing to such subsections are directed to the instrument authors, as are readers with comments or questions
about instrument content, development, and testing.
A Note on Validity Testing:

While the Evaluation Center had originally inquired about validity testing according to form/type (e.g.,
construct, criterion, etc.), a careful review of the authors’ completed sections led us to the conclusion that
almost all authors who had performed quantitative validity testing had done so using other (generally
more established) measures as criteria. We therefore decided to organize the quantitative validity testing
segments primarily around use of other measures, noting whether such testing had been performed and, if
it had been performed, which measures had been used.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 20

Measuring the Promise: A Compendium of Recovery Measures, Volume II

Table 3.1: Instruments at a Glance: Measures of Individual Recovery

Development Reliability Validity Availability

Instrument tested/

Fee for use

Other criteria

Copyrighted
Multicultural
involvement
involvement

Established
consistency
# of items

Permission
Consumer

Test-Retest

Inter-rater

measures
Internal
Measure Author Domains Versions

used
CROS 3.0 The Colorado Hope ◊ Daily functioning ◊ Consumer
Health Networks Coping with symptoms ◊ Quality Staff
Partnership of life ◊ Treatment satisfaction VIP
38 a X X X X X Xb X X X

IMR Scales Mueser et al. No domains: intended to Consumer

measure a range of aspects of Provider
illness management and recovery 15 X X X X X Xc

MHRM Young & Bullock Overcoming stuckness ◊ Consumer

Empowerment ◊ Learning ◊
Basic functioning ◊ Well-being ◊
New potentials ◊ Advocacy 30 X X X X X Xd Xe Xf

Ohio Ohio DMH; Quality of life◊ Safety and health Consumer

Outcomes original authors ◊ Symptom distress ◊ Overall Provider
System of specific scales empowerment
67 g X X X X Xh X X

Note. Readers interested in individual level recovery measures may also want to review the section on the Recovery Enhancing Environment Measure (REE), a systems measure

The Evaluation Center @ HSRI

containing a free-standing subscale measuring individual recovery.
a
consumer & staff versions; VIP version has 33 items. b consumer version: BPRS (Overall & Gorham, 1962), staff version: BASIS-32 (Eisen, 1996), both versions: WQLI (Becker, Diamond, & Sandfort,
1993). c client version: CSI (Shern, Lee, & Coen, 1996) & RAS (Corrigan, Salzer, Ralph, Sangster, & Keck, 2004), staff version: Multnomah CAS (Barker, Barron, McFarland & Bigelow, 1993). d
Empowerment Scale (Rogers, Chamberlin, Ellison, & Crean, 1997), CD-RISC (Connor & Davidson, 2003), Resilience Scale (Wagnild & Young, 1993) & Community Living Scale (Smith & Ford, 1990). e
discriminated between groups at different levels of recovery based on participation in recovery programming; reflected change in participants in an evidence-based practice designed to promote recovery.
f
free for use if citation and contact info retained on form. g consumer version; provider version has 12 items, 3 of which have sub-items. h items were drawn from original scales that had established validity
with symptom and quality of life measures
21
Measuring the Promise: A Compendium of Recovery Measures, Volume II

Table 3.1: Instruments at a Glance: Measures of Individual Recovery (Continued)

Development Reliability Validity Availability

Other criteria

Copyrighted
tested/used
Multicultural
# of items

Involvement
involvement

Established
consistency

Fee for use

Permission
Consumer

Instrument

Test-Retest

Inter-rater

measures

required
Internal
Measure Author Domains Versions
POP Campbell, Demographics ◊ Service use ◊ Consumer
Cook, Jonikas, & Employment ◊ Community life ◊
Einspahr Quality of life ◊ Well-being ◊ 241 X X X X X Xj X Xk
Program satisfaction i
Reciprocal Silver, Bricker, No domains, intended to measure Consumer
Support Scale Pesta, & Pugh mutual support
14 X X X Xl

RAS Giffort, Confidence/hope ◊ Willingness Consumer

Schmook, to ask for help ◊ Goal orientation
Woody, ◊ Reliance on others ◊ No 41 X X X X Xm
Vollendorf, & domination by symptoms
Gervain
RMT Ralph Not yet established; designed Consumer
to measure internal and external
factors relevant to one or more 91 X Xn
stages of recovery
RAFRS Leavy, McGuire, Relationships ◊ Activities Consumer
Rhoades, &
20 X Xo X
McCool

The Evaluation Center @ HSRI

i
listed are modules, each of which is comprised of one or more scales; see narrative section for further details. j recovery scale: RAS (Giffort, Schmook, Woody, Vollendorf & Gervain, 1995) & Social
Acceptance Scale (Campbell and Schraiber, 1989). k permission required for use by non-consumer researchers and organizations. l correlated with an original self-esteem scale. m Rosenberg Self-
Esteem (Rosenberg, 1965), Empowerment Scale: Self-orientation (Rogers et al., 1997), Social Support Questionnaire (Sarason, Levine, Basham & Sarason, 1983) & Subjective QOL (Lehman, 1983).
n
author requests data from instrument use. o involvement limited; see narrative section for further detail
22
Measuring the Promise: A Compendium of Recovery Measures, Volume II

Table 3.2: Instruments at a Glance: Recovery Promoting Environment Measures

Development Reliability Validity Availability

Instrument tested/

Other criteria

Copyrighted
Multicultural
involvement

involvement

Established
consistency

Fee for use

# of items

Permission
Consumer

Test-Retest

Inter-rater

measures

required
Internal
Measure Author Domains Versions

used
AACP-ROSE American Administration ◊ Treatment ◊ One version 46
Association Supports ◊ Organizational Culture that may be
of Community completed
Psychiatrists by most
X X X
stakeholders

REE Ridgway Stage of recovery ◊ Recovery Consumer 166

elements ◊ Program performance
◊ Special needs ◊ Org. climate ◊
Recovery markers ◊ Feedback a X X X X X X Xb

ROSI Onken, Person-centered decision- Consumer 42d

Dumont, making & choice ◊ Invalidated Survey,
Ridgway, personhood ◊ Self-care & Admin. data
Dornan, & wellness ◊ Basic life resources profile
Ralph ◊ Meaningful activities &
roles ◊ Peer advocacy ◊ Staff
treatment knowledge ◊ Access
◊ Peer support ◊ Choice ◊ X X Xe Xe Xf Xg
Staffing ratios ◊ System culture &
orientation ◊ Consumer inclusion
in governance ◊ Coercion c

The Evaluation Center @ HSRI

RSA O’Connell, Life goals ◊ Involvement ◊ Diversity Person in 36
Tondora, of treatment options ◊ Choice ◊ recovery
Croog, Evans, Individually tailored services Family/
& Davidson Advocate
X X X X Xf
Provider
Agency CEO
a
also contains demographics section. b fees to be determined. c tentative – based on prototype testing and originating Phase I domains/themes. d
consumer survey; administrative data profile
has 23 items. e prototype testing only. f permission recommended. g fees to be determined; will include access to technical assistance
23
 The Evaluation Center @ HSRI

MEASURES OF INDIVIDUAL RECOVERY

1. Consumer Recovery Outcomes System (CROS 3.0)

2. Illness Management and Recovery (IMR) Scales

3. Mental Health Recovery Measure (MHRM)

4. Ohio Mental Health Consumer Outcomes System

5. Peer Outcomes Protocol (POP)

6. Reciprocal Support Scale

7. Recovery Assessment Scale (RAS)

8. Recovery Measurement Tool Version 4 (RMT)

9. Relationships and Activities that Facilitate Recovery Survey (RAFRS)

Measuring the Promise: A Compendium of Recovery Measures, Volume II 24

 The Evaluation Center @ HSRI

1. CONSUMER RECOVERY OUTCOMES SYSTEM (CROS 3.0)

THE COLORADO HEALTH NETWORKS PARTNERSHIP
Information provided by Anita Miller, Psy.D.
Introduction
Aim: The Consumer Recovery Outcomes System (CROS) project started in 1997. Consumers and profes-
sionals beginning to adopt the recovery model wanted to demonstrate achievements in rehabilitation that
went beyond symptom reduction and decreased levels of service usage. The developers’ mission was to cre-
ate, with consumer and staff input, an outcomes system that would provide meaningful data on previously
untapped aspects of rehabilitation, and that would present these data in a useful way.
Conceptual Foundation: Developers designed the CROS based on a theoretical model of change that
involves each of the instrument’s domains: Hope for the Future, Daily Functioning, Coping, and Quality
of Life. The model proposes that a combination of hope and dissatisfaction with current circumstances
creates readiness for rehabilitation. As skills are learned and developed, functioning and coping increase.
This cycle continues because as the consumer gains confidence, new hope emerges and new goals are for-
mulated.
Within mental health settings, a consumer’s satisfaction with treatment can be a crucial factor in psy-
chiatric rehabilitation. In addition, interpersonal factors between 1) consumer and staff, 2) consumer
and family/friends, and 3) staff and significant others can provide great boosts or barriers to the recovery
process. The capability to compare answers and scale scores among CROS respondents provides an oppor-
tunity to achieve consensus about a consumer’s strengths, goals, and progress.
Development: Questions were drafted based on feedback obtained through focus groups of consumers
and staff who were asked what information would be useful in measuring consumer movement toward
their personal definition of recovery. The initial Consumer & Staff versions of CROS were piloted and
revised into CROS Version 1.0, which included a draft Very Important Person (VIP) Questionnaire. After
analyzing internal consistency and scale correlations, CROS 2.0 was developed and fully implemented in
outpatient mental health settings. CROS 3.0 was created in order to shorten the scales and provide con-
gruence between questions on the three separate questionnaires so that scale scores could be directly com-
parable. A CROS training course was designed and a questionnaire and report processing were automated.

Back
Stakeholders Involved in Instrument Development: Consumer/survivors, providers (clinicians), re-
searchers, advocates, and administrators.
Involvement of Consumer/Survivors in Instrument Development: Each of the three versions of CROS was presented
to consumer focus groups for feedback on 1) what information would be useful, 2) the clarity, ease, and
accuracy of interpretation of the questions, 3) the format of the questionnaires, 4) the burden and time
needed for completion of the questionnaires, and 5) the format and content of the various reports. In men-
tal health systems where the instrument was part of the standard treatment process, consumers/survivors
were asked to share general thoughts about CROS and, more particularly, how its implementation had
impacted treatment, either through the use of the individualized report in the treatment planning process
or through the study of aggregate data and outcomes over time.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 25

 The Evaluation Center @ HSRI

Back Instrument Description

Versions of the Instrument:
 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups
1. Consumer CROS 3.0 Questionnaire
2. Staff CROS 3.0 Questionnaire
3. Very Important Person (VIP) CROS 3.0 Questionnaire

Items and Domains: The Consumer and Staff versions of the CROS both contain 38 items. The VIP Ques-
tionnaire contains 33 items. All versions of the CROS are comprised of subscales that measure the domains
Hope for the Future, Daily Function, Coping with Clinical Symptoms, and Quality of Life (Table 3.3). The
Consumer Questionnaire includes one additional subscale that measures the domain Treatment Satisfac-
tion, while the Staff version has five questions that relate primarily to service use. All versions have three
additional items that relate to medication and substance use. Domains were developed through content
and factor analyses. Items in all versions are rated on a 4-point Likert scale. The descriptive anchors used
to rate the items vary.
Table 3.3
CROS Subscales and Items
Subscale Items
Hope for the future 7
Daily function 8
Coping with clinical symptoms 6
Quality of life 9
Treatment satisfaction (Consumer Version only) 5
Additional items: medication & substance use 3

CROS Sample Items

CROS Consumer Version:

I plan and keep a daily routine on my own.
Response options: all the time, most the time, sometimes, almost never

CROS Staff Version and VIP Version:

How often does the consumer plan and keep a daily routine on his/her own?
Response options: all the time, most the time, sometimes, almost never

Populations: The CROS is intended for use with adults from diverse ethnic/racial backgrounds who have
been diagnosed with a serious mental illness, have a dual diagnosis, or who receive services for substance
abuse. The sample of respondents during testing, included individuals diagnosed with a serious mental
illness and individuals with a dual diagnosis. Two ethnic/racial groups were included in the sample:
White or Caucasian, and Hispanic or Latino. Statistical comparisons of Caucasians and Hispanics found
no significant differences of Consumer Scale scores across the two groups. However, staff scored Hispanic

Measuring the Promise: A Compendium of Recovery Measures, Volume II 26

 The Evaluation Center @ HSRI

consumers significantly higher than Caucasians in two domains: Hope for Future and Quality of Life.
Differences between consumers who did or did not also carry a substance abuse diagnosis have not been
examined.

Service Settings: The CROS is intended for use in inpatient settings, outpatient settings, peer-run pro-
grams, and residential settings. The CROS has been administered in two service settings: outpatient and
residential.

Reading Level: Consumers were asked informally in a focus group setting to comment on the readability
and clarity of the CROS; this feedback indicated that the respondents considered the reading level of the
instrument to be appropriate.

Translations: None

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify)

Administration Time: CROS takes an average of 5-15 minutes for all respondents (consumers, staff, and
family/friends) to complete.

Qualification/Training Requirement: The only requirement is that the CROS administrators receive a
brief training in the purpose, goals, and format of the questionnaires so that they can answer respondents’
questions.

Scoring: There are explicit guidelines indicating how to score responses.

Supporting Material Available: Available material includes: Informed consent material; information
on administering the instrument; interviewer/administration training; guidelines to scoring responses;
guidelines to interpreting data scores; technical assistance; training on the “Treatment Progress Report,”
CROS’s automated report on a specific set of data, which is focused on sharing CROS outcomes with con-
sumers to strengthen the collaborative working relationship during the treatment planning session; and
training on how to select data to populate CROS’s Aggregate and Outcomes reports and how to evaluate
these reports.

Testing and Psychometric Properties

Cognitive Testing: Specific cognitive testing did not inform the development of this instrument although
consumers were asked informally in a focus group setting to comment on the readability and clarity of the
instrument.

Field Testing: Analysis of psychometric characteristics of the most current version of CROS (CROS 3.0)
is based on an initial sample of 576 consumers and staff who were receiving services from or working at
community mental health centers. About one-third of the cases were obtained from a mental health center
in a medium-size Colorado city; another third came from five rural mental health centers in Colorado; and

Measuring the Promise: A Compendium of Recovery Measures, Volume II 27

 The Evaluation Center @ HSRI

the final third came from a mental health center in a small Nebraska city. The mean age of the consumer
population was 45 with a range of 22 to 98 years. Forty-seven percent of the consumers were male. About
80% of the consumer population was Caucasian, while 16% was Hispanic. The most common primary
diagnosis (49%) was some type of schizophrenic disorder; 20% had a primary diagnosis of depressive dis-
order; 17% had a primary diagnosis of bipolar disorder; 13% had a dual diagnosis.

Reliability
Back
The values of the test statistics for CROS 3.0 internal consistency, test-retest reliability and interrater
reliability are offered in the following paragraphs and tables. Please note that the names of the specific
techniques used to generate these statistics were not available at the time of publication of this volume.

Internal Consistency: Table 3.4 shows the internal consistency statistics by subscale for both the consumer
and staff version scales.

Internal Consistency of CROS Subscales

Consumer
Table 3.4 Version Staff Version
Hope for Future .90 .89
Daily Functioning .83 .86
Coping with Clinical Symptoms .86 .85
Quality of Life .84 .89
Treatment Satisfaction .79 n/a

Test-Retest Reliability: Test-retest reliability for the consumer version was determined using data from a
sub-sample of 102 consumers who completed the CROS twice, with an average of eight days between
administrations. The staff version was administered to sub-sample of 106 staff, with an average of eleven
days between administrations. The results for both are shown in Table 3.5.
Table 3.5
Test-Retest Reliability of CROS Subscales
Consumer Version Staff Version
Hope for Future .70 .80
Daily Functioning .69 .89
Coping with Clinical Symptoms .76 .79
Quality of Life .75 .87
Treatment Satisfaction .74 n/a

Interrater Reliability: Interrater reliability for the staff version was tested using responses from 97 pairs of
staff, with staff in each pair having completed CROS 3.0 in reference to the same consumer. Test statistic
values for the subscales ranged from .47 to .65.

Validity
Face Validity: The measurement of the face validity of CROS 3.0 is an ongoing process as the consumers,
staff, and family/friends who are familiar with CROS 3.0 are asked on a periodical basis to give feedback
about whether the instrument makes sense in terms of content, domain identification, and CROS 3.0’s re-
lationship to the recovery process.

Relationship to Established Measures: Three existing assessment instruments were chosen to examine the
validity of the CROS 3.0: the Behavior and Symptom Identification Scale (BASIS-32) (Eisen, 1996) for

Measuring the Promise: A Compendium of Recovery Measures, Volume II 28

 The Evaluation Center @ HSRI

consumers, the Brief Psychiatric Rating Scale (BPRS) (Faustman, 1994, Miller & Faustman, 1996; Overall
& Gorham, 1962) for staff, and the consumer and staff versions of the Wisconsin Quality of Life Index
(WQLI) (Becker, Diamond, & Sainfort, 1993). The analysis was based on a subset of 322 consumers and
321 staff from the original group. Correlations of CROS 3.0 Consumer Scale scores with BASIS-32 total
and individual scale scores and WQLI scores are “relatively high.” Correlations between the four CROS
3.0 Staff scales and the BPRS and the staff WQLI are all substantially positive without suggesting that the
same factors are being measured.

Refusal Rate: The refusal rate in the CROS 3.0 instrument development studies was 0%. In mental health
settings where CROS 3.0 is being used on a voluntary, regular basis (e.g., once every 6 months), the refusal
rate is approximately 5%. Orienting the consumer to the instrument, its usefulness in their individual
treatment process, and its potential to bring consumers’ voices to bear on the larger mental health system
has proven effective in increasing participation.

Rates of Missing Data: The rate of missing data for both consumers and staff is less than 0.001. In other
words, questions were answered 99.99% of the time.

Normality Testing: Scores on all the consumer and staff scales are negatively skewed. Consumer scores
are consistently more skewed than staff scores. In addition, consumer scale score means are consistently
and significantly (p<. 01) higher than staff scale score means. The results regarding consumer-report of
treatment satisfaction are consistent with overwhelmingly positive ratings of treatment satisfaction that
have been repeatedly reported in previous research.

Plans for Further Testing and Evaluation: Researchers are planning a national study of CROS 3.0, which
will include over 1100 consumer/staff pairs.

Utility
Quality Improvement Uses: Using the program evaluation reports, the CROS Aggregate Report, and
the CROS Outcomes Report, clinicians, clinical supervisors, and administrators in mental health settings
can benchmark program effectiveness, monitor the impact of clinical or quality improvement initiatives,
and collect needs assessment data. CROS was designed with program evaluation efforts in mind so many
different training materials and other supports are available.

Intended Level of Analysis: Data generated from using the CROS may be analyzed at multiple levels:

Program Level:
 Provider Organization
 Specific Service

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency

 Individual
 Other (specify): The consumer’s support system outside the mental health system

Current/Past Uses: CROS has been and continues to be used in a variety of community mental health settings.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 29

 The Evaluation Center @ HSRI

Summary
Strengths:
• Created with consumer input.
• Robust psychometric testing results.
• Capable of eliciting matched data from three respondent perspectives.
• Process and data are strengths-based and recovery-oriented.
• Designed to be clinically useful in collaborative treatment approaches; reports are user-friendly to
consumers and providers.
• Highly automated processing with data sharing and data storage features available.

Weaknesses:
• Has not been tested on an ethnically or geographically diverse population. The upcoming large and
national study will alleviate this weakness.
• Significance when used outside recovery-oriented mental health services is unknown.
• Low ceiling on the consumer instrument might make identification of change over time difficult.
Additional training materials emphasize instructing consumers in how to interpret the rating
alternatives available and providing frank appraisals of their clinical status.
• Psychometric status of the VIP version is not fully understood. Initial attempts at gathering such
data have revealed a number of difficulties. Some consumers are estranged from family members and
may have few other knowledgeable acquaintances. Some consumers wish to retain a certain level of
privacy regarding their clinical status and are reluctant to involve friends or family members in their
treatment planning or evaluation. In addition, staff are sometimes hesitant to collect information
from family or friends of consumers. Gaining the perspective of this third group of people remains an
important endeavor.
• Predictive validity has yet to be established. Future studies need to address such questions as: 1)
How are CROS 3.0 scores related to treatment history?; 2) Do improvements in CROS 3.0 scores
signal recognizable clinical improvement or potential for clinical improvement?; 3) Do patterns of
CROS 3.0 scores provide useful clues for treatment planning?; 4) How do clinical staff make use
of CROS 3.0 in comparing their perceptions of the consumer’s strengths and disabilities with the
consumers’ own assessments?

Permission to Use
Back
• The CROS is copyrighted by CROS,LLC. Permission is required from CROS, LLC prior to using
the instrument. Prices for using the CROS vary depending on whether the Complete Processing
Package Option or the Site License Option is selected.
• The Complete Processing Package includes questionnaires, training, data processing and scoring,
technical support and a variety of reports. Agencies pay a subscription fee for each consumer who
will participate in CROS. CROS is priced on a per user, per month (pupm) basis. Final pricing is
determined by volume and number of planned administrations per year. For 2 administrations per
year, the price ranges from $7176.00 for 100 consumers to $10,800.00 for 500 consumers.

An agency choosing the Annual Site License option will get permission to reproduce and use the question-
naires. An administration manual and scoring instructions are included. Algorithms for spreadsheet scor-
ing and the production of the various reports are available for an additional $50.00. The Site License prices
range from $200.00 per year for use of the Consumer Questionnaire for 1-99 consumers to $400.00 per year
for use of the Consumer, Staff and VIP Questionnaires for 100+ consumers.
Measuring the Promise: A Compendium of Recovery Measures, Volume II 30
 The Evaluation Center @ HSRI

Instrument contact:
Anita Miller, Psy.D.
CROS, LLC
7150 Campus Drive, Suite 300
Colorado Springs, CO 80920
Phone: 800-804-5040 ext.1444
Email: anita.miller@valueoptions.com
Web site: http://www.crosllc.com

References and Suggested Readings

Becker, M., Diamond, R., & Sainfort, F. (1993). A new patient focused index for measuring quality of life in
persons with severe and persistent mental illness. Quality of Life Research, 2, 239-251.

Bloom, B.L., & Miller, A. (2004). The Consumer Recovery Outcomes System (CROS 3.0): Assessing clinical status and
progress in persons with severe and persistent mental illness. Unpublished manuscript retrieved October 13,
2005 from the CROS LLC Web site: http://www.crosllc.com/CROS3.0manuscript-090204.pdf

Eisen, S.V. (1996). Behavior and symptom identification scale (BASIS-32). In L.I. Sederer & B. Dickey,
(Eds.), Outcome assessment in clinical practice (pp. 65-69). Baltimore: Williams &Wilkins.

Faustman, W.O. (1994). Brief psychiatric rating scale. In M.E. Maruish (Ed.), The use of psychological testing for
treatment planning and outcome assessment (pp. 371-401). Hillsdale: N.J., Lawrence Erlbaum.

Miller, L.S. & Faurstman, W.O. (1996). Brief psychiatric rating scale (BPRS). In L.I. Sederer & B. Dickey
(Eds.), Outcome assessment in clinical practice (pp. 105-109) Baltimore: Williams & Wilkins.

Overall J.E., & Gorham, D.R. (1962). The brief psychiatric rating scale. Psychological Reports, 10, 799-812.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 31

 The Evaluation Center @ HSRI

2. ILLNESS MANAGEMENT AND RECOVERY (IMR) SCALES

MUESER, K.T., GINGERICH, S., SALYERS, M.P., MCGUIRE, A.B., REYES, R.U., & CUNNINGHAM, H.

Information provided by Kim Mueser, Ph.D., and Michelle Salyers, Ph.D.

Introduction

Aim: Researchers developed the Illness Management and Recovery (IMR) Scales (Mueser, Gingerich,
Salyers, McGuire, Reyes, & Cunningham, 2004) to measure outcomes targeted by the Illness Management
and Recovery Program. The IMR program is an evidence-based practice designed to assist individuals with
psychiatric disabilities develop personal strategies to manage their mental illness and advance toward
their goals.

Conceptual Foundation: The IMR Scales were developed as a measure of illness management, based on
the stress-vulnerability model of severe mental illness. According to this model, the severity of a mental
illness and likelihood of relapses are determined by the interaction between biological vulnerability and
socio-environmental stressors, both of which can be influenced. Biological vulnerability can be reduced
by adherence to prescribed medications and reduction or avoidance of alcohol or drug use. The effects of
stress on vulnerability can be reduced by improved coping skills, social support, and involvement in mean-
ingful activities.

Development: Items were generated by IMR practitioners and consumers in order to tap the various con-
tent areas targeted by the IMR program with as few items as possible. Feedback was obtained from other
clinicians and consumers about item selection and the wording of items and modifications were under-
taken accordingly.

Stakeholders Involved in Instrument Development: Consumer/survivors, family/friends of consumer/

Back
survivor, members of racial and ethnic minority groups, providers, researchers, and advocates.

Involvement of Consumer/Survivors in Instrument Development: Consumer/survivors were among the stakeholders

involved in the development of the IMR program, upon which the IMR Scales are based. Consumer/survi-
vor involvement included helping to generate different items and reviewing multiple versions of the scales.
Involvement of Members of Racial/Ethnic Minority Groups in Instrument’s Development: Some of the consumers de-
scribed above were members of ethnic/racial minority groups.
Instrument Description
Versions of the Instrument:
 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups
1. Client version
2. Clinician version

Items and Domains: Both versions of the IMR Scales, the Clinician Version and the Client Version, con-
tain 15 items. The Scales are not divided into domains; rather, each item addresses a different aspect of
illness management and recovery. All items are rated on a 5-point Likert scale with the response anchors
varying dependent upon the item.
Measuring the Promise: A Compendium of Recovery Measures, Volume II 32
 The Evaluation Center @ HSRI

Illness Management and Recovery Sample Items

Client Version:
Progress towards goals: In the past 3 months, I have come up with…
Response options: 1= No personal goals; 2= A personal goal, but have not done anything to finish the goal; 3 = A personal goal and
made it a little way toward finishing it; 4 = A personal goal and have gotten pretty far in finishing my goal; 5 = A personal goal
and have finished it.
Clinician Version:
Progress towards goals: In the past 3 months s/he has come up with…
Response options: 1= No personal goals; 2= A personal goal, but has not done anything to finish the goal; 3 = A personal goal and
made it a little way toward finishing it; 4 = A personal goal and has gotten pretty far in finishing the goal; 5 = A personal goal and
has finished it.

Populations: The IMR Scales are intended to be used to assess adults from diverse ethnic/racial back-
grounds who have been diagnosed with a serious mental illness, including those who have a dual diag-
nosis. Testing of the instrument included an ethnically/racially diverse sample (Asian, Black or African
American, White, Hispanic or Latino) of respondents who had a diagnosis of serious mental illness, some
of whom had a dual diagnosis. Subgroup analyses have not been conducted to determine whether signifi-
cant differences exist across ethnic/racial groups or among groups with different diagnoses.

Service Setting: The IMR Scales are intended for use in an array of service settings including the criminal
justice system, inpatient service settings, outpatient service settings, peer-run programs, and residential
service settings. Testing was conducted using a sample of respondents drawn from an outpatient service
setting.

Reading Level: Respondents’ informal feedback suggests that they found the reading level appropriate.

Translations/Adaptations: Hebrew. A Spanish translation is underway.

Practical Issues
Method of Administration:

 Self-administered  Group interview

 Self-administered in a group  Individual interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify):

Administration Time: Individual interview takes approximately 10 minutes.

Qualification/Training Requirement: None.

Scoring: Items are summed on the IMR Scales (separately) to form a single score for each scale.

Supporting Material: Informed consent material, guidelines to scoring responses, and technical assis-
tance.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 33

 The Evaluation Center @ HSRI

Testing and Psychometric Properties

Cognitive Testing: None.

Field Testing: Initial psychometric testing was conducted using responses from 50 adults with severe
mental illness served in a large psychosocial rehabilitation agency and 20 clinicians. Participants (consum-
ers and clinicians) completed the scales twice with an interval of two weeks between each administration.

Back Reliability
Internal Consistency and Test-Retest: Both internal consistency and test-retest statistics are shown in Table 3.6.
As noted above, test-retest results are based on an interval of two weeks between first and second admin-
istration of the scale.
Table 3.6
IMR Scales Internal Consistency and Test-Retest Statistics
Version Cronbach’s alpha r
Client version .70 .82
Clinician version .71 .78

Validity
Face Validity: IMR toolkit developers, researchers, and clinicians using the toolkit reviewed the items for
comprehensiveness and applicability to the modules covered in the IMR program.

Relationship to Established Measures: Validity of the Clinician IMR Scale was supported by a significant corre-
lation (r = .48, p < .001) between the Clinician IMR Scale and clinician rated functioning on the Multnomah
Community Ability Scale (Barker, Barron, McFarland & Bigelow, 1993). Validity of the Client IMR Scale
was supported by significant correlations between the Consumer IMR Scale and self-reported symptom
distress on the Colorado Symptom Inventory (Shern, Lee, & Coen, 1996) and perceptions of recovery on
the Recovery Assessment Scale (Corrigan, Salzer, Ralph, Sangster, & Keck, 2004) (r = -.38 and .54, p < .01)
(Table 3.7).
Table 3.7
IMR Scales Correlations with Other Measures
Scale r
IMR Scale Client version and Colorado Symptom Inventory .38
IMR Scale Client version and Recovery Assessment Scale .54
IMR Scale Clinician version and Multnomah Community Ability Scale .48

Further testing and evaluation: Currently, the developers are examining the criterion validity of the IMR
Scales by studying the relationship between the IMR ratings and hospitalization and employment in the
context of an implementation study.

Utility
Quality Improvement Uses: If done quarterly (or some other regular interval), results can be fed back to
clinicians and consumers to inform progress in IMR or other illness self-management training programs.
The results can be used to track progress over time, and to compare between programs. This is currently
being done in a statewide implementation of IMR.

Intended Level of Analysis: Individual

Measuring the Promise: A Compendium of Recovery Measures, Volume II 34

 The Evaluation Center @ HSRI

Current/Past Uses: The IMR Scales are currently being used to guide clinical practice and to evaluate the
impact of the IMR program in research. Clinically, the consumer and clinician can both rate the consumer
on progress and then compare results to discuss perceptions of progress in the program.

Summary

Strengths:
• Strong face validity.
• Brief and easily administered.
• Designed to tap a range of illness self-management domains.
• Informed by theory (the stress vulnerability model).
• Developed by a team of researchers, clinicians, and consumers.
• Includes objective descriptors for the ratings.
• Provides both consumer and clinician ratings of IMR.

Weaknesses:
• Validation process is in the early phases.
• Predictive validity of the scales still needs to be assessed.

Permission to Use

The IMR Scales are not copyrighted and can be used freely without contacting the authors or listed con-
tact. There is not a user’s fee associated with the scales.

Instrument contact:
Kim T. Mueser, Ph.D.
New-Hampshire-Dartmouth Psychiatric Research Center
Main Building
105 Pleasant St.
Concord, NH 03301
Email: Kim.T.Mueser@Dartmouth.edu

References and Suggested Readings

Barker, S., Barron, N., McFarland, B., & Bigelow, D. (1993). Multnomah community ability scale: user’s manual.
Western Mental Health Research Center, Oregon Health Sciences University. Portland, OR.

Corrigan, P. W., Salzer, M., Ralph, R., Sangster, Y., & Keck, L. (2004). Examining the factor structure of the
Recovery Assessment Scale. Schizophrenia Bulletin, 30, 1035-1041.

Mueser, K.T., Gingerich, S., Salyers, M.P., McGuire, A.B., Reyes, R.U., and Cunningham, H. (2004). The Illness
Management and Recovery (IMR) Scales (Client and Clinician Versions). Concord, NH: New Hampshire-
Dartmouth Psychiatric Research Center.

Shern, D., Lee, B., & Coen, A. (1996). The Colorado symptom inventory: A self-report measure for psychiatric symptoms.
Tampa, FL: Louis de la Parte Mental Health Institute.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 35

 The Evaluation Center @ HSRI

3. MENTAL HEALTH RECOVERY MEASURE (MHRM)

YOUNG, S.L.,& BULLOCK, W.A
Information provided by Wesley A. Bullock, Ph.D.

Introduction

Aim: The Mental Health Recovery Measure (MHRM) (Young & Bullock, 2003) is a self-report instrument
designed to comprehensively assess the recovery process for individuals with serious mental illness. The
current level of the respondent’s recovery is assessed without relying on the measurement of symptoms or
symptom management.

Conceptual Foundation: The item content of the MHRM and the MHRM subscales are based upon a
specific conceptual model of mental health recovery that is grounded in the recovery experience of persons
with psychiatric disabilities (Young & Ensing, 1999). The conceptual subscales that emerged in under-
standing the recovery process were: Overcoming Stuckness, Self-Empowerment, Learning and Self-Re-
definition, Basic Functioning, Overall Well-Being, and New Potentials. Spirituality and Advocacy/Enrich-
ment are also recovery processes that are assessed by the MHRM.

Development: The development of the MHRM involved a grounded theory analysis of qualitative data to
develop a model of recovery based upon the experiences of individuals with psychiatric disabilities. The
model was informed by 18 interviews in which individuals with psychiatric disabilities discussed their
recovery experiences (Young & Ensing, 1999; Ralph, Kidder, & Phillips, 2000). Subsequent development
of the MHRM has been based on testing and use with over 200 mental health consumers in a variety of
inpatient, forensic, and community mental health settings. Reliability and Rasch modeling (Rasch, 1980)
resulted in a series of revisions to the original instrument, which was adapted to a 41-item scale and later
revised to its current 30-item version.

Back Stakeholders Involved in Instrument Development: Consumer/survivors, members of racial and ethnic
minority groups, and researchers.

Involvement of Consumer/Survivors in Instrument Development: The MHRM was derived from a qualitative,
grounded theory analysis of the recovery narratives of mental health consumers.

Involvement of Members of Racial/Ethnic Minority Groups in Instrument Development: Twenty-eight percent of the
sample from which the conceptual model of recovery was developed was African-American.

Instrument Description
Versions of the Instrument:

 One version of the instrument

 Baseline/follow-up versions of the instrument
 Versions for different stakeholder groups

Items and Domains: The MHRM contains 30 items. All items are rated using a 5-point Likert scale that
ranges from “strongly disagree” to “strongly agree.” The majority of the items, excluding two items that
measure spirituality, have been established as subscales to measure one of seven domains: Overcoming
Stuckness, Self-Empowerment, Learning and Self-Redefinition, Basic Functioning, Overall Well-Being,
New Potentials, and Advocacy/Enrichment. Each of these seven domains is comprised of four items. The

Measuring the Promise: A Compendium of Recovery Measures, Volume II 36

 The Evaluation Center @ HSRI

domain entitled Advocacy/Enrichment was recently established to better assess the upper end of the re-
covery trajectory by assessing advocacy activities, coping with stigma, and financial quality of life.

The original six domains (Overcoming Stuckness, Self-empowerment, Learning and Self-redefinition, Basic
Functioning, Overall Well-Being, and New Potentials) were developed from a grounded theory analysis
(Glaser & Strauss, 1967) of qualitative recovery interview data. Further psychometric analyses based on
the responses from mental health consumers informed the development of the current item content and
subscale structure of the MHRM. The methods used included a comparison of each item to the total reli-
ability within the subscale, evaluation of principal components factor analysis with Varimax rotation, and
Rasch modeling (Bullock & Young, 2003).

MHRM Sample Item

I still grow and change in positive ways despite my mental health problems.
Response options: strongly disagree, disagree, not sure, agree, strongly agree

Populations: The MHRM is intended for use and has been tested with adults diagnosed with a serious
mental illness from several ethnic/racial groups: Black or African American, White, Hispanic or Latino,
and Mixed Ethnicity (White and/or African-American and/or Latino). The most recent normative sample
(N=279) included the following percentages of minority representation: African-American 24%; Latino
4%; Mixed Ethnicity 7%; Asian .5%. No significant differences were found between ethnic groups for the
mean Total MHRM score, although the mean for African-Americans (M=83) was slightly higher than the
mean for Whites (M=78) in this sample. The mean across all ethnic groups for Total MHRM = 80 (SD=20).

Service Settings: The MHRM is intended for use and has been tested with consumers who receive servic-
es in the following service settings: criminal justice system, inpatient setting, outpatient service setting,
peer-run program, and residential service setting.

Reading Level: Flesch-Kincaid Reading Level is Grade 7.7.

Translations: None (Spanish translation currently in progress).

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify)

Administration time: Approximately five minutes.

Qualification/Training Requirement: No specific qualification or training is required. The MHRM can

be self-administered.

Scoring: There are explicit guidelines indicating how to score responses and norms with which to com-
pare data.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 37

 The Evaluation Center @ HSRI

Supporting Material Available: Available materials include guidelines to scoring responses and guidelines
to interpreting data scores.

Testing and Psychometric Properties

Cognitive Testing: None.

Field Testing: Initial psychometric analyses were performed on individuals with psychiatric disabilities
(N=180) drawn from four settings: Urban county jail, n=91; Community Support Network (“persons with
severe psychiatric disabilities adjudicated ‘not guilty by reason of insanity’ participating in assertive com-
munity treatment program”), n=30; Community mental health center clients (approximately half lived in a
group residential facilities), n=35; Consumer “Leadership Education” participants, n=24 (Bullock & Young,
2003).

Additional analyses for the current 30 item MHRM were based on responses from individuals drawn from
five community mental health center sites and two community-based sites that provide peer support for
mental health consumers (N=279).

Reliability
Back
Internal Consistency: The MHRM analyses are based on responses from 279 mental health consumers (Table
3.8). The MHRM total score: alpha =.93.
Table 3.8
MHRM Subscale Internal Consistency
Subscale alpha
Overcoming Stuckness .60
Self-Empowerment .82
Learning and Self-Redefinition .79
Basic Functioning .62
Overall Well-Being .86
New Potentials .62
Advocacy/Enrichment .66
Spiritualitya .89
a
Not currently established as a subscale.

Test-Retest Reliability: A small sample (N=18) of mental health consumers completed the MHRM at one-week
and two-week test intervals:
1-week test-retest reliability: r=.92
2-week test-retest reliability: r=.91

Validity
Face Validity: The item content was developed from statements made by consumers describing their recov-
ery process.

Relationship to Established Measures: Correlations between the MHRM total score and scores on related mea-
sures are shown in Table 3.9. The correlations between the MHRM total score and measures of empower-
ment and resilience (Breedlove, 2005) were calculated using data collected from mental health consumers
(N=150) drawn from two community mental health center sites. The correlation between the MHRM
total score and community living skills/activities of daily living has also been measured (N=180) (Bullock
& Young, 2003).

Measuring the Promise: A Compendium of Recovery Measures, Volume II 38

 The Evaluation Center @ HSRI

Table 3.9
Correlations between MHRM Total Score and Other Measures
Measure r N
MHRM and the Empowerment Scale (Rogers, Chamberlin, Ellison, & Crean, 1997) .67 150
MHRM and the Conner-Davidson Resilience Scale (Connor & Davidson, 2003) .73 150
MHRM and the Resilience Scale (Wagnild & Young, 1993) .75 150
MHRM and the Community Living Scale (Smith & Ford, 1990) .57 180

Relationship to Other Criteria: The MHRM has been shown to discriminate between groups of individu-
Back
als at different levels of recovery based on participation in treatment or recovery programming (Bullock,
Wuttke, Klein, Bechtoldt, & Martin, 2002; Bullock & Young, 2003).

The MHRM has been shown to demonstrate significant change (improvement) for individuals following
completion of an evidence-based practice (the “Illness Management and Recovery” program) designed to
promote recovery (Bullock, O’Rourke, Farrer, Breedlove, Smith, & Claggett, 2005).
Normality Testing: The MHRM uses a Likert scale scored from 0 – 4 (strongly disagree, disagree, not
sure, agree, strongly agree). Scores are obtained by summing the scores for items on the Total scale/sub-
scale. There are no reverse scored items. The scale has a theoretical range from 0 – 120. In practice, scores
have been obtained in the range from 22 – 120. The mean for the Total MHRM = 80 (SD=20), based on an
average Total MHRM score for N=215 individuals drawn from five community mental health center sites
and two community-based sites that provide peer support for individuals with serious mental illness.

Plans for Further Testing and Evaluation: The use of the MHRM as an outcome measure is continuing
to be evaluated. Normative data for the MHRM are continuing to be collected across different sites and
with different mental health consumer populations.

Utility
Quality Improvement: Refer to Current/Past Uses section below.

Intended Level of Analysis: Data generated from using the MHRM may be analyzed at multiple levels:

Program Level:
 Provider Organization
 Specific Service (program evaluation)

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency

 Individual
 Other (specify):

Current/Past Uses: The MHRM was developed for use as an individual self-report change measure and as
a program evaluation tool. The MHRM is used as an outcome measure of changes in mental health recovery
for persons who are completing individual or group treatments designed to promote the recovery process.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 39

 The Evaluation Center @ HSRI

Summary

Strengths:
• Specifically designed to provide a comprehensive assessment of mental health recovery for persons
with psychiatric disabilities, without relying on measurement of psychiatric symptom expression or
symptom management.
• Comprised of theory-grounded items, based upon a specific model of recovery that was developed
from qualitative research into the phenomenology of recovery from the perspective of persons living
with psychiatric disabilities.
• Designed for ease of use and scoring.

Weaknesses: None noted.

Back
Permission to Use
The MHRM is copyrighted. However, the instrument may be reproduced freely as long as the author
citation and author contact information is retained on the form. Users are encouraged to contact the author
for further information on scoring and normative data for the MHRM.

Instrument contact:

Wesley A. Bullock, Ph.D.

Department of Psychology (#948)
University of Toledo
2801 W. Bancroft St.
Toledo, OH 43606-3390
Phone: 419-530-2719
Email: wesley.bullock@utoledo.edu

Measuring the Promise: A Compendium of Recovery Measures, Volume II 40

 The Evaluation Center @ HSRI

References and Suggested Readings

Breedlove, A. (2005). The Role of Resilience in Mental Health Recovery. Unpublished doctoral dissertation,
University of Toledo, Toledo, OH.

Bullock, W.A., O’Rourke, M., Farrer, E., Breedlove, A., Smith M.K., & Claggett, A. (2005, August).
Evaluation of the Illness Management and Recovery (IMR) Program. Presented at the 113th annual meeting of
the American Psychological Association Meeting, Washington, D.C.

Bullock, W.A. & Young, S.L. (2003, August). The Mental Health Recovery Measure (MHRM). Presented
at the 111th annual meeting of the American Psychological Association Meeting, Toronto, Canada.

Bullock, W. A., Ensing, D. S., Alloy, V. E., & Weddle, C. C. (2000). Leadership education: Evaluation of
a program to promote recovery in persons with psychiatric disabilities. Psychiatric Rehabilitation
Journal, 24 (1), 3-11.

Bullock, W. A., Wuttke, G. H., Klein, M., Bechtoldt, H., & Martin, J. (2002). Promoting mental health
recovery in an urban county jail. New Research in Mental Health, 15, 305-314.

Connor, K.M,. & Davidson, J.R.T. (2003). Development of a new resilience scale: The Connor-Davidson
Resilience Scale (CD-RISC). Depression and Anxiety, 18(2), 76-82.

Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago, IL: Aldine.

Ralph, R.O., Kidder, K., & Phillips, D. (2000). Can we measure recovery? A compendium of recovery and recovery-
related instruments. Cambridge, MA: the Evaluation Center @ Human Services Research Institute.

Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago, IL: The University of
Chicago Press.

Rogers, E. S., Chamberlin, J., Ellison, M. L., & Crean, T. (1997). A consumer-constructed scale to measure
empowerment among users of mental health services. Psychiatric Services, 48, 1042-1047.

Smith, M. K., & Ford, J. (1990). Measuring the emotional/social aspects of loneliness and isolation. Journal
of Social Behavior and Personality, 2, 257-270.

Wagnild, G.M., & Young, H.M. (1993). Development and psychometric evaluation of the Resilience Scale.
Journal of Nursing Measurement, 1(2), 165-178.

Young, S. L. (1999). Development and evaluation of a recovery enhancement group for mental health consumers.
Unpublished doctoral dissertation, University of Toledo, Toledo, OH.

Young, S. L., & Bullock, W.A. (2003). The mental health recovery measure. Available from the University
of Toledo, Department of Psychology (#918), Toledo, OH 43606-3390.

Young, S. L., & Ensing, D. S. (1999). Exploring recovery from the perspective of people with psychiatric
disabilities. Psychiatric Rehabilitation Journal, 22, 219-231.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 41

 The Evaluation Center @ HSRI

4. OHIO MENTAL HEALTH CONSUMER OUTCOMES SYSTEM

(OHIO OUTCOMES SYSTEM)
OHIO DEPARTMENT OF MENTAL HEALTH OFF ICE OF PROGRAM EVALUATION AND RESEARCH
Information provided by Dee Roth, M.A.

Introduction

Aim: The Director of the Ohio Department of Mental Health (ODMH), Michael F. Hogan, Ph.D., was
concerned that Ohio’s mental health system did not collect sufficient data on consumer outcomes for use
as a quality indicator. To make such data available, he convened a task force, the Ohio Mental Health
Outcomes Task Force (OTF), that would develop a statewide approach to measuring consumer outcomes
in Ohio’s public mental health system. This effort resulted in the development of the Ohio Mental Health
Consumer Outcomes System (herein called the Ohio Outcomes System). Data collected through this sys-
tem will be used mainly for management of consumer care, quality improvement, and public accountability
(Ohio Mental Health Outcomes Task Force, 2001).

Conceptual Foundation: “Recovery and Resiliency are foundations of ODMH’s current initiatives. The
concepts of Recovery and Resiliency are reflected in the Outcomes System’s values, the Outcomes instru-
ments, and the measurement process.” (Ohio Department of Mental Health, 2004a, p.2)

Development: The Outcomes Task Force (OTF) developed the Ohio Outcomes System Adult Forms A
and B, as well as the Provider Adult Form A instruments, by incorporating a substantial number of items
and scales from established instruments. In addition, the OTF developed some new items. The develop-
ment of the instruments began with the OTF identifying 24 outcomes to be measured. The OTF then
reviewed 126 established outcome instruments looking for entire instruments, subscales, or single items
designed to measure the chosen outcomes. Potential instruments were reviewed based on five criteria: Di-
rect and Indirect Cost; Psychometric Properties; Consistency with Principles of Consumer Recovery and
Empowerment; Cultural Sensitivity; Consistency with OTF Outcomes; and Consistency with Principles
for Child and Adolescent Service System Program.

The Adult Form instruments are built on items and scales from the following instruments (Ohio Depart-
ment of Mental Health, 2004b):
1. Adult Consumer Forms A and B: The entire Symptom Distress Scale (Task Force on Consumer-
Oriented Mental Health Report Card, 1996).
2. Adult Consumer Forms A and B: Selected items from the Quality of Life Questionnaire (Greenley,
Greenberg, & Brown, 1997).
3. Adult Consumer Forms A and B: Selected items from the Quality of Life Interview (Lehman, 1988).
4. Adult Consumer Form A only: The entire Making Decisions Empowerment Scale (Rogers,
Chamberlin, Ellison, & Crean, 1997).
5. Provider Adult Form A only: Substantially modified Multnomah Community Ability Scale and
the Basic Living Skills scale of the Adult Functioning Scales from ODMH 508 Certification/
Recertification Face (Barker & Barron, 1993)
6. Provider Adult Form A only: Two items from the Hoosier Assurance Plan Instrument (Newman,
Deliberty, Hodges, McGrew & Tejeda, 1997).

Measuring the Promise: A Compendium of Recovery Measures, Volume II 42

 The Evaluation Center @ HSRI

Note that of the three forms, Adult Form A may be of the most interest to researchers seeking recovery
measures, given its use of the entire Making Decisions Empowerment Scale as well as its incorporation of
several independent items relevant to recovery and selected items from the Quality of Life Interview.

Stakeholders Involved in Instrument Development: Consumer/survivors, family/friends of consumer/

Back
survivors, members of racial and ethnic minority groups, providers, researchers, advocates, administrators,
local community mental health board staff, and taxpayers.

The task force charged with developing the Ohio Outcomes System included representatives from multiple
stakeholder groups. The group made decisions by consensus and used “a highly participative decision pro-
cess,” (Ohio Mental Health Outcomes Task Force, 2001).

Instrument Description
Versions of the Instrument:
 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups
1. Adult Consumer Form A (for adults with severe and persistent mental illness)-67 items
2. Provider Adult Form A -12 items, 3 of which have sub-items
3. Adult Consumer Form B (adults with less severe mental illnesses)- 39 items
Note: Outcomes instruments are also available for youth, but are not described here.

Items and Domains: The Adult Consumer Form A contains 67 items, the Provider Adult Form A contains
a total of 32 items and sub-items, and the Adult Consumer Form B contains 39 items. The Consumer
Form item totals include 6 demographic items, none of which are listed in the tables that follow. For or-
ganizational purposes, the outcomes measured by the Ohio Outcome System have been grouped into four
domains1: Clinical Status, Quality of Life, Functional Status, and Safety and Health. The Outcomes Task
Force’s choice of domains was greatly influenced by the work of Rosenblatt and Attkisson (1993), as the
Task Force mapped all of their desired outcomes into the four domains described by these authors. The
specific outcomes measured for the Adult Consumer instruments and the Provider Adult instrument are
listed in Tables 3.10, 3.11, and 3.12.
Table 3.10
Adult Consumer Form A Domains and Items
Domains Items
Overall Quality of Life (Scale) 12
Quality of Life (Independent items) 9
Financial Status (Subscale) 3
Safety and Health (Independent items) 7
Symptoms Distress (Scale) 15
Additional symptom items 2
Overall Empowerment (Scale) 28
Self-Esteem/Self-efficacy (Subscale) 9
Power/Powerlessness (Subscale) 8
Community Activism and Autonomy (Scale) 6
Optimism and Control Over the Future (Subscale) 4
Righteous Anger (Subscale) 4
Note. Some items contribute to more than one subscale.
1
The Outcomes System “recognizes that in reality many of the outcomes involve more than one domain” (Ohio Department of Mental
Health, 2004a, p.2).

Measuring the Promise: A Compendium of Recovery Measures, Volume II 43

 The Evaluation Center @ HSRI

Table 3.11
Provider Adult Form A Domains and Items
Domains Items
Community Functioning (Computed score)
Social Contact 1
Social Interaction 1
Social Support 1
Housing Stability 1
Forced Moves 1
Activities of Daily Living (Subscale) 8
Meaningful Activities (Subscale) 6
Primary Role 1
Addictive Behaviors 1
Criminal Justice 1
Aggressive Behavior 1
Safety and Health (Independent items) 9

Table 3.12
Adult Consumer Form B Domains and Items
Domains Items
Overall Quality of Life (Scale) 12
Quality of Life (Independent items) 9
Financial Status (Subscale) 3
Safety and Health (Independent items) 7
Symptom Distress (Scale) 15
Note. Some items contribute to more than one subscale.

The instruments are composed of close-ended questions and Likert scale items. The majority of items in
all three versions are Likert scale items that are rated on a 4-point or 5-point scale. The sets of descriptive
anchors used to rate these items vary.

Ohio Outcomes System Sample Items

Adult Consumer Form A and Adult Consumer Form B:

The individual is asked to rate their satisfaction with various aspects of their life over the last six months.
How do you feel about:
The amount of freedom you have?
Ratings options: terrible, mostly dissatisfied, equally satisfied/dissatisfied, mostly satisfied, very pleased
Provider Adult Form A:
How effective is the client’s social support network in helping the client meet his/her needs? NOTE: A
support network may consist of interested family, friends, acquaintances, coworkers, peers, or social clubs,
etc.
Rating options: very ineffective, ineffective, mixed or dubious effectiveness, effective, very effective, unsure

Populations/Settings: The Ohio Outcomes System is intended for use and has been tested with adults
from diverse ethnic/racial backgrounds who have been diagnosed with a serious mental illness or who
have a dual diagnosis. During testing, consumers from the following ethnic/racial groups were included

Measuring the Promise: A Compendium of Recovery Measures, Volume II 44

 The Evaluation Center @ HSRI

in the sample: American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or
Other Pacific Islander, White, Hispanic or Latino, and African Somali. Subgroup analyses have not been
conducted to establish if significant differences exist across ethnic/racial groups or among different di-
agnoses, e.g., serious mental illness or dual diagnosis. However, the plans are underway to examine such
issues in the upcoming fiscal year.

Service Settings: The Ohio Outcomes System is intended to be used in outpatient settings, peer-run
programs, and residential settings. Testing of the instruments included data collected from respondents in
all of the above service settings except for peer-run programs. The instruments have also been tested in a
community-based health clinic for Somali clients.

Reading Level: Flesch-Kincaid Reading Levels are 5.8 for the Adult Consumer Form A, 5.7 for the Adult
Consumer Form B, and 10.4 for the Provider Adult Form A.

Translations: The translations currently available for the Adult instruments include:

H Adult Consumer Form A: Japanese, Mexican, Puerto Rican, Russian, Somalian

Adult Provider Form A: Japanese
Adult Consumer Form B: Japanese, Mexican, Puerto Rican

Translation/Adaptation Method: Instruments were translated into the respective language and then
translated back to English.

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify): Adult Consumer Forms can be self-administered using a hand-held computerized
device.

Administration time:
Adult Consumer Form A – 30 to 40 minutes
Adult Consumer Form B – approximately 15 minutes
Provider Adult Form A – 5 to 10 minutes

Qualification/Training Requirement: Minimal training is needed.

Scoring: There are explicit guidelines indicating how to score responses and norms with which to com-
pare data.

Supporting Material Available: Information on administering the instrument, interviewer/administration

training, guidelines to scoring responses, and guidelines to interpreting data scores (see Quality Improve-
ment Uses section for further details).

Testing and Psychometric Properties

Cognitive Testing: None.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 45

 The Evaluation Center @ HSRI

Field Testing:
Adult Consumer Form A: Data were analyzed from nearly 1,500 individuals that participated in the Out-
comes Implementation Pilot.

Provider Adult Form A: Preliminary data analysis from the Outcomes production database was generated
from data from over 10,000 individuals.

Adult Consumer Form B: Data were analyzed from 888 individuals that participated in the Outcomes
Implementation Pilot (Ohio Department of Mental Health, 2004a).

Back Reliability
Internal Consistency: Current statistics on the internal consistency of each domain for each of the three forms
of the instrument are shown in Table 3.13.

Table 3.13
Ohio Outcomes System Internal Consistency
Cronbach’s alpha N
Adult Consumer Form A
Making Decisions Empowerment Scale .77 1,376
Quality of Life items .86 1,442
Symptom Distress Scale2 .93 1,479
Provider Adult Form A
Community Functioning Scale .72 23,540
Adult Consumer Form B
Quality of Life items .92 887
Symptom Distress Scale2 . 97 888

Please note that the internal consistency of the instruments is periodically re-examined and updated re-
sults are posted on the Ohio Mental Health Consumer Outcomes System Web site.

Validity
Relationship to Established Measures: The Adult Consumer Forms A and B symptom items were derived from
the Symptom Distress Scale, which has proven to have adequate discriminate validity with the Beck
Depression Inventory and a number of scales from the Minnesota Multiphasic Personality Inventory. Ad-
ditionally, significant correlations between the Making Decisions Empowerment Scale and symptom dis-
tress and quality of life indicators have been noted (Ohio Department of Mental Health, 2004a).

Refusal Rate: The refusal rate for the instruments is unknown. Consumers may choose not to complete
Adult Form A or B. There is a place on the forms to indicate the consumer’s choice of not responding.
However, these blank forms are not consistently submitted.

Rates of Missing Data: An analysis was done of completed and partially completed forms. The average
rates of missing responses for each form follow.

Adult Consumer Form A – 2.5 (n = 104,505)

Adult Consumer Form B – 1.2 (n = 73,302)
Provider Adult Form A – 4.5 (n = 115,544)

2
Symptom items were derived from The Symptom Distress Scale, which has proven to have adequate internal consistency and discriminate validity with the
Beck Depression Inventory and a number of scales from the Minnesota Multiphasic Personality Inventory. The Empowerment items were derived from the
Making Decisions Empowerment Scale, which has demonstrated a high degree of internal consistency.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 46

 The Evaluation Center @ HSRI

Plans for Further Testing and Evaluation: The Mexican version will be evaluated to determine the ex-
tent to which the instrument contains concepts that are understood by individuals who speak different
Spanish dialects.

Utility

Quality Improvement Uses: The data collected can be used by various constituents for different purposes.
For example, a consumer may use the data to assist him/herself in development of a treatment plan, while a
mental health board may use the data to assist themselves in identifying services that are successful or “best
practices.” Other data uses may include the identification of service areas in need of improvement, program
and system accountability, and system planning. Refer to the Ohio Mental Health Consumer Outcomes
System Procedural Manual for more information: http://www.mh.state.oh.us/oper/outcomes/outcomes.
index.html

Supporting resources include a Data Entry and Reports Template and a Data Entry and Reports Manual.
H H

In addition, Statewide Outcomes Reports are produced on a regular basis, disseminated, and posted on the
Ohio Outcomes System Web site. The reports are intended to provide constituents in the mental health
system with statewide data that they can use to compare an individual’s scores, average agency, or board
area scores.

Intended Level of Analysis: Data are intended to be analyzed at multiple levels:

Program Level:
 Provider Organization
 Specific Service

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency

 Individual Level
 Other (specify):

Current/Past Uses: The Ohio Outcomes System is currently being used to collect outcome data on
individuals with psychiatric disabilities who receive services from Ohio’s public mental health system.
Consumers and mental health practitioners use outcome data in individualized recovery planning and to
monitor progress. Outcome data are also incorporated into the Ohio Department of Mental Health’s Bal-
anced Score Card, which provides a system-wide view of the status of Ohio’s public mental health system.
In addition, local boards and agencies can use the outcome data to monitor and improve the quality of
services and programs.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 47

 The Evaluation Center @ HSRI

Summary
Strengths:
• Provides first-hand information regarding consumers’ view of their most pressing problems and
concurrent strengths. These, in turn, are used to guide development of a recovery service plan or
treatment plan.
• Provides valuable information about changes in consumers’ well-being and recovery when
administered over time.
• Capable of gathering data that can be used by agencies and/or mental health systems to project
service needs across the treatment spectrum.
Weaknesses:
• Not a diagnostic instrument.

Back
Permission to Use
The OMHCOS is copyrighted. Permission is required for use of the instrument outside of the state of
Ohio. The adult and child/adolescent sets of instruments are free for use within Ohio, however, out-of-state
parties must pay a small usage fee for the child/adolescent instruments.

Instrument contact:
Dee Roth, M.A.
Ohio Department of Mental Health
Phone: 614-466-8651
Email: rothd@mh.state.oh.us

Web site: http://www.mh.state.oh.us/oper/outcomes/outcomes.index.html H

Measuring the Promise: A Compendium of Recovery Measures, Volume II 48

 The Evaluation Center @ HSRI

References and Suggested Readings

Barker, S., & Barron, N. (1993). User’s manual for the Multnomah Community Ability Scale. Network
Behavioral Healthcare, Inc. Portland, OR.

Greenley, J.R., Greenberg, J., & Brown, R. (1997). Measuring quality of life: A new and practical survey
instrument. Social Work, 42, 244-250.

Lehman, A. (1988). A quality of life interview for the chronically mentally ill. Evaluation and Program Planning,
11, 51-62.

Ohio Mental Health Outcomes Task Force. (2001, Revised and Updated). Vital signs: a statewide approach to
measuring consumer outcomes in Ohio’s publicly –supported community mental health system. Final report of the
Ohio Mental Health Outcomes Task Force. (1996-1997). Columbus, OH. Ohio Department of Mental
Health. http://www.mh.state.oh.us/oper/outcomes/outcomes.index.html

Ohio Department of Mental Health. (2004a, February). The Ohio Mental Health Consumer Outcomes System:
Frequently asked questions. Columbus, OH. Ohio Department of Mental Health. http://www.mh.state.
H

oh.us/oper/outcomes/outcomes.index.html H

Ohio Department of Mental Health (2004b, May) The Ohio Mental Health Consumer Outcomes System Procedural
Manual, 6th edition. Columbus, OH. Ohio Department of Mental Health. http://www.mh.state.oh.us/
oper/outcomes/outcomes.index.html

Ohio Department of Mental Health (2005, February) Ohio Mental Health Consumer Outcomes at a Glance (Adult
Consumers). Columbus, OH. Ohio Department of Mental Health. http://www.mh.state.oh.us/oper/
H

outcomes/outcomes.index.html H

Newman, F. L., Deliberty, R., Hodges, K., McGrew, J., & Tejeda, M.J. (1997). Indiana Hoosier Assurance Plan
packet. Cambridge, MA: The Evaluation Center @ Human Services Research Institute.

Rogers, E.S., Chamberlin, J., Ellison, M.L., & Crean, T. (1997). A consumer-constructed scale to measure
empowerment among users of mental health services. Psychiatric Services, 48, 1042-1046.

Rosenblatt, A., & Attkisson, C.C. (1993). Assessing outcomes for sufferers of severe mental disorder: A
conceptual framework and review. Evaluation and Program Planning, 16, 347-363.

Task Force on Consumer-Oriented Mental Health Report Card. (1996). The MHSIP: Consumer-oriented mental
health report card. Washington, D.C.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 49

 The Evaluation Center @ HSRI

5. PEER OUTCOMES PROTOCOL (POP)

CAMPBELL, J., COOK, J.A., JONIKAS, J.A.,. & EINSPAHR, K.
Information provided by Judith Cook, Ph.D., and Jean Campbell, Ph.D.

Introduction
Aim: The Peer Outcomes Protocol (POP) Questionnaire (Campbell, Cook, Jonikas, & Einspahr, 2004a.)
was developed as part of the Peer Outcomes Protocol Project (POPP). This project was established
to provide mental health peer support and consumer-operated programs and groups with a validated
evaluation protocol to measure outcomes of interest to people in recovery. The protocol was developed for
use by program members and leaders/facilitators, even if they do not have access to researcher expertise
and consultation (Campbell, Cook, Jonikas, & Einspahr, 2004b.).

The POP is composed of seven modules: Demographics, Services Use, Employment, Community Life, Qual-
ity of Life, Well-Being, and Program Satisfaction. The module that is most closely related to this volume is
the Well-Being Module. This module contains two scales: the Personhood and Empowerment Scale and
the Recovery Scale.

Conceptual Foundation: The instrument used empowerment theory and peer support/self-help theo-
retical approaches that, by definition, are intended to promote recovery. Given that consumer-operated
programs are often run by volunteers who are not mandated to track and report their outcomes, it was felt
that such a protocol was needed by the field. It was also a key assumption that the protocol should be de-
signed to measure domains that are important to people in recovery themselves and that existing outcome
protocols often fall short in this regard. Finally, given the informal structure of many such programs, it
was felt that the protocol should be user-friendly and modularized to reflect the diverse goals of consumer-
run programs.

Development: The POP’s development was based primarily on the following three processes: 1) A com-
prehensive review of outcome instruments and indicators used by peer community-based support pro-
grams; 2) The results of concept mapping conducted by a consumer researcher with a group of people in
recovery; and 3) Review and refinement of items by: a Consumer Advisory Board; leading survey research-
ers in the Uunited States; University of Illinois at Chicago (UIC) National Research Training Center staff,
and staff from the UIC Survey Research Lab.

Stakeholders Involved in Instrument Development: Consumer/survivors, family/friends of consumer/

Back
survivor, members of racial and ethnic minority groups, providers, researchers, advocates, and administra-
tors.

Involvement of Consumer/Survivors in Instrument Development: As indicated above, the instrument was developed
by a consumer/survivor researcher and her staff and refined with the assistance of a Consumer Advisory
Board.

Involvement of Members of Racial/Ethnic Minority Groups in Instrument Development: Members of racial and ethnic
minorities were involved as POP developers; as reviewers; and as participants in the POP pilot-test.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 50

 The Evaluation Center @ HSRI

Back
Instrument Description
Versions of the instrument:

 One version of the instrument

 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups

Items and Domains: The POP Questionnaire contains a total of 241 items organized into seven modules:
Demographics Module, Service Use Module, Employment Module, Community Life Module, Quality of
Life Module, Well-Being Module, and the Program Satisfaction Module. Most of the modules include
scales, some of which are original scales and others that were previously established. Table 3.14 provides
an overview of the modules and, if applicable, the scales within the module. All modules were developed
through content analysis of existing instruments and concept mapping. A Principal Components Factor
Analysis with Varimax Rotation was used in psychometric testing to refine the POP’s scales.

Table 3.14

POP Modules and Scales

Items in
Modules and Scales
Modules
Demographics Module 20
Service Use Module 32
Employment Module
24
Employment Satisfaction Scale (items 3.12-3.24)
50
Community Life Module
Community Satisfaction Scale (items 4.10-4.15)
Quality of Life Scale (Lehman, 1988) (items 4.16-26)
Social Satisfaction Scale (items 4.27-4.32)
Discrimination Scale (items 4.33-4.36)
Social Acceptance Scale (Campbell & Schraiber, 1989) (items 4.43-4.50)

Quality of Life Module 28

SF-12 Health Survey (Ware, Kosinski, & Keller, 1996) (items 5.1-5.12)
Quality of Life Scale (Lehman, 1988) (item 5.13)
Subjective Quality of Life (items 5.14-5.21)
Program Quality of Life (items 5.22-5.28)
Well-Being Module 37
Personhood and Empowerment Scale (items 6.1-6.24)
Recovery Scale (items 6.25-6.37)
50
Program Satisfaction Module
Program Satisfaction Scale (items 7.1-7.40)
Coercion Scale (Campbell, Wieselthier, Einspahr, & Evenson in Campbell et al.
2004a ) (items 7.41-7.50)

Measuring the Promise: A Compendium of Recovery Measures, Volume II 51

 The Evaluation Center @ HSRI

The POP is comprised of close-ended questions, open-ended questions, and Likert scale items. All items in
the Well-Being Module are rated on a 4-point Likert scale that ranges from 1 = Disagree to 4 = Agree.

POP Well-Being Module Sample Item

I take an active role in decisions about my mental health services.

Response options: 1 = disagree, 2 = somewhat disagree, 3 = somewhat agree, 4 = agree

Populations:
The POP is intended for use and has been tested with adults from diverse ethnic/racial backgrounds-
American Indian or Alaska Native, Asian, Native Hawaiian or Other Pacific Islander, White, Hispanic or
Latino, and Black or African American- who have been diagnosed with a serious mental illness. Subgroup
analyses have not been conducted to determine if significant differences exist across ethnic/racial groups.
Service Settings: The POP is intended to be used with individuals who receive services in outpatient set-
tings, peer-run programs, and residential settings. The respondents who completed the instrument during
testing were recipients of services in outpatient and peer-run service settings.
Reading Level: The Flesch-Kincaid reading level of the components ranged from 4.6-8.2 with an overall
measurement of 7.0. However, interviews with respondents were also conducted to determine the best
mode of administration (self-administered, telephone, & face-to-face) and determined that face-to-face
was the only acceptable mode for this version of the protocol.

Translations: None

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify):_______________

Administration Time: The entire POP Questionnaire takes approximately one hour to complete. Admin-
istration time for the Well-Being Module is less since it is composed of a relatively small subset of items.
Qualification/Training Requirement: Limited training is necessary. A Question-by Question guide is
available to assist beginning interviewers administer the POP Questionnaire.

Scoring: There are explicit guidelines indicating how to score responses.

Supporting Material Available: Information on administering the instrument, interviewer/administra-

tion training, guidelines to scoring responses, guidelines to interpreting data scores, technical assistance,
response cards for interviewees.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 52

 The Evaluation Center @ HSRI

Testing and Psychometric Properties

Cognitive Testing: After determining face-to-face to be the only acceptable mode of administration for
the instrument, the developers tested the understandability of the POP by conducting a “think-aloud”
with a group of POP respondents. Revisions were made when respondents were confused by a question.

Field Testing: Psychometric analysis was performed with volunteers who were individuals with psychi-
atric disabilities and members of the St. Louis Empowerment Center (N=100). The sample consisted of the
first 100 members of the center who consented to participate in the study. The re-test sample (n=41) was
generated by randomly selecting individuals from the initial sample who were available within two weeks
after the first interview (Campbell, Einspahr, Evenson, & Adkins, 2004).

Reliability
Back
Both internal consistency and test-retest reliability statistics for all POP scales are shown in Table 3.15. As
noted above, test-retest results are based on an elapsed time of two weeks or less between first and second
instrument administrations.
Table 3.15

POP Scale Internal Consistency and Test-Retest Reliability

Internal Test-Retest
Scale Consistency
alpha (N) r (N=41)
Employment Satisfaction .71 (55) .47
Community Satisfaction .72 (96) .73
Social Satisfaction .76 (98) .54
Social Acceptance .93 (97) .66
Health .85 (99) .88
Quality of Life .74 (99) .63
Personhood .83 (100) .73
Empowerment .89 (99) .63
Recovery .86 (96) .61
Program Quality of Life .88 (99) .72
Program Satisfaction .95 (88) .78
Felt Coercion .83 (97) .85

Validity
Relationship to Established Measures: Four of the POP scales were correlated with established criterion scales.
The results are shown in Table 3.16.

Table 3.16
POP Correlations with Criterion Scales
Measures r
Social Acceptance Scale (Campbell & Schraiber, 1989) with POP’s Recovery Scale .55
Rosenberg Self-Esteem Scale (Rosenberg, 1965) with POP’s Personhood Scale .76
Recovery Assessment Scale (Giffort, Schmook, Woody, Vollendorf, & Gervain, 1995) with POP’s Recovery Scale .63
Empowerment Decision-Making Scale (Rogers, Chamberlin, Ellision, & Crean, 1997) with POP’s Empowerment Scale .46
CSQ-8 Satisfaction Scale (Larsen, Attkisson, Hargreaves & Nguyen, 1979) with POP’s Program Satisfaction Scale .55

Measuring the Promise: A Compendium of Recovery Measures, Volume II 53

 The Evaluation Center @ HSRI

Within each module, scales were correlated with one another to determine the degree to which they were
related. Results of this process are shown in Table 3.17.
Table 3.17
POP Scale Intercorrelations within Modules
Scales r
Community Satisfaction & Social Satisfaction .51
Community Satisfaction & Social Acceptance .39
Health & Quality of Life .66
Personhood & Empowerment .85
Personhood & Recovery .88
Empowerment & Recovery .79
Program Quality of Life & Program Satisfaction .83
Program Quality of Life & Coercion .38
Program Satisfaction & Coercion .59

Plans for Further Testing and Evaluation: None at this time.

Utility

Quality Improvement Uses: The POP may be used to improve the quality and delivery of the program’s
or group’s services, to identify service gaps, to secure funding , or to provide the mental health field with
information on the effect of peer-run programs and groups on consumers’ outcomes (Campbell, Cook,
Jonikas, & Einspahr, 2004b).

Intended Level of Analysis: Data are intended to be analyzed at the program level:

Program Level:
 Provider Organization
 Specific Service

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency

 Individual
 Other (specify): Peer-support programs and groups

Current/Past Uses: Parts of the POP were adapted for use in the Consumer Operated Services Program
(COSP) Multi-Site Research Initiative, funded by the Center for Mental Health Services, Substance Abuse
and Mental Health Services Administration. For more information on that study visit the COSP Web site:
http://www.cstprogram.org/consumer%20op/. H

Measuring the Promise: A Compendium of Recovery Measures, Volume II 54

 The Evaluation Center @ HSRI

Summary
Strengths:
• Developed by consumers for consumers.
• Measures outcomes of importance to people in mental health recovery.
• Available for free download.
• Psychometric testing conducted.
• Reviewed by leading survey researchers.
• Designed to permit assessment of specific domains.
• Can be used in program and support group settings.

Weaknesses:
• Not available in languages other than English.
• Not tested with children.
• Not tested with individuals who identified themselves as having substance use problems.
• Lengthy if administered in its entirety.

Permission to Use
Back
People with psychiatric disabilities and non-profit, mental health consumer-run programs/organizations
may reproduce and use the research protocol and documentation for their own personal use without
permission. The authors would appreciate being acknowledged in such instances. All other rights are
reserved and written permission must be obtained from the UIC Center. There are no user fees associated
with the POP. The POP is copyrighted by the University of Illinois at Chicago, 2004.

Instrument Contact:
Additional information may be found at the Peer Outcomes Protocol Project website: http://www.psych.
uic.edu/uicnrtc/pophome.htm

All components of the instrument (Administration Manual; A Question-by-Question Guide, Survey In-
strument; Response Cards, and Psychometric Report) are available for free download: http://www.psych.
H

uic.edu/uicnrtc/popmanual.htm. A paper copy of the materials can be obtained for $20 from the UIC Cen-
H

ter.
UIC National Research and Training Center on Psychiatric Disability
Attention: Dissemination Coordinator
104 South Michigan Avenue, Suite 900
Chicago, IL 60603
Phone: 312-422-8180
Questions about the POP can be directed to its first author.
Jean Campbell, Ph.D.
Missouri Institute of Mental Health
Email: Jean.Campbell@mimh.edu
H H

Phone: 314-877-6457

Measuring the Promise: A Compendium of Recovery Measures, Volume II 55

 The Evaluation Center @ HSRI

References and Suggested Readings

Campbell, J., Cook, J., Jonikas, J., & Einspahr, K. (2004a). Peer outcomes protocol questionnaire. Chicago, IL:
University of Illinois at Chicago.

Campbell, J., Cook, J., Jonikas, J., & Einspahr, K. (2004b). Peer outcomes protocol (POP): Administration manual.
Chicago, IL: University of Illinois at Chicago.

Campbell, J., Einspahr, K., Evenson, R., & Adkins, R. (2004). Peer outcomes protocol (POP): Psychometric
properties of the POP. Chicago, IL: University of Illinois at Chicago.

Cambell, J. and Schraiber, R. (1989). The Well-Being Project: Mental health clients speak for themselves. Sacramento,
CA: California Department of Mental Health.

Giffort D., Schmook, A., Woody, C., Vollendorf, C., & Gervain, M. (1995). Construction of a scale to measure
consumer recovery. Springfield, IL, Illinois Office of Mental Health.

Larsen, D.L., Attkisson, C.C., Hargreaves, W.A., & Nguyen, T.D. (1979). Assessment of client/patient
satisfaction : development of a general scale. Evaluation and Program Planning, 2, 197-207.

Lehman, A. (1988). A quality of life interview for the chronically mentally ill. Evaluation and Program Planning,
11, 51-62.

Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.

Rogers, E.S., Chamberlin, J., Ellision, M. L., & Crean, T. (1997). A consumer-constructed scale to measure
empowerment among users of mental health services. Psychiatric Services, 48(8), 1042-1047.

Ware, J.E., Kosinski, M. & Keller, S.D. (1996). A 12-item short-form health survey: Construction of scales
and preliminary tests of reliability and validity. Medical Care, 34(3), 220-233.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 56

 The Evaluation Center @ HSRI

6. RECIPROCAL SUPPORT SCALE

SILVER, T., BRICKER, D., PESTA, Z., & PUGH, D.
Information provided by Thelma Silver, Ph.D., LISW

Introduction
Aim: The Reciprocal Support Scale (Silver, Bricker, Pesta, & Pugh, 2002) was developed to measure
mutual support in a study designed to evaluate the development and impact of a recovery oriented
mentoring and education program entitled Leadership Class on its participants.

Back
Stakeholders Involved in Instrument Development: Consumer/survivors, family/friends of consumer/
survivor, providers, researchers, and advocates.
Instrument Description

Versions of the Instrument:

 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups

Items and Domains: The Reciprocal Support Scale has 14 items. All items are rated using a 5-point Likert
scale. Principal components factor analysis revealed that all items load onto one factor, indicating that the
Reciprocal Support Scale is a univariate measure of support.

Reciprocal Support Scale Sample Item:

I find it easy to communicate my needs to my recovery partner.

Response options: 1 = almost never, 2= rarely, 3 = sometimes, 4 = often, 5 = almost always

Populations: The Reciprocal Support Scale is intended for use with adults who have been diagnosed with
a serious mental illness or who are receiving treatment for substance abuse. During testing, the individu-
als who responded to the Reciprocal Support Scale were predominately white and carried a diagnosis of
serious mental illness or a diagnosis of substance abuse. No subgroup analyses have been conducted with
the scale.

Service Settings: The Reciprocal Support Scale is intended to be administered to and has been tested with
consumers receiving services in an outpatient program.

Reading Level: Respondents’ informal feedback suggests that they found the reading level appropriate.

Practical Issues
Method of Administration:
 Self-administered  Group interview
 Self-administered in a group  Individual interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify):

Administration Time: Approximately 20 minutes.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 57

 The Evaluation Center @ HSRI

Qualification/Training Requirement: No training is needed.

Scoring: There are explicit guidelines indicating how to score responses.

Testing and Psychometric Properties

Field Testing: Silver et al. (2002) tested the Reciprocal Support Scale with 80 individuals: 42 of the indi-
viduals were involved in mental health services as consumers or providers of community support and 38
individuals were involved in addictions treatment as a sponsor or recipient of sponsorship. The majority
of the sample, 86%, was White. The age of participants ranged from 39-57; 49% of participants were fe-
male.

Reliability
Back
Internal consistency: Cronbach’s alpha for the Reciprocal Support Scale was found to be .95.

Validity
Relationship to Established Measures: The Reciprocal Support Scale score was found to be correlated with
scores on a self-esteem scale derived from the Ohio Department of Mental Health’s Outcomes System
(Ohio Department of Mental Health, 2004). The correlation between the scales was significant (r =.28,
p<.05).

Utility

Intended Level of Analysis:

Program Level:
 Provider Organization
 Specific Service

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency

 Individual
 Other (specify):

Permission to Use
The Reciprocal Support Scale is currently not copyrighted and can be used freely. There is not a user fee
associated with the instrument.

Instrument contact:
Thelma Silver, Ph.D., LISW
Email: doovil@aol.com
H H

Measuring the Promise: A Compendium of Recovery Measures, Volume II 58

 The Evaluation Center @ HSRI

References and Suggested Readings

Ohio Department of Mental Health (2004, May). The Ohio Mental Health Consumer Outcomes System procedural
manual, 6th Edition. Columbus, OH: Ohio Department of Mental Health.

Silver, T., Bricker, B., Pesta, Z., & Pugh, D. (2002). Impact of teaching mental health best practices and
recovery processes on constituent populations of the mental health system. In D. Roth (Ed.), New
research in mental health, Vol.15 (pp.331-335). Columbus, OH: Ohio Department of Mental Health.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 59

 The Evaluation Center @ HSRI

7. RECOVERY ASSESSMENT SCALE (RAS)

GIFFORT, D., SCHMOOK A., WOODY, C., VOLLENDORF, C., & GERVAIN, M.

Information provided by Pat Corrigan, Psy.D.

Introduction
Aim: The Recovery Assessment Scale (RAS) was developed as an outcome measure for program evaluations.
For example, the RAS was incorporated into a federally funded multisite study on consumer-operated
services called the Consumer-Operated Services Program (COSP) Multisite Research Initiative.

Conceptual Foundation: Based on a process model of recovery, the RAS attempts to assess aspects of
recovery with a special focus on hope and self-determination.

Development: Giffort and colleagues developed the Recovery Assessment Scale (Giffort, Schmook,
Woody, Vollendorf, & Gervain, 1995) through narrative analysis of four consumers’ recovery stories. The
analysis informed the development of a 39-item scale. A review of the scale items by an independent group
of 12 consumers resulted in the scale’s revision. The revisions yielded the current version of the RAS,
which is a scale 41-items in length (Corrigan, Giffort, Rashid, Leary, & Okeke, 1999).

Stakeholders Involved in Instrument Development: Consumer/survivors, providers, and researchers.

Back
Involvement of Consumer/Survivors in Instrument Development: As noted above, consumers were involved in the
development of both the original scale and in the revision.

Instrument Description
Versions of the Instrument:

 One version of the instrument

 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups

Items and Domains: The RAS has 41 items. All items are rated using the same 5-point Likert scale that
ranges from 1 = “strongly disagree” to 5 = “strongly agree.” The RAS’s subscales (Table 3.18) measure five
domains: Personal Confidence and Hope, Willingness to Ask for Help, Goal and Success Orientation, Reli-
ance on Others, No Domination by Symptoms. Seventeen of the scale’s items are not incorporated into the
current factor structure.
Table 3.18
RAS Subscales and Items
Subscales Items
Personal Confidence and Hope 9
Willingness to Ask for Help 3
Goal and Success Orientation 5
Reliance on Others 4
No Domination by Symptoms 3
Corrigan, Salzer, Ralph, and Sangster (2004) used exploratory and confirmatory factor analysis (CFA) to
establish the factor structure of the RAS. Exploratory factor analysis was performed using principal com-
ponent analysis and Varimax rotation on a random subset of half of the sample. This analysis yielded eight
factors. With the remainder of the sample, structural equation models that corresponded with the item-

Measuring the Promise: A Compendium of Recovery Measures, Volume II 60

 The Evaluation Center @ HSRI

factor loadings were used to cross-validate the factors. Three factors were removed due to an unsatisfac-
tory fit. A second CFA validated the five factor structure. The alphas for the five factors ranged from .74
to .87: personal confidence and hope (alpha=.87); willingness to ask for help (alpha=.84); goal and success
orientation (alpha=.82); reliance on others (alpha=.74); no domination by symptoms (alpha =.74).

RAS Sample Item:

I have my own plan for how to stay or become well.

Response options: strongly disagree, disagree, not sure, agree, strongly agree

Populations: The RAS is intended for use and has been tested with adults from diverse ethnic/racial
backgrounds -American Indian or Alaska Native, Asian, Native Hawaiian or Other Pacific Islander, White,
Hispanic or Latino, and Black or African American- who have been diagnosed with a serious mental ill-
ness. Subgroup analyses have not been conducted to determine if significant differences exist across eth-
nic/racial groups.

Service Settings: The RAS is intended for use and has been tested with consumers who receive services in
two service settings: outpatient setting and peer-run programs.

Reading Level: Unknown.

Translations: None.

Practical Issues

Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify)

Administration Time: Individual interview takes approximately 20 minutes.

Qualification/Training Requirement: RAS interviewers must be able to reliably read and score items.

Scoring: There are explicit guidelines indicating how to score responses.

Supporting Material: Available materials include administration and scoring guidelines.

Testing and Psychometric Properties

Cognitive Testing: None.

Field Testing: The RAS has been field tested twice. Corrigan et al. (1999) initially tested the RAS with
35 consumers in the University of Chicago partial hospitalization program. Participants had a diagnosis
of serious mental illness, at least three hospitalizations within the past two years and an inability to work
as a result of their mental illness. The ethnic/racial make-up of the sample was 57.1% African American,
37.1% European American, and 5.8% other. Females made up 35.1% of the sample and the mean age was
33.1 (SD 9.2).

Measuring the Promise: A Compendium of Recovery Measures, Volume II 61

 The Evaluation Center @ HSRI

Subsequent testing examined the RAS’s factor structure and the symptom variables that are correlates of
individual factors (Corrigan et al. 2004). Analyses are based on responses from the baseline assessment of
consumers participating in the Consumer Operated Services Program (COSP) Multi-site Research Initia-
tive. The sample size was originally 1,824 but missing items possibly lowered the sample to 1,750. Partici-
pants had a DSM-IV, Axis I diagnosis consistent with serious mental illness and a significant functional
disability as a result from the mental illness. The sample included individuals from diverse ethnic/racial
backgrounds: 23.8% African American, 74.5% European American, 3.4% Latino or Hispanic, 18.1% Native
American, and 1.4% Asian or Pacific Islander. 60.1% of the sample was female and the mean age was 41.8
(SD 10.4).

Reliability
Back
Internal Consistency: RAS responses in initial testing yielded a Cronbach’s alpha =.93 (N=35).

Test-Retest Reliability: Respondents in the initial testing completed the scale twice within 14 days; Pearson
Product Moment Correlation r=.88 (N=35).

Validity
Relationship to Established Measures: As shown in Table 3.19, the RAS total score was found to be correlated
with five psychosocial variables (N=35). The RAS total score is positively associated with the Rosenberg
Self-Esteem Scale (Rosenberg, 1965) = .55, Empowerment Scale: Self-orientation (Rogers, Chamberlin,
Ellison, & Crean, 1997) = -.71, short version of the Social Support Questionnaire (Sarason, Levine, Basham,
& Sarason, 1983) = .48, and subjective component of the Quality of Life Interview (Lehman, 1983) = .62.
The RAS total score is inversely associated with the expanded version of the Brief Psychiatric Rating
Scale (Lukoff, Liberman, & Nuechterlein, 1986) = -.44, however this correlation coefficient did not meet
the Bonferroni Criterion for significance. Stepwise multiple regression indicated that the Rosenberg Self-
Esteem Scale and the Empowerment Scale scores are significant predictors of the total Recovery Scale Score
(Corrigan et al., 1999).
Table 3.19

RAS Total Score Correlations with other Measures

Measure r
Rosenberg Self-Esteem Scale .55
Empowerment Scale (Self-Orientation) -.71
Social Support Questionnaire (Short Version) .48
Quality of Life Interview (Subjective Component) .62
Brief Psychiatric Rating Scale (Expanded Version) -.44a
a
Does not meet Bonferroni Criterion for significance.

Validity of the RAS was further explored by running a series of regressions in which each of the five RAS
factors was regressed on a set of five recovery-related measures: Empowerment Scale (Rogers et al.,
1997); Short Version Lehman’s Quality of Life Interview (Lehman, 1983); Herth Hope Index (Herth, 1991);
Life Regard Index’s Meaning of Life Subscale (Battista and Almond, 1973; Debats, 1990); and Hopkins
Symptom Checklist (Derogatis, Lipman, Rickels, Uhlenhuth, & Covi, 1974). Herth Hope Index scores
were found to positively predict scores on each of the five RAS factors; the remaining four measures each
predicted two or more RAS factors, suggesting a complex inter-relationship between the RAS factors and

Measuring the Promise: A Compendium of Recovery Measures, Volume II 62

 The Evaluation Center @ HSRI

the constructs measured by the five established instruments. The overall r for each of the five regressions
ranged from .83 for the Personal Confidence and Hope factor to .52 for the Willingness to Ask for Help fac-
tor (Corrigan et al., 2004).

Utility

Quality Improvement Uses: Program evaluation.

Intended Level of Analysis: Individual.

Current/Past Uses: The RAS was one of the outcome measures used in the COSP research initiative. This
project was a federally funded effort to examine the impact of consumer-operated services on consumer/
survivors’ outcomes, when used in conjunction with traditional mental health services. To learn more
about the research initiative, including information on sites that participated in the study, visit the COSP
Web site: http://www.cstprogram.org/consumer%20op/
H H

Summary
Strengths:
• Psychometric testing conducted.
• Suggested by consumer feedback to be a sound assessment of recovery as a process.

Weaknesses:
• Further data needed to assess diversity issues and validate a recovery model.
• Sensitivity to consumer change not established.

Permission to Use
The RAS is not copyrighted and can be used freely. There is not a user’s fee associated with the instrument.

Instrument contact:
Patrick W. Corrigan, Psy.D.
Joint Center for Psychiatric Rehabilitation
at the Illinois Institute of Technology
Email: corrigan@iit.edu
H H

Phone: 312-567-6751
Also, the measure can be downloaded (free of charge) at www.stigmaresearch.org.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 63

 The Evaluation Center @ HSRI

References and Suggested Readings

Battista, J., and Almond, R. (1973). The development of meaning in life. Psychiatry, 36(4):409-427.

Debats, D.L. (1990). The Life Regard Index: Reliability and validity. Psychological Reports, 67(1):27-34.

Derogatis, L.R., Lipman, R.S., Rickels, K., Uhlenhuth, E.H., & Covi, L. (1974). The Hopkins Symptom
Checklist (HSCL): A self-report symptom inventory. Behavioral Science, 19(1):1-15.

Corrigan P.W., Giffort D., Rashid F., Leary, M., & Okeke, I. (1999). Recovery as a psychological construct.
Community Mental Health Journal, 35(3), 231-239.

Corrigan, P.W., Salzer, M., Ralph, R., & Sangster, Y. (2004). Examining the factor structure of the Recovery
Assessment Scale. Schizophrenia Bulletin, 30(4), 1034-1041.

Corrigan, P., McCorkle, B., Schell, B., & Kidder, K. (2003). Religion and spirituality in the lives of people
with serious mental illness. Community Mental Health Journal, 39(6), 487-499.

Giffort, D., Schmook, A., Woody, C., Vollendorf, C., & Gervain, M. (1995). Construction of a scale to measure
consumer recovery. Springfield, IL: Illinois Office of Mental Health.

Herth, K. (1991). Development and refinement of an instrument to measure hope. Scholarly Inquiry for Nursing
Practice, 5(1): 36-51.

Lehman, A.F. (1983). The effects of psychiatric symptoms on quality of life assessments among the chronic
mentally ill. Evaluation and Program Planning, 6, 143-151.

Lukoff, D., Liberman, R.P., & Nuechterlein, K.H. (1986). Manual for the expanded Brief Psychiatric Rating
Scale (BPRS). Schizophrenia Bulletin, 12, 594-602.

Rogers, E.S., Chamberlin, J., Ellision, M. L., & Crean, T. (1997). A consumer-constructed scale to measure
empowerment among users of mental health services. Psychiatric Services, 48(8), 1042-1047.

Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.

Sarason, I.G., Levine, H.M., Basham, R.B., & Sarason, B.R. (1983). Assessing social support: The Social
Support Questionnaire. Journal of Personality and Social Psychology, 44, 127-139.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 64

 The Evaluation Center @ HSRI

8. RECOVERY MEASUREMENT TOOL VERSION 4.0 (RMT)

RALPH, R. O.

Information Provided by Ruth O. Ralph, Ph.D.

Introduction
Aim: The Recovery Measurement Tool (RMT) was developed to be a measure of individual recovery

Conceptual Foundation: The instrument is based upon a model of recovery developed by the Recovery
Advisory Group3. The Recovery Model offers a visual representation of the recovery process. The model is
composed of four parts (Ralph, 2004):
1. Stages of recovery: These include anguish, awakening, insight, action plan, determined commitment
to get well, and well-being/empowerment. An individual’s movement through the stages of recovery
may not be linear; individuals may move forward and backward.
2. Internal and external domains: Internal domains include cognitive, emotional, spiritual, and
physical factors; external domains include activity, self-care, social relations, and social supports.
The first and second parts of the model have been represented together by creating a grid in which
the stages and domains meet, with the stages in the recovery process positioned on one axis and the
domains positioned on the other.
3. External influences: These may be positive or negative, all having an impact on the individual’s
recovery.
4. “The Big Picture”: This provides a summary of the model and may be visually represented by a
circle, with the individual in recovery at the center, surrounded by the identified external influences
(Ralph, 2004).

Development: A group of consumers in the state of Maine used the grid of the Recovery Model to guide
the development of items for the Recovery Measurement Tool. Consumers developed at least one item to
correspond with the intersection of components in the model located on the grid. As a result, a 100 item
instrument was developed. Nine items were discarded because of their similarity to other items in the
instrument (Ralph, 2004).

Stakeholders Involved in Instrument Development: Consumer/survivors and researchers.

Back
Involvement of Consumer/Survivors in Instrument Development: As described above, consumer/survivors reviewed
the Recovery Model and developed items for each intersecting box in the Recovery Grid. Consumer/sur-
vivors also reviewed how to write a good item and the meanings of reliability and validity. In addition,
they reviewed a variety of item response sets and chose the response set based on one used by Hill House,
Cleveland consumers.

Instrument Description
Versions of the Instrument:
 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups
3
The Recovery Advisory Group consisted of the following fourteen consumer leaders in the field of mental health: Jean Campbell, Ph.D..,
Missouri; Sylvia Caras, Ph.D., California; Jeanne Dumont, Ph.D., New York; Daniel Fisher, M.D., Massachusetts; J.Rock Johnson, J.D., Nebraska:
Carrie Kaufmann, Ph.D., Pennsylvania; Kathryn Kidder, M.A., Maine; Ed Knight, Ph.D., Colorado; Ann Loder, Florida; Darby Penny, New York;
Jean Risman, Maine; Ruth Ralph, Ph.D., Maine; Wilma Townsend, Ohio; and Laura Van Tosh, Maryland.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 65

 The Evaluation Center @ HSRI

Items and Domains: The RMT is comprised of 91 items all of which are rated using a 5-point Likert scale
that indicates the degree to which the respondent identifies with an item. Response categories range from
“Not at all like me” to “Very much like me.” Respondents are also provided with the option of indicating
that an item is not applicable. Domains have not been established for the RMT yet.

RMT Sample Item

Helping others find meaning and purpose helps me feel connected and empowered.
Response options: not at all like me, not very much like me, somewhat like me, quite a bit like me, very much like me, and not
applicable

Populations: The RMT is intended for use with adults who have been diagnosed with serious mental
illness.

Settings: The RMT is intended for use with consumer/survivors who are receiving services in a peer-run
program or an outpatient setting.

Reading Level: Unknown.

Translations: None.

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet4  Observational method
 Other (specify):

Administration Time: Testing with a few consumers indicated that the administration time is about 20
minutes.

Administration Qualification/Training Requirement: Must be able to read, as well as care about other
people.

Scoring: There are no explicit guidelines indicating how to score responses.

Supporting Material Available: There are no supports for this instrument at this time. A manual will be
developed after testing has taken place.

Testing and Psychometric Properties

Although the RMT has not yet been formally tested, it should be noted that the RMT has a high degree
of face validity. As indicated earlier, the instrument was developed based on the dimensions of recovery
in the Recovery Advisory Group Recovery Model, which was developed by a group of consumer leaders
meeting monthly by teleconference over a year’s time. A grid of statements was developed to describe the
Recovery Model in all of its aspects. The RMT items were developed from this grid, to reflect all of the
dimensions of recovery.
4
Development and testing of online version included in proposal currently under review

Measuring the Promise: A Compendium of Recovery Measures, Volume II 66

 The Evaluation Center @ HSRI

Plans for further testing and evaluation: The RMT developers plan to conduct further testing to shorten
the instrument and identify domains. At the time of publication of this volume, the developers had sub-
mitted a proposal seeking funding for such testing.

Utility
Quality Improvement Uses: The instrument may be used in two ways: 1) consumer/survivors may use
the tool to identify where they are in the Recovery Model and 2) organizations interested in using the
instrument for quality improvement purposes may review responses to monitor the extent that programs
or services influence respondents’ recovery over time.

Intended Level of Analysis: Data are intended to be analyzed at multiple levels:

Program Level:
 Provider Organization
 Specific Service

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency

 Individual Level
 Other (specify):

Summary
Strengths:
• Developed by consumer/survivors.
• Based upon a model of recovery developed by the Recovery Advisory Group, which consisted of
fourteen consumer/survivor leaders in the field of mental health.
Weaknesses:
• Not formally tested.
• Members of racial/ethnic minority groups were not involved in the instrument’s development.

Back Permission to Use

The RMT is not copyrighted and can be used freely. There is not a user’s fee associated with the instru-
ment; however the author requests data from the instrument’s use.

Instrument contact:
Ruth O. Ralph, Ph.D.
Phone: (207) 934-0579
Email: ruth.ralph@maine.edu

References and Suggested Readings

Ralph, R. O. (2004). At the individual level: A personal measure of recovery. In NASMHPD/NTAC e- Report on
Recovery. Retrieved March, 2005 from http://www.nasmhpd.org/spec_e-report_fall04measures.cfm.
H H

Measuring the Promise: A Compendium of Recovery Measures, Volume II 67

 The Evaluation Center @ HSRI

9. RELATIONSHIPS AND ACTIVITIES THAT FACILITATE RECOVERY SURVEY (RAFRS)

LEAVY, R.L., MCGUIRE, A.B, RHOADES, C. & MCCOOL, R.

Information provided by Richard Leavy, Ph.D.

Introduction
Aim: The Relationships and Activities that Facilitate Recovery Survey (RAFRS) was developed to identify
the most important factors that mental health consumers feel have contributed to their recovery. It was
one component, among a set of predictors, used to predict changes in the subjective quality of life.

Conceptual Foundation: RAFRS is based on factors that research and consumers’ advice suggest are im-
portant to recovery (Leavy, McGuire, Rhoades, & McCool, 2002).

Development: Relationships and activities identified by research performed by Roth, Crane-Ross, Han-
non, and Hogan (1999) formed the core of the instrument. These variables were presented to a sample of
mental health consumers for review. They added factors to the list based on their personal experiences and
those of their peers.

Stakeholders Involved in Instrument Development: Consumer/survivors and advocates.

Back
Involvement of Consumer/Survivors in Development: Consumers’ input informed the development of the RAFRS
items; additionally, the research team included consumer members.

Involvement of Members of Racial/Ethnic Minority Groups in Development: The research team included one member
of a racial/ethnic minority group; information on additional involvement of members of racial/ethnic mi-
nority groups is not available.

Instrument Description
Versions of the Instrument:
 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups

Items and Domains: The RAFRS contains 18 items rated on a Likert scale, plus two additional open-
ended items. The open-ended items ask the participant to nominate the top two factors from the previous
18 items that facilitated their recovery in the past six months. The RAFRS’s items are organized into two
domains, Relationships and Activities, each containing nine items. Discussions with consumers informed
the development of the domains.

RAFRS Sample Item

In the last 6 months, talking with other people who have problems like mine has been helpful in my
recovery.
Response options: No contact; Yes, helped a lot; Yes, helped a little; No, didn’t help; Made things worse

Measuring the Promise: A Compendium of Recovery Measures, Volume II 68

 The Evaluation Center @ HSRI

Populations: The RAFRS is intended for use with adults who have been diagnosed with a serious mental
illness or who have a dual diagnosis from multiple ethnic/racial backgrounds. The instrument has been
administered to adults diagnosed with a serious mental illness from two ethnic/racial groups: White and
Black or African American.

Service Setting: The RAFRS is intended for use in and has been used in outpatient service programs.

Reading Level: The instrument has not been administered in a self-report format. In the study described,
the survey was read aloud to participants. However, informal feedback suggests that it is relatively easy to
read for most respondents who are comfortable reading English.

Translations: None.

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify):

Administration Time: Approximately 5 minutes.

Qualification/Training Requirement: Ability to read English, at least minimal interviewing skills.

Scoring: There are no explicit guidelines indicating how to score responses.

Testing and Psychometric Properties

Cognitive Testing: None.

Field Testing: The instrument was not field tested prior to the research study in which it was used, a
longitudinal study investigating correlates of quality of life among adults with psychiatric disabilities (see
Current/Past Uses below).

Reliability: The reliability of the measure has not been established. Internal consistency could not be
computed due to the high percentage of missing data resulting from “not applicable” responses to items
that relate to relationships and activities not universally experienced by respondents (e.g., having a sibling,
having a pet, participating in workshops).

Validity
Face Validity: The instrument was developed with significant consumer involvement, helping to assure its
face validity.

Refusal Rate: 100 percent of research participants completed the RAFRS.

Rates of Missing Data: None.

Plans for Future Testing: None at the present time.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 69

 The Evaluation Center @ HSRI

Utility

Quality Improvement Uses: The RAFRS can be used to assess the factors mental health consumers feel
are most important to their recovery.

Current/Past Uses: The RAFRS was one instrument in a battery of instruments used in a longitudinal
study designed to investigate the correlates of subjective quality of life for individuals with psychiatric dis-
abilities, using symptomatology, self-related attitudes, and social support variables. At baseline the sample
size of the study was 109 (Leavy et al., 2002). The sample was re-interviewed three times at six-month
intervals. The RAFRS item “Relationship with their Community Support Professional (CSP)” was consis-
tently noted as the most important factor facilitating their recovery. Receiving the second most mentions
was “my best friend.” Surprisingly, medication was rarely mentioned (Leavy, R. L., in press).

Summary
Strengths:
• Easy to administer.
• Short.
• Easily understood.
• Provides clear information from consumers about those factors helpful to their recovery.
Weaknesses:
• May not represent a comprehensive list of recovery-related factors.
• Includes factors that were very infrequently experienced in a population of consumers, but were
relevant to the sample at the time of the study (e.g., attending training sessions about the Recovery
Model).
• Format presents challenges to quantifying reliability and validity.

Permission To Use
The RAFRS is not copyrighted and can be used free of charge.

Instrument contact:
Richard Leavy, Ph.D.
Department of Psychology
Ohio Wesleyan University
Email: mailto: rlleavy@owu.edu

References and Suggested Readings

Leavy, R.L, McGuire, A.B., Rhoades, C., and McCool, R. (2002). Predictors of subjective quality of life in
mental health consumers: Baseline results. In D. Roth (Ed.), New Research in Mental Health, Vol. 15, (pp
246-251). Columbus, OH: Ohio Department of Mental Health.

Leavy, R. L. (in press). Predicting the subjective quality of life of mental health consumers. In D. Roth (Ed.),
New Research in Mental Health, Vol. 16. Columbus, OH: Ohio Department of Mental Health.

Roth, D., Crane-Ross, D., Hannon, M. J., & Hogan, M. F. (1999). Toward best practices: Top ten findings from the
Longitudinal Consumer Outcomes Study.[Brochure]. Columbus, OH: Ohio Department of Mental Health,
Office of Program Evaluation & Research.
Measuring the Promise: A Compendium of Recovery Measures, Volume II 70
 The Evaluation Center @ HSRI

MEASURES OF RECOVERY PROMOTING ENVIRONMENTS

1. AACP ROSE - Recovery Oriented Service Evaluation

2. Recovery Enhancing Environment Measures (REE)

3. Recovery Oriented Systems Indicators Measure (ROSI)

4. Recovery Self-Assessment

Measuring the Promise: A Compendium of Recovery Measures, Volume II 71

 The Evaluation Center @ HSRI

1. AACP ROSE- RECOVERY ORIENTED SERVICES EVALUATION

AMERICAN ASSOCIATION OF COMMUNITY PSYCHIATRISTS (AACP)
Information provided by Wesley E. Sowers, M.D.

Introduction
Aim: The AACP ROSE was designed as a self assessment tool that would allow organizations to monitor
their progress toward developing recovery enhancing services in a quantifiable manner.

Conceptual Foundation: The instrument was derived from a set of guidelines developed by the AACP de-
scribing policies and practices which promote recovery for persons with histories of mental health and/or
substance use problems. The items in AACP ROSE are derivatives of indicators of achievement that were
developed for each described element in the “Guidelines.” The theory is that agencies that score highly
will have good recovery outcomes.

Development: The instrument was developed through a consensus process within the AACP. Addition-
ally, informal feedback was requested from consumers and family members.
Stakeholders Involved in Instrument Development: Consumer/survivors, family/friends of consumer/
Back
survivor, members of racial and ethnic minority groups, providers, and administrators.

Involvement of Consumers in Development: Informal feedback was requested from consumers and family
members.

Involvement of Members of Racial/Ethnic Minority Groups in Development: Persons from diverse and ethnic back-
grounds are members of the AACP and The Quality Management Committee that was primarily respon-
sible for the instrument’s development. Likewise, stakeholders who were engaged to provide feedback
were of diverse backgrounds.

Instrument Description
Versions of the Instrument:
 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups
 Other (specify): One version of the instrument can be completed by various stakeholders: Service user;
Family member of service user; Service provider-clinician; Service provider-administrator; Stakeholder
advocate; Other

Items and Domains: The AACP ROSE is comprised of 46 items. The instrument’s items are organized
into four domains (Table 3.20), all of which are designed to be used as subscales. These domains were de-
veloped by informal consensus and no testing has been done yet to support their structure. All items are
rated using a 5-point Likert scale that ranges from 0 = Strongly Disagree to 4 = Strongly Agree.
Table 3.20
AACP ROSE Domains and Items
Domain Name Items
Administration 11
Treatment 18
Supports 11
Organizational Culture 6

Measuring the Promise: A Compendium of Recovery Measures, Volume II 72

 The Evaluation Center @ HSRI

AACP ROSE Sample Item

Promotion of recovery is included in organization’s mission and vision.

Response options: 0 = strongly disagree, 1 = mostly disagree, 2 = somewhat, 3 = mostly agree, 4 = strongly agree

Populations: The AACP ROSE is intended for use with programs/services designed for adults from di-
verse ethnic/racial backgrounds who have been diagnosed with a serious mental illness, dual diagnosis, or
substance abuse. It is important to note that the instrument is applied to services, and so is relevant for
anyone who uses them. It is not designed for specific individuals or groups of individuals but rather for
organizations that serve them.

Service Settings: The AACP ROSE is intended to be used in an array of service settings, including the
criminal justice system, inpatient setting, outpatient setting, peer-run programs, and residential programs.

Reading Level: Respondents’ informal feedback suggests that they found the reading level appropriate.

Translations: None.

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Self-administered via the internet  Observational method
 Other (specify)

Administration time: Approximately 8-10 minutes.

Scoring: There are no explicit guidelines indicating how to score responses.

Supporting Material Available: There are guidelines to interpreting data scores.

Testing and Psychometric Properties

The AACP ROSE has not yet been formally tested.

Plans for further testing and evaluation: The developers hope to obtain broader feedback from the field
prior to any formal testing.

Utility

Quality Improvement Uses: AACP ROSE was designed to function primarily as a quality improvement
tool, being used as a means for identifying opportunities for improvement.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 73

 The Evaluation Center @ HSRI

Intended Level of Analysis: Data are intended to be analyzed at the Program Level and/or System Level.

Program Level:
 Provider Organization
 Specific Service

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency
 Other (specify):

Current/Past Uses: The AACP ROSE is a recently developed instrument and, as of now, has not yet
been used. However this will soon change as the instrument has been approved for distribution and field
testing.

Summary
Strengths:
• Simple, available, and useful.
• Can be completed by various stakeholders to provide comparative ratings.
Weaknesses:
• Has not been formally tested.

Permission to Use
Back
The AACP ROSE is copyrighted by the American Association of Community Psychiatrists but can be used
freely. There is not a user’s fee associated with the instrument.
Instrument contact:
Wesley E. Sowers, M.D.
Allegheny County Office of Behavioral Health
304 Wood Street, 5th Floor
Pittsburgh, PA 15222
Phone: 412-350-3716
Email: WSowers@dhs.county.allegheny.pa.us

Measuring the Promise: A Compendium of Recovery Measures, Volume II 74

 The Evaluation Center @ HSRI

2. RECOVERY ENHANCING ENVIRONMENT MEASURE (REE) ALSO KNOWN AS THE DEVELOPING

RECOVERY ENHANCING ENVIRONMENT MEASURE (DREEM)
RIDGWAY, P.A.
Information provided by Priscilla Ridgway, Ph.D.
Introduction
Aim: The REE was developed to provide the mental health field with a multifaceted measure that collects
information on personal recovery, a set of organizational climate factors that support resilience, and a
set of programs/services that influence recovery. The REE was designed to be used in strategic planning
processes and organizational change, or systems transformation efforts. Ideally, such activities would
involve a wide variety of stakeholders, with consumer input being the central concern (Ridgway & Press,
2004).

Conceptual Foundation: Mental health recovery is a socially constructed concept that is evolving and
crystallizing through greater understanding of the lived experience of resilience and rebound among
people with serious psychiatric disabilities. The REE instrument is a consumer-driven assessment of the
service user’s own state, and his or her preferences, needs and desires, and assessments concerning the as-
sistance provided by the helping system that support and uphold recovery.

Recovery is viewed as a complex multi-stage, multi-faceted nonlinear process or journey experienced by

people with prolonged psychiatric disorders, which can be facilitated and/or impeded by the formal help-
ing system. Recovery concerns much more than clinical remission, it relates to many facets of life, thus, the
assessment of recovery and recovery markers or intermediate outcomes must therefore be holistic. While
the journey of recovery is unique for each person, general patterns can be discerned from the experience of
groups of service users.

Recovery must be consumer-driven; therefore transformation of service settings to better facilitate and
support personal recovery should focus primarily upon the voice, experiences, and preferences of service
recipients.

Development: The items in the REE were developed based upon: consumers’ first person accounts of
their recovery and the supports that assisted them in this process; an informal review of practices that are
believed to promote recovery, i.e. promising practices; and a review of literature on factors that promote
resilience or “rebound from adversity” in general. The REE measure was pre-tested and refined based on
feedback from consumers in the Kansas Consumers as Providers training program and other consumers
who were served by a Kansas Community Support Program day treatment program. Development of the
instrument also benefited from the extensive input of two established researchers (Patricia E. Deegan, a
consumer leader, and Allan Press, a statistician and measure designer). The REE then underwent two large
field tests, one in Kansas and one in Massachusetts (N=500+), and was psychometrically tested and revised
before being finalized (Ridgway & Press, 2004).

Stakeholders Involved in Instrument Development: Consumer/survivors, members of racial and ethnic

Back
minority groups, and researchers.

Involvement of Consumer/Survivors in Instrument Development: As indicated above, consumer survivors were inte-
grally involved in all stages of development. The instrument developer (Ridgway) also has personal experi-
ences of recovery.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 75

 The Evaluation Center @ HSRI

Back
Involvement of Members of Racial/Ethnic Minority Groups in Instrument Development: Members of minority groups
were included in pre-testing and pilots of the instrument, representing approximately one third of those
involved in the pilot test study groups.

Instrument Description
Versions of the Instrument:
 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups

Items and Domains: The REE has a total of 166 items, however individuals answer up to 20 fewer items
if they skip questions in the special needs section that do not apply to them. Instrument items are orga-
nized into eight domains: Demographics, Stage of Recovery, Importance Ratings on Elements of Recovery,
Program Performance Indicators, Special Needs, Organizational Climate, Recovery Markers, and Con-
sumer Feedback. The Performance Indicators measure staff behaviors or agency practice relating to the
elements of recovery (e.g., hope, positive sense of self). The REE identifies 24 such recovery elements, each
of which is associated with a subscale comprised of three program Performance Indicators.

The domains were developed through content analysis of recovery narratives, emerging promising prac-
tice, and a literature review of resilience-enhancing features of helping environments. The Stage of Recov-
ery, Recovery Elements importance ratings, 24 sets of Performance Indicators, Organizational Climate,
and Recovery Markers were treated as individual subscales and tested accordingly. The Recovery Markers
subscale is available as a free-standing instrument (the Recovery Markers Questionnaire or RMQ) and
that segment of the REE has been tested in program evaluation research and functions as a test of change
over time. The 24 sets of Recovery Performance Indicators are each treated as a subscale but the total
mean score can also serve as a subscale score to measure performance.

Table 3.21
REE Domains and Items
Domain Items
Demographics 4
Stage of Recovery 1
Importance Ratings on Elements of Recovery 24
Program Performance Indicators
24 subscales comprised of 3 items that rate program/staff performance on each 72
recovery element.
Special Needs
20
5 special needs subscales comprised of 4 items.
Organizational Climate (resilience-enhancing factors) 14
Recovery Markers (process and intermediate outcomes) 27
Consumer Feedback (open-ended questions) 4

The REE is composed of several types of questions, including close-ended questions (Demographics, Stage
of Recovery), Likert Scale items with a 5-point agreement response scale ranging from “strongly agree” to
“strongly disagree” (Elements of Recovery, Program Performance Indicators, Special Needs, Organizational
Climate, and Recovery Markers) and open-ended questions (Consumer Feedback).

Measuring the Promise: A Compendium of Recovery Measures, Volume II 76

 The Evaluation Center @ HSRI

REE Sample Item

The first sample item is one of the 24 Recovery Elements. The three items marked a, b, and c are Program
Performance Indicators.
a) Having a sense of meaning in life is important to my recovery.
b) Staff help me make sense out of what is happening in my life.
c) Staff ask me what is meaningful to me.
This program encourages me do things that give my life meaning.
The response options for all of the sample items are: strongly agree, agree, neutral, disagree, strongly disagree.

Populations: The REE is intended for use with adults from diverse ethnic/racial backgrounds who have
been diagnosed with a serious mental illness or who have a dual diagnosis. Individuals from several
ethnic/racial groups were included in the sample during testing: Black or African American (limited test-
ing), White, Hispanic or Latino (limited testing), and limited testing with members from other minority
groups. The sample of respondents had a diagnosis of serious mental illness or a dual diagnosis. Subgroup
analyses have not yet been conducted to establish whether significant differences exist across ethnic/racial
groups or across groups of individuals with a diagnosis of serious mental illness or a dual diagnosis.

Service Settings: The REE is intended for use with individuals who receive services in outpatient service
settings, peer-run programs, residential service settings, and comprehensive community support pro-
grams. Testing of the instrument included data gathered from individuals receiving services in all of the
above mentioned settings except for peer-run programs.

Reading Level: The Flesch-Kincaid Grade Level of the instrument is 7.8.

Translations: None at this time.

Adaptations: One adaptation of the instrument has been done for UK English speakers, using UK terms
(e.g. “tic this” rather than “check this”).

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify):

Administration Time: The time for an individual to self-administer the REE averages 25 minutes; in an
interview format the REE takes an average 30-45 minutes, with the longest time needed for completion of
an interview being 1.5 hours.

Qualification/Training Requirement: If conducted as an interview, interviewers should be trained to en-

sure that they are familiar with confidentiality, ethics, and how to introduce and conduct an interview.

Scoring: There are explicit guidelines indicating how to score responses and norms with which to com-
pare data [available in a user’s manual].

Measuring the Promise: A Compendium of Recovery Measures, Volume II 77

 The Evaluation Center @ HSRI

Supporting Material Available: Information on administering the instrument, guidelines to scoring re-
sponses, guidelines to interpret data scores and technical assistance are available. Consultation is available
for a fee in three areas: study planning, data analysis using scantron technology, and report preparation.
The instrument is available in two formats: a scantron format of the instrument (computer read) and
WORD format.

User Guide: Ridgway, P., & Press, A. (2004). Assessing the recovery-orientation of your mental health program:
A user’s guide for the Recovery-Enhancing Environment Scale (REE). Version 1. Lawrence, Kansas: University of
Kansas, School of Social Welfare, Office of Mental Health Training and Research.

Testing and Use / Psychometric Properties

Cognitive Testing: Pilot tests included individual and group discussions with participants about how
they perceived the instrument, confusing questions, and language they didn’t understand (e.g., an informal
“think-aloud”).

Field Testing: The psychometrics properties of the REE are based on 2 field tests. The first field test used
a mail-out scantron format survey to collect data from individuals with severe and prolonged psychiatric
disabilities served by Community Support Programs (CSP) in several areas of Kansas. Every CSS client
of the seven largest community mental health centers was surveyed by mail. A total of 381 usable surveys
were attained. The sample was predominantly white, female, long-term service users, most of whom
viewed themselves as being in the stage of active recovery.

The second study was conducted in Massachusetts and used face-to-face interviews to collect data from
individuals with psychiatric disabilities served by a large mental health agency. All clients of the residen-
tial services of a large agency were invited to participate; over 50% of program participants voluntarily
participated and received a small payment. Interviewers were agency consumers who had received train-
ing and who had completed the REE themselves. A total of 143 usable surveys were obtained. The sample
was predominantly male, white, of younger middle age, predominantly long-term service users, and most
described themselves as being in the stage of active recovery.

Full reports of each of these field tests are available from priscilla.ridgway@yale.edu.

Back Reliability
Internal Consistency: As noted earlier, the performance indicators are organized into 24 subscales, each
associated with a unique recovery element. The Cronbach’s alphas for these subscales were found to range
from .72 to .87. To further empirically validate the subscales, correlation matrix analysis was performed on
the entire set of performance indicators. The average correlation among subscale items was .61 while the
average correlation among non-subscale items was .44, supporting the subscales’ representation of the 24
identified elements of recovery. The Cronbach’s alpha for the overall set of 72 performance indicators was
found to be .94. In the remaining REE domains, the special needs subscales alphas were found to be greater
than or equal to .88 and the organizational climate subscale yielded an alpha of .97 (Ridgway & Press,
2004).

Validity: High face validity. Quantitative indicators of validity have not been assessed.

Response Rate: The response rate for the mailed REE ranged up to 30+% for some centers but averaged
13.6% for all the CSS programs combined.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 78

 The Evaluation Center @ HSRI

Refusal Rate: The response rate for the REE as an interview is over 50%.

Rates of Missing Data: There is almost no missing data when conducted as an interview.

Plans for Further Testing and Evaluation: There is a plan to gather data sets from additional sites and
run further psychometric tests. Also, planned is the development of an REE-short form (seeking a field
test site).

Utility
Quality Improvement Uses: The REE can be used in strategic planning and organizational change efforts.
Examples of ways the REE data can be used include the following:
• Educate staff and consumers about mental health recovery.
• Orient program toward recovery.
• Target specific program innovations and organizational change efforts.
• Assess the impacts that program change/interventions have on personal recovery.
• Compare the performance of agencies and programs.
• Support on-going quality improvement efforts.

Intended Level of Analysis: Data are intended to be analyzed at the Program Level and/or System Level:
Program Level:
 Provider Organization
 Specific Service

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency
 Other (please specify):

Current/Past Uses:
The REE has been used in whole or in part by a wide variety of programs and systems. The scale was
used for agency self-assessment in one Massachusetts agency, while another has adapted the assessment
to adolescent programs and used this adaptation to gather data. Part of the instrument was used to
assess a State Hospital Program in a New England state. Several state hospitals are looking at using the
instrument. An area in Illinois is currently conducting an assessment using the REE, other large agencies
in several states are looking at or are beginning to use the REE, e.g., Mental Health Corporation of Denver,
Value Options. The recovery markers section has been used to assess change over time in a supported
education intervention in Kansas. The REE has been adapted and a User’s Manual prepared for use in
the United Kingdom by Piers Allott. The instrument is to be piloted in the Midlands region of England,
with potential for broader application in the U.K. Interest has been expressed by researchers in Scotland,
Ireland, and Australia.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 79

 The Evaluation Center @ HSRI

Summary
Strengths:
• Provides opportunity for comprehensive assessment.
• Can help agencies learn more about recovery, find out where consumers are in the process of
recovery, assess what elements of recovery enhancing practices already exist and which need to be
improved or introduced.
• Data attained with the REE can be used to plan and target program transformation and to assess
program performance and the impact of change efforts over time.
Weaknesses:
• Fairly extensive.
• Requires a commitment of resources and a willingness to enter into a process of leaning and change.
• Should only be undertaken by agencies that seek to learn from service users and to transform.

Back Permission to Use

The REE is copyrighted by Priscilla A. Ridgway, 2005. Permission is required from the author prior to
using the instrument. Whether fees will be associated with the instrument’s use has yet to be determined.
Instrument Contact:
Priscilla Ridgway, Ph.D., M.S.W., Assistant Professor
Yale Program for Recovery and Community Health
319 Peck Street, Building 6W Suite 1C
New Haven, CT 06513
Phone: 203-764-8667
Email: priscilla.ridgway@yale.edu

References and Suggested Readings

Ridgway, P. (2003, May 28). The Recovery Enhancing Environment Measure (REE): Using measurement tools to
understand and shape recovery-oriented practice. Plenary paper presentation. Washington, DC: The 2003
Joint National Conference on Mental Health Block Grants and National Conference on Mental
Health Statistics.

Ridgway, P., Press, A., Anderson, D. & Deegan, P.E. (2004). Field testing the Recovery Enhancing Environment
Measure: The Massachusetts experience. Byfield, MA: Pat Deegan & Associates.

Ridgway, P.A., & Press, A.N. (2004, June 3). An instrument to assess the recovery and resiliency orientation of
community mental health programs: The Recovery Enhancing Environment Measure (REE). Conference
presentation. Washington, DC: The 2004 Joint National Conference on Mental Health Block
Grants and Mental Health Statistics.

Ridgway, P., & Press, A. (2004). Assessing the recovery-orientation of your mental health program: A user’s guide for the
Recovery-Enhancing Environment scale (REE). Version 1. Lawrence, Kansas: University of Kansas, School
of Social Welfare, Office of Mental Health Training and Research.

Ridgway, P.A., Press, A.N., Ratzlaff, S., Davidson, L. & Rapp, C.A. (2003). Report on field testing the Recovery
Enhancing Environment Measure. Lawrence, KS: University of Kansas School of Social Welfare Office of
Mental Health Research and Training.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 80

 The Evaluation Center @ HSRI

3. RECOVERY ORIENTED SYSTEMS INDICATORS MEASURE (ROSI)

DUMONT, J. M., RIDGWAY, P., ONKEN, S. J., DORNAN, D. H., & RALPH, R. O.
Information provided by Steve Onken, Ph.D., and the ROSI Research Team.

Introduction
Aim: The Recovery Oriented Systems Indicators Measure (ROSI) (Dumont, Ridgway, Onken, Dornan,
& Ralph, 2005) is designed to assess the recovery orientation of a mental health system. The recovery
orientation refers to the helping and the hindering forces within a system. It is developed from and
grounded in the lived experiences of adults with serious and prolonged psychiatric disorders. Thus,
the ROSI consumer self-report survey and administrative profile are designed to assess the recovery
orientation of community mental health systems for adults with serious and prolonged psychiatric
disorders.

The ROSI was developed from the Phase I findings5 of a three phase national research project, Mental
Health Recovery: What Helps and What Hinders? A National Research Project for the Development of Re-
covery Facilitating System Performance Indicators, conducted by a five member research team, consisting
primarily of consumers/survivors who are also researchers. The project aimed to: 1) increase knowledge
about what facilitates or hinders recovery from psychiatric disabilities; 2) devise a core set of indicators
that measure elements of a recovery-facilitating environment; and 3) integrate the items into system per-
formance evaluation and quality improvements efforts, helping to generate comparable data across sys-
tems.

Conceptual Foundation: The project is a joint effort between a variety of stakeholders, including state
mental health authorities and a consortium of sponsors. Phase I involved a national, multi-site qualitative
design guided by a grounded theory approach to identify the person-in-environment factors that help or
hinder recovery for people experiencing serious and prolonged psychiatric disorders. Specifically, though
recovery was seen as a deeply personal journey, a conceptual paradigm for organizing and interpreting
mental health recovery emerged from the many commonalities in people’s experiences. Recovery is facili-
tated or impeded through the dynamic interplay of many forces that are complex, synergistic, and linked.
Recovery is a product of dynamic interaction among characteristics of the individual (self-agency, holism,
hope, a sense of meaning and purpose), characteristics of the environment (basic material resources, social
relationships, meaningful activities, peer support, formal services, and staff), and the characteristics of the
exchange (hope, choice, empowerment, referent power, independence, interdependence). Each of these
emergent domains/themes in turn contains a rich and complex network of helping and hindering elements.

Phase II6 involved an extensive item development and refinement process that repeatedly grounded the
ongoing measure development work in the lived experiences of people with serious and prolonged psy-
chiatric disorders while maintaining a concentrated effort towards obtaining a parsimonious item set
for measurement of recovery orientation. Within this process, items focusing on external environmental
forces, particularly formal systems, were emphasized and items focusing on the internal, personal process
of recovery were deemphasized.
5
The Phase I Research Report, Mental Health Recovery: What Helps and What Hinders? A National Research Project for the Development of Recovery
Facilitating System Performance Indicators: A National Study of Consumer Perspectives on What Helps and Hinders Recovery is available in PDF format at
the following website: <http://www.nasmhpd.org/>. Click on “publications,” scroll to “National Technical Assistance Center for State Mental
Health Planning (NTAC) Publications and Reports,” scroll to “Technical Reports” and the report and appendices are under the 2002 listing.
6
The Phase II Technical Report will also be available online at the NTAC Web site.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 81

 The Evaluation Center @ HSRI

Development: In Phase I, nine State Mental Health Authorities (SMHAs) used purposive sampling to
recruit 115 consumers that participated in 10 structured focus groups. Researchers used rigorous, constant
comparative analytic methods involving qualitative coding, codebook development, cross coding, and
recoding of the focus group transcripts to develop a single set of findings. All nine SMHAs conducted
member checks with focus group participants regarding the coding report for their respective focus group.
Fifty-nine of the original focus group members (51%) participated. The research achieved a “confirmability
index” (agreement that the coding captured the original content) of 99%.

In Phase II, the Research Team used these findings to develop recovery oriented performance indicators.
Two sets emerged: 73 consumer self-report data items and 27 administrative data items. In partnership
with the participating states, the team refined the self-report set based on consumer review (a think-aloud
process), state input, and a readability check, and then conducted a prototype indicator test involving a di-
verse cross-section of 219 consumer/survivors in seven states. The Research Team then used the prototype
self-report data results to evaluate each item as to: (a) importance rating, (b) factor loading values within
a Varimax rotated component matrix, (c) response scale distribution and direction, (d) Phase I originating
theme, (e) items assessing similar content, (e) clarity of wording, and (f) Phase I member check priorities.
The Research Team also generated specific measure definitions (i.e., numerators and denominators) for the
27 administrative data items, yielding 19 administrative data indicators with 30 corresponding measures.
The 10 participating states and all state Directors of Consumer Affairs were then surveyed on the adminis-
trative data items as to (a) the feasibility of implementing each, (b) the importance of each for improving
system recovery orientation, (c) whether or not the data articulated in the definition was currently being
collected, and (d) specific comments on each.

These analyses led to further refinement, resulting in 42 self-report items being crafted into an adult con-
sumer self-report survey, 16 indicators and 23 corresponding administrative-data measures being crafted
into an authority/provider administrative-data profile.

Stakeholders Involved in Instrument Development: Consumer/survivors, family/friends of consumer/

Back
survivor (Note: through a MHSIP internet-based survey on development of second generation performance
indicators for the MHSIP Quality Report that contained the core themes and sub-themes of the Phase I
research), members of racial and ethnic minority groups, providers (Note: through the MHSIP internet
based survey), researchers, advocates, administrators, and state mental health authority partners, who also
may be providers and/or administrators.

Involvement of Consumer/Survivors in Instrument Development: Consumers/survivors were on the research team

and thus involved in designing, implementing, analyzing, interpreting, and reporting the findings. Con-
sumers/survivors were also involved in other steps of data collection, for example, as focus group co-facili-
tators. Consumers/survivors provided all of the data in Phase I, and most all of the data in Phase II. Con-
sumer participants were diverse: they varied across race, ethnicity, age, gender, income, education, housing
(including homeless participants), etc; had different usage patterns (including co-occurring substance
use problems); were from urban, rural, and suburban areas; and had varying degrees of knowledge about
recovery, consumer organizations and/or peer support.

Involvement of Members of Racial/Ethnic Minority Groups in Instrument Development: State Mental Health Authori-
ties organized the recruitment efforts for Phase I and II implementing a purposive approach to involve con-

Measuring the Promise: A Compendium of Recovery Measures, Volume II 82

 The Evaluation Center @ HSRI

sumers from diverse racial/ethnic and demographic backgrounds and a wide range of diagnoses and mental
health service and self-help experiences. State Mental Health Authorities paid particular attention to
recruiting consumers/survivors existing day to day in public mental health systems and not often involved
in advisory roles and committees.

Instrument Description
Back
Versions of the Instrument:

 One version of the instrument

 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups

Items and Domains: The ROSI Adult Consumer Self-Report Survey contains 42 items and the Adminis-
trative-Data Profile 23 items. A very preliminary factor analysis of the 42 self-report items from the Adult
Consumer Self-Report Survey resulted in domains of:
• Person-Centered Decision-Making & Choice
• Invalidated Personhood
• Self-Care & Wellness
• Basic Life Resources
• Meaningful Activities & Roles
• Peer Advocacy
• Staff Treatment Knowledge
• Access
Administrative-Data Profile:
• Peer Support
• Choice
• Staffing Ratios
• System Culture and Orientation
• Consumer Inclusion in Governance
• Coercion.

The Research Team used factor analysis to identify the domains for the Adult Consumer Self-Report Sur-
vey. The Research Team approached the factor analysis as exploratory, so that there was no attempt to
pre-specify the number of factors and their loadings. This approach was taken both because of the modest
sample size (N=219), and because the Research Team recognized that there were potential interrelation-
ships among the items that would likely differ from the conceptual scheme or set of research domains that
had been identified in Phase I. The Team relied on Principal Component Analysis as the extraction method
and used Varimax rotation with Kaiser Normalization as the rotation method.

The Research Team maintained the domain and sub-domain structure that resulted from the Phase I re-
search findings for the Administrative-Data Profile. Currently there are no psychometrically established
subscales. However, large scaling piloting and psychometric testing is planned for in Phase III, contingent
on funding.

The ROSI includes one open-ended question (last question on the Adult Consumer Self-Report Survey),
close-ended questions (some Administrative-Data Profile items), and two Likert Scales (for the Adult

Measuring the Promise: A Compendium of Recovery Measures, Volume II 83

 The Evaluation Center @ HSRI

Consumer Self-Report Survey items). Additionally, many of the Administrative-Data Profile items consist
of operationally defined numerators and denominators.

ROSI Sample Items

Adult Consumer Self-Report Survey:

I do not have enough good service options to choose from.
Response options: strongly disagree, disagree, agree, strongly agree, does not apply to me
Administrative-Data Profile:
The percent of local mental health provider agencies whose mission statements explicitly include a recovery
orientation.
Numerator: The number of local mental health provider agencies whose mission statement includes a
recovery orientation.
Denominator: The total number of local mental health provider agencies.

Populations: The ROSI prototype test (N=219) of the Adult Consumer Self-Report Survey has been ad-
ministered to adults from diverse ethnic/racial backgrounds-American Indian or Alaska Native, Asian, Na-
tive Hawaiian or Other Pacific Islander, White, Hispanic or Latino, and Black or African American- who
have been diagnosed with a serious mental illness or those with co-occurring disorders. The Administra-
tive-Data Profile has not been tested.

Settings: The ROSI is intended for use with individuals who receive services in outpatient settings, resi-
dential service programs, as well as intended for use at the mental health authority or behavioral health
care authority level. An earlier version of the ROSI has been tested with individuals receiving services in
an outpatient setting, during a prototype test.

Reading Level: The 42 items of the adult Consumer Self-Report Survey have a Flesch-Kincaid Grade Level
test mean reading score of 5.7, with a 1.0 minimum and 12.0 maximum reading level.

Translations/Adaptations: The New York State Office of Mental Health translated the Adult Consumer
Self-Report Survey into Spanish.

Practical Issues
Method of Administration:

Adult Consumer Self-Report Survey. The ROSI Consumer Self-Report Survey currently does not have sub-
scales and thus all 42 items should be administered. Please note that regardless of administering method,
surveyors should develop a definition sheet for some of the terms used in the 42 items of the ROSI
Consumer Self-Report Survey. The definition sheet needs to be tailored to the specific mental health
service component being assessed. For example, item #21 uses the term “program.” In the definition
sheet, please explain or define for the participants what is meant by “program” - a specific intervention,
(e.g., supported employment), a specific site, (e.g., Westside Wellness Clinic), or all services within a
specific organization (e.g., Buck County Mental Health Center). When administering, please point out
to the participants that some of the items are negatively worded, for example, “Staff do not understand
my experience as a person with mental health problems.” Please instruct the participants to read each
item carefully in order to answer the negatively worded items accurately. The Research Team strongly

Measuring the Promise: A Compendium of Recovery Measures, Volume II 84

 The Evaluation Center @ HSRI

recommends that someone (such as a volunteer or peer specialist) be available to respondents during
administration of the measure. This person can provide reading support and assistance, as well as answer
questions. If the administration method is by regular mail or via internet, assistance should be available
through a toll free number staffed by appropriately trained personnel, such as peer specialists.

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify): Sign or language interpreter signing or reading items and recording responses.

Administrative-Data Profile. Please note that the Commissioner or Director, Chief Information Officer or
Quality Assurance Director at the authority level can assign the appropriate person or division for the
various items. If the data are not currently collected at the provider level within an authority network, in-
dividual provider agencies must be contacted and their respective Executive Director or Quality Assurance
Director can assign responsibility.

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify):

Administration Time: This determination is planned for in Phase III, contingent on funding. Currently
some initial pilot sites seem to average about 30 minutes for the Adult Consumer Self-Report Survey when
administered by a consumer interviewer.

Qualification/Training Requirement: Such individuals should be familiar with the Mental Health Re-
covery: What Helps and What Hinders Phase I and II Reports and trained in survey administration. The
use of peer or consumer/survivors administering the Adult Consumer Self-Report Survey is encouraged.

Scoring: There are no explicit guidelines indicating how to score responses. Simple descriptive statistics
are used to score individual items. A user’s manual is planned for Phase III, contingent on funding.

Supporting Material: Technical assistance is available (on a fee basis) and there is a handout available
from the Research Team.

Testing and Psychometric Properties

Cognitive Testing: The self-report items were crafted in such a way as to maintain the wording of the
item as expressed in the Phase I focus groups, where possible. The Research Team conducted a think-
aloud to test the self-report indicator set with consumer respondents in order to further refine the items
and improve the survey. Organized by the research partners from one State Mental Health Authority
(SMHA), a Research Team member facilitated the process. The ten diverse volunteer consumers recruited
by the SMHA participated. The Project then used a design-prototype-test cycle to determine how well the
draft survey functioned in the field, and subsequently used the findings from the field test effort to further
refine the instrument. Each SMHA had a trained staff member, who was at times a consumer-interviewer,
administer the survey. The survey administrator followed a specified protocol. This allowed the person

Measuring the Promise: A Compendium of Recovery Measures, Volume II 85

 The Evaluation Center @ HSRI

administrating the prototype survey to explain the three separate steps the respondent had to take to
answer each item (i.e. respond to the item, rate the importance of the item, and circle any unclear words
or phrases). Seven SMHAs were able to test the ROSI self-report prototype for a total of 219 completed
surveys.

The Research Team designed a survey to gain feedback on the original 19 indicators and 30 corresponding
operational definitions for the Administrative-Data Profile. The survey solicited feedback as to: (a) the
feasibility of implementing each operationalized measure (e.g., rating each as very feasible, fairly feasible,
limited feasibility, not at all feasible); (b) the importance of each measure for improving the system’s re-
covery orientation (rated as very important, fairly important, limited importance, not at all important);
(c) whether or not the data articulated in the measure were currently being collected (rated as yes or no);
and, (d) other specific comments on each measure. The Research Team then surveyed the 10 participating
SMHAs. The Research Team also surveyed the nine members of the MHSIP Consumer Expert Panel and
the members of the National Association of Consumer/Survivor Mental Health Administrators (NAC/
SMHA). The Team took this step in order to continue the process of grounding the work in the lived expe-
riences of consumers/survivors. The team selected these two consumer/survivor groups because they had
working knowledge of performance indicators and outcome measurement.
Field Testing: Field testing is planned for in Phase III, contingent on funding.

Back Reliability
Internal Consistency : This determination is planned for in Phase III, contingent on funding. During the pro-
totype test, a reliability coefficient was computed for the reduced set of 42 items on the Adult Consumer
Self-Report Survey, resulting in a Cronbach’s alpha of .95. Only 48 surveys could be included as the re-
maining surveys had one or more item responses missing or marked as “Does Not Apply to Me.”

Test-Retest Reliability: This determination is planned for in Phase III, contingent on funding. This involves
computing intra-class correlation coefficients to move toward establishing both measures’ test-retest reli-
ability, or the extent to which repeated measurements that are made under constant conditions provide
the same result. An attempt to account for intervening variables that may have occurred between the
first and second time will likely require the collection and analysis of additional data such as event data,
psychiatric/service system data, and other life and organizational events data that define what may have
transpired between the two points in time.

Validity
Face Validity: As documented earlier, the ROSI measures were crafted from the qualitative data provided
by consumers/survivors in a multi-site national research study. As the measure items were developed, re-
peated scientifically sound efforts regrounded the measure development process in the lived experiences of
consumers/survivors including maintaining original consumer/survivor wording where possible, a think-
aloud session, a prototype test, and feedback survey.

Additionally, the Research Team proposes conducting factor analysis in Phase III to assess the factorial
structure of the theoretical construct. Factor analyses of the prototype test data suggest a multi-factorial
structure for the ROSI Adult Consumer Self-Report Survey, which will be examined further with a larger
pool of pilot test data.

Other quantitative measures of validity are planned for in Phase III, contingent on funding.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 86

 The Evaluation Center @ HSRI

Further Testing and Evaluation:

The Research Team is considering a number of tasks for Phase III, contingent upon funding. These tasks
include: 1) exploring potential means to establish structural and/or factorial relationships between the
ROSI Adult Consumer Self-Report Survey and the Administrative-Data Profile; 2) using Item Response
Theory (IRT) in the testing of the ROSI Adult Consumer Self-Report Survey where, for example, test
properties are ‘sample free’ compared with Classical Test theory where test properties are ‘sample
dependent;’ 3) exploring the development of benchmark performance standards, using obtained mean
scores and other scoring schemes; 4) working to create a psychometrically sound ROSI-SF (Short Form)
Self-Report Survey and further streamline the administrative profile; and 5) examining the relationship
between level of personal recovery and the extent to which recovery orientation has been promoted or
hindered by the system. This would entail adding a means to measure individual recovery to accompany the
ROSI pilot in some sites.

Utility
Quality Improvement Uses:
The ROSI measures should be considered among an emerging set of “tools for transformation” that are
available to policy-makers, administrators, planners, evaluators, consumers, family organizations, advocates,
and others involved in efforts to fundamentally transform state, county, and local mental health systems.
The set of ROSI measures are an important resource for systems as they plan for change, strategically and
intensively target their efforts and resources, and seek to understand the impact of their work as they move
forward in efforts to shift mental health programs and systems to a recovery orientation. The ROSI, either
used alone, or along with other recovery assessment tools, will allow and support systematic analyses
and evaluation of change efforts. Some of the ways the ROSI measures can be used as tools in systems
transformation include:
1. To create a “baseline” dataset to assess the current status of the recovery orientation of a program or
local system. A local planning group could use such data to help them formulate a strategic plan to
guide their systems change efforts.
2. To set specific benchmarks that target desired increments of progress toward achieving a recovery
orientation. ROSI data could be gathered at several points in time to inform continuous quality
improvement efforts. The ROSI can provide managers with a means to guide or gauge efforts at
improving their agency or system. Specific indicators could be targeted for improvement or general
trends could be tracked to assess the achievement of increases in a recovery-orientation over the
course of time. The ROSI gives a system a means to track increases in performance indicators
associated with processes that facilitate recovery and to track reductions in indicators that
consumers report hinder the potential for personal recovery.
3. To measure general change over time in the recovery-orientation of the program or system. This
effort would involve creating a plan to sample consumers at specified intervals and follow-up to
identify trends in the data. Research using the ROSI could also help measure the impact of specific
targeted program or systems change efforts. Using the ROSI to gather follow-up data after new
programming is implemented and comparing the ratings to baseline data could inform program
evaluation efforts.
4. The performance of provider agencies can be compared by gathering uniform data on the ROSI
across a local, regional, or county system. Data from all agencies operating in a local system can be
gathered, aggregated and compared to assess the relative performance of local, county or regional
mental health systems operating across a given state.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 87

 The Evaluation Center @ HSRI

5. ROSI data can be used as part of an ongoing process of sensitizing and educating mental health
providers about important elements that facilitate or impede mental health recovery.
6. The ROSI can be used as part of other targeted studies of mental health recovery to develop a better
understanding of how agency-level or systems-level performance on key indicators relate to other
recovery elements, processes, or outcomes.

The research team cautions the use of the ROSI prior to pilot testing. The psychometric properties have
not yet been determined. The measures may be altered somewhat before they achieve final form. There is
no user’s guide or analysis package to guide the use of the ROSI and the analysis of data. The development
of a ROSI User’s Manual will be a major contribution of the proposed Phase III. For a person or agency,
however, that chooses to move forward on using the ROSI measures in the near future the Research Team
would ask:
1. That the Research Team be informed of anyone using the ROSI measures. This notification can be
done by contacting the Research Team through either of the Co-Principal Investigators, Steven
Onken or Jeanne Dumont.
2. That anyone using the ROSI ought to strive to use the Consumer Self-Report Survey and the
Administrative-Data Profile jointly. The Consumer Self-Report Survey is complemented by the
Administrative-Data Profile. Data that are generated by doing the Consumer Self-Report Survey
alone are incomplete. The Administrative-Data Profile gathers data on important indicators of the
recovery orientation of a system that are not covered on the consumer survey.
3. That anyone using the ROSI measures agree to use the measures as currently formatted, and not shift
the items around, change the wording of any of the items, or shorten the measures by only gathering
data on a subset of items.
4. That anyone using the Consumer Self-Report Survey and/or Administrative-Data Profile to maintain
their dataset agree to consider a request to share their data with the Research Team, once a proposed
Phase III pilot test of the ROSI is underway. The request will be subject to approval by the local
site’s research review, confidentiality, and IRB processes as necessary. The local site would continue
to ‘own’ the data that have been collected, but would share the dataset in aggregate form with the
Research Team.
5. That anyone using the ROSI also agree to gather a small set of additional data that includes self-
report survey respondent demographic variables, agency/authority-level descriptors, and methods of
data collection. These forms are included with the measures.

Note: The ROSI measures may become incorporated into the data standards and technology platform of
Decision Support 2000+ (DS2000+). The DS2000+ Initiative has developed standards for collecting and
reporting population, enrollment, encounter, financial, organizational, and human resource data as well
as for system performance and consumer outcomes measurement. It has also developed an online informa-
tion system (www.ds2kplus.org) that provides tools for a wide range of users to: conduct surveys; collect,
store, analyze, and benchmark data; and share information across the field.

Intended Level of Analysis: The intended level of analysis of ROSI data is at the system level:

Measuring the Promise: A Compendium of Recovery Measures, Volume II 88

 The Evaluation Center @ HSRI

Program Level:
 Provider organization
 Specific service

System Level:
 State mental health system
 Local mental health system
 Behavioral health care organization
 Multi-service agency (Note: May require some adaptation of the Administrative-Data Profile measure)
 Other (specify):

Current/Past Uses: Intended uses covered in Quality Improvement Uses section. Instrument is now
moving into pilot test phase.

Summary
Strengths:
• Constructed from the lived experiences of consumers/survivors through a scientifically rigorous
qualitative and quantitative multi-phase, multi-site process with continuous re-grounding in such
lived experiences.
• Development implemented by a team primarily consisting of researchers with such lived
experienced.
• Development process also informed through structured review of various stakeholder groups,
particularly state mental health authorities.
Weaknesses:
• Considerations to reduce response burden and achieve as short a set of items as possible in both
the Adult Consumer Self-Report Survey and the Administrative-Data Profile may have limited
the depth of the assessment of the recovery orientation of the service systems and full coverage of
complexity of the recovery construct.

Permission to Use
Back
The ROSI will be in the public domain. Permission is recommended but not required for use of the
instrument. Fees associated with the instrument will include any needed or requested technical assistance
or training.

Instrument contact:
Steven J. Onken, Ph.D.
Columbia University School of Social Work
Email:so280@columbia.edu
Phone: (212) 851-2243

or

Jeanne M. Dumont, Ph.D.

Consultant in Mental Health Services Research
E-mail: jdumont@lightlink.com
Phone (607) 273-8021

Measuring the Promise: A Compendium of Recovery Measures, Volume II 89

 The Evaluation Center @ HSRI

References and Suggested Readings

Dumont, J. M., Ridgway, P., Onken, S. J., Dornan, D. H., & Ralph, R. O. (2005). Mental health recovery:
What helps and what hinders? A national research project for the development of recovery
facilitating system performance indicators. Phase II technical report: Development of the recovery
oriented system indicators (ROSI) measures to advance mental health system transformation.
Alexandria, VA: National Technical Assistance Center for State Mental Health Planning. Soon
available online through the NTAC Web site: http://www.nasmhpd.org/ntac.cfm

Onken, S. J., Dumont, J. M., Ridgway, P., Dornan, D. H., & Ralph, R. O. (2002, October). Mental health
recovery: What helps and what hinders? A national research project for the development of recovery
facilitating system performance indicators. Phase one research report: A national study of consumer
perspectives on what helps and hinders mental health recovery. Alexandria, VA: National Technical
Assistance Center for State Mental Health Planning.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 90

 The Evaluation Center @ HSRI

4. RECOVERY SELF-ASSESSMENT (RSA)

O’CONNELL, M., TONDORA, J., CROOG, G., EVANS, A., & DAVIDSON, L.,
Information provided by Maria O’Connell, Ph.D.

Introduction
Aim: In conjunction with the Connecticut Department of Mental Health and Addiction Services (DMHAS)
recovery initiative, the Recovery Self-Assessment (RSA) (O’Connell, Tondora, Croog, Evans, & Davidson,
2005) was developed by the authors to assess the degree to which recovery-supporting practices are evident
in the Connecticut Department of Mental Health and Addiction Services agencies. The authors wanted to
move beyond the rhetoric of recovery by operationalizing principles of recovery into standards and practices
that could be observed, measured, and fed back to mental health organizations in ways that allowed these
organizations to use data to inform program improvement and organizational change efforts.

Conceptual Foundation: The RSA contains 36 items associated with nine principles of recovery identified
through extensive literature review and input from persons in recovery, family members, service providers,
and administrators. These principles are: renewing hope and commitment; redefining self; incorporating
illness; being involved in meaningful activities; overcoming stigma; assuming control; becoming empowered
and exercising citizenship; managing symptoms; and being supported by others (O’Connell et al. 2005).

Development: Davidson, O’Connell, Sells, and Staeheli (2003) conducted an extensive literature review of
recovery from mental illness and addictions to identify common principles of recovery and recovery-oriented
practices. As noted above, based on the literature, 9 principles of recovery were identified and used to gen-
erate the initial 80-items (Davidson et al.). Experts in clinical and community psychology, consumers and
direct service providers of mental health and addiction services, and family members provided feedback and
suggestions for the revision and/or addition of new items. The items were then edited, balanced with regard
to conceptual domain, and selectively eliminated to generate the current, 36-item version of the RSA. The
RSA was adapted for completion by CEO/agency directors, providers, persons in recovery, and family/sig-
nificant others/advocates. Principle components factor analysis revealed five primary factors, all with good
to excellent levels of internal consistency: Life Goals, Involvement, Diversity of Treatment Options, Choice,
Individually Tailored Services.

Stakeholders Involved in Instrument Development: Consumers/survivors, family/friends of consumer/sur-

Back
vivor, members of racial and ethnic minority groups, providers, researchers, advocates, and administrators.

Involvement of Consumer/Survivors in Development: Consumers produced some of the literature reviewed to iden-
tify principles of recovery and recovery-oriented practices (Davidson et al. 2003; O’Connell et al. 2005). As
noted above, consumers were also involved in the expert review of the original pool of 80 items which led to
the development of the current version of 36-items. Additionally, consumers were engaged in a series of fo-
cus groups and discussion sessions pertaining to the development of a model and definition of recovery. Data
gleaned from these focus groups also informed the development of the items contained in the RSA.

Involvement of Members of Racial/Ethnic Minority Groups in Development: The members of the focus groups and the
item-review team were ethnically and racially diverse. In Connecticut, the minority groups most represented
were people of African and/or Hispanic origin.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 91

 The Evaluation Center @ HSRI

Instrument Description
Versions of the Instrument:
 One version of the instrument
 Baseline/follow-up versions of the instrument
 Versions for different stakeholders groups
1. Persons in Recovery Version
2. Family/Significant Others/Advocates Version
3. Provider Version
4. CEO/Agency Director Version

Items and Domains: The RSA contains 36 items. As shown in Table 3.22, the instruments are constituted
of five subscales that measure the domains: Life Goals, Involvement, Diversity of Treatment Options,
Choice, and Individually-Tailored Services. Domains were developed by Principal Components Factor
Analysis with Varimax rotation. All items are rated using the same 5-point Likert scale that ranges from
strongly disagree to strongly agree. Also included is the option of marking “not applicable” for any given
item.
Table 3.22
RSA Domains and Items
Domain Name Items
Life Goals 11
Involvement 8
Diversity of Treatment Options 6
Choice 6
Individually-Tailored Services 5

RSA Sample Items

Person in Recovery Version:

Staff focus on helping me to build connections in my neighborhood and community.
Response options: 1 = Strongly Disagree - 5= Strongly Agree or N/A

CEO/Directors Version and Provider Version:

Helping people build connections with their neighborhoods and communities is one of the primary
activities in which staff at this agency are involved.
Response options: 1 = Strongly Disagree - 5= Strongly Agree or N/A

Family/Significant Other/Advocate Version:

Staff focus on helping people in recovery to build connections in their neighborhood and community.
Response options: 1 = Strongly Disagree - 5= Strongly Agree or N/A

Populations: The RSA is intended for use with programs/services for adults who have been diagnosed
with a serious mental illness, dual diagnosis, or substance abuse. The RSA has been tested with individu-
als diagnosed with serious mental illness, dual diagnosis, or substance abuse from various ethnic/racial
populations: Black or African Americans, White, and Hispanic or Latino. During testing, respondents also

Measuring the Promise: A Compendium of Recovery Measures, Volume II 92

 The Evaluation Center @ HSRI

included mental health and addiction service providers, family members or significant others, and admin-
istrators/directors of state-funded mental health and addiction services. Subgroup analyses by ethnicity/
race or by diagnosis have not been conducted to establish whether differences exist across groups.

Service Setting: The RSA is intended for use with individuals who receive and/or provide services in inpa-
tient settings, outpatient settings, peer-run programs, residential programs, and social programs. The RSA
has been tested in the previously mentioned settings and is designed to assess recovery-oriented practices
regardless of setting. There are some settings where some of the items may be more or less applicable (e.g.,
“most services take place in the community” may not be applicable to an inpatient or criminal justice set-
ting); however, it is argued that the RSA items may reflect more “ideal” recovery-supporting practices that
could be applied in any setting.

Reading Level: Respondents’ informal feedback suggests that they found the reading level appropriate.

Translations: None

Practical Issues
Method of Administration:

 Self-administered  Individual interview

 Self-administered in a group  Group interview
 Mail administration  Phone interview
 Completed via the internet  Observational method
 Other (specify):_______________

Administration Time: Less than 10 minutes.

Qualification/Training Requirement: None, the instrument is self-administered.

Scoring: There are explicit guidelines indicating how to score responses.

Supporting Material Available: Information on administering the instrument, interviewer/administration

training, guidelines to scoring responses, guidelines to interpreting data scores, technical assistance.

Testing and Psychometric Properties

Cognitive Testing: None.

Field Testing: An initial pilot of the survey was conducted in 2002 with 148 individuals at 10 mental
health and addiction agencies receiving funding from the Connecticut Department of Mental Health and
Addiction Services. Revisions were made following the initial pilot.

A second study was conducted with all state funded agencies providing mental health services (N=208).
Each agency was sent 16 copies of the survey (one Agency Director version, five Provider versions, five Per-
sons in Recovery versions, and five Family Member/Significant Other/Advocate versions). A total of 3,328
surveys were mailed to agency directors across the state. Completed surveys were received from 974 indi-
viduals in 82 facilities. Included in the analysis were 967 surveys of which 68 were from the CEO/Agency
Director Version, 344 from the Provider Version, 326 from the Person in Recovery Version, and 229 from
the Family/Significant Others/Advocate Version (O’Connell, et al., 2005).

Measuring the Promise: A Compendium of Recovery Measures, Volume II 93

 The Evaluation Center @ HSRI

Back
Reliability
Internal Consistency: Cronbach’s alpha for each of the five domains is shown in Table 3.23.

Table 3.23
RSA Subscale Internal Consistency
Domain alpha
Life Goals .90
Involvement .87
Diversity of Treatment Options .83
Choice .76
Individually-Tailored Services .76

Interrater Reliability: Interrater reliability per se has not been examined. Each agency is rated from the per-
spective of the person in recovery, provider, family member/significant other, and director. The between-
group consistency varies as expected: people have different views of the degree to which agencies provide
recovery-oriented services.

Validity
Face Validity: The face validity of the instrument is supported. Items were derived from extensive litera-
ture reviews and discussions with persons in recovery, mental health and addiction service providers, fam-
ily members, and administrators.

Quantitative indicators of validity are pending.

Response Rate: As noted above, 16 copies of the RSA were mailed to each of 208 agencies. The individual
response rate was 29%, or 974 surveys returned out of 3328 mailed. These returned surveys came from 82
of the 208 agencies, for an agency response rate of 39%.

Rates of Missing Data: Data were generally complete. However, four agencies submitted instruments that
contained more than 30% of N/A responses. These agencies were not included in the analysis.

Plans for Further Testing and Evaluation: The Person-In-Recovery Version of the RSA is being rede-
signed based on consumer feedback. The Person-In-Recovery Version is intended for consumers to rate
the practices of the providers at their agency—many of which consumers may not be privy to. A revised
version will include items that are more specifically related to recovery-oriented practices that individuals
have experienced at the agency, rather than individuals’ perceptions of agency practices. A few additional
items have been added to all versions of the RSA to better reflect practices associated with Recovery-Ori-
ented Standards developed by the authors. Studies examining the construct and criterion-related validity
of the RSA are planned in several states.

Utility

Quality Improvement Uses: The RSA operationalizes the principles of recovery, identifying objective
practices that can be assessed from multiple perspectives. The RSA illustrates how research can be trans-
lated into everyday practice through the use of self-assessment and structured feedback. Data from the
RSA can be utilized to create a “Recovery Profile” for an agency that would help agency personnel and
stakeholders review their relative standing in comparison to other agencies, their relative strengths, and
areas of improvement.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 94

 The Evaluation Center @ HSRI

Intended Level of Analysis: Data are intended to be analyzed at the Program Level and/or System Level:

Program Level:
 Provider Organization
 Specific Service

System Level:
 State Mental Health System
 Local Mental Health System
 Behavioral Health Care Organization
 Multi-Service Agency

 Other (please specify):

Current/Past Uses: The RSA has been used to conduct a statewide assessment of recovery-oriented
practices in Connecticut. It has also been used in several organizations nationally as a self-evaluative tool.
These organizations include mental health centers, Veterans Administration hospitals, social-rehabilita-
tion programs, and other state mental health evaluations. The RSA has recently been selected for use by
the V.A. as a measure of system recovery-oriented practices.
Summary

Strengths:
• Strong link to theory.
• Participatory process of development—consumers, family members, administrators, and service
providers.
• Measures perceptions of provider practices thought to be indicative of a recovery-oriented or
recovery-supportive environment.
• Strong face validity.
• Excellent internal consistency on factors.
• Self-administered and brief: the 36-item scale takes less than 10 minutes to administer.
• Easy to score.
• Provides immediate feedback to agencies about ways in which they may be able to change or
enhance their practices to better provide a recovery-oriented environment.
Weaknesses:
• May be more prone to socially desirable responses (strong face validity)—anonymous administration
is strongly recommended.
• Complete response in part dependent upon consumers’ knowledge of the degree to which a
particular agency endorses or engages in a particular practice (note Plans for further testing above).
• Does not attempt to assess the degree of recovery of the individual. There is no claim and research
has yet to show that the RSA reflects the importance of the practices to the individual or the degree
of recovery of the individual.
Conclusion:
In summary, agencies have been charged with the challenge of developing a more recovery-oriented system
of care, only to be faced with the questions: 1) what exactly does that mean? and 2) what would that look
like? Based on extensive and informed publications, focus groups, and additional stakeholder feedback, the
RSA offers a measure of clearly-defined practices that are thought by stakeholders to be reflective of a more
recovery-oriented or recovery-supportive environment.
Measuring the Promise: A Compendium of Recovery Measures, Volume II 95
 The Evaluation Center @ HSRI

Back Permission to Use

The RSA is not copyrighted. Permission is recommended but not required for use of the instrument. There
is not a user’s fee associated with the instrument.

Information contact:
Maria O’Connell, Ph.D.
Yale University School of Medicine
319 Peck Street, Building 6, Suite 1C
New Haven, CT 06519
Phone: (203) 764-7593
Email: maria.oconnell@yale.edu

References and Suggested Readings

Connecticut Department of Mental Health and Addiction Services (undated). Recovery self-assessment.
Executive Summary. New Haven, CT.

Davidson, L., O’Connell, M., Sells, D., & Staeheli, M. (2003). Is there an outside to mental illness? In L.
Davidson, Living outside mental illness. Qualitative studies of recovery in schizophrenia.(pp. 31-60).New York:
New York University Press.

O’Connell, M., Tondora, J., Croog, G., Evans, A., & Davidson, L. (2005). From rhetoric to routine: Assessing
perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatric
Rehabilitation Journal, 28 (4), 378-386.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 96

 The Evaluation Center @ HSRI

CHAPTER 4. CONCLUDING REMARKS

Recovery has been well established as an issue of critical importance to people with psychiatric disabilities
and to those who provide, research, and fund mental health services. The narrative knowledge base of con-
sumer experience of recovery has now been developing for decades and is in evidence in both the consumer
and the traditional professional literature. Such documentation should remain central to our understand-
ing of the phenomenon of recovery, but the recent system-wide drive for recovery-oriented services calls as
well for the development of instruments that can be used to support and measure this anticipated trans-
formation (United States DHHS, 2005).

The instruments in this volume clearly represent a major advancement in the process of this development
in terms of our ability to measure both recovery in individuals and recovery orientation in systems. The
instruments’ individual qualities have been detailed in the preceding pages, but it is important to note that
they also offer a number of important strengths as a group.
Strengths of the instruments:
• The instruments are based largely on consumers’ experiences and consumer expertise.
Development of many of the instruments was grounded in consumer focus groups, interviews,
reviews of consumer narratives and other consumer-based literature; refinement often involved
solicitation of consumer feedback. Perhaps most importantly, many of the instruments’ authors
were self-identified consumer/survivor researchers.

• Measures range widely in terms of length, administration method, domains covered and
respondent versions offered. Such a range improves the chances that researchers will be able to
select an instrument that meets their needs.

• A number of the instruments have undergone considerable psychometric testing and

use. Researchers who require a more established measure may well find one that meets their
requirements, despite the early stage of development of the field.

• Many of the instruments are available free of charge, and those that are not generally offer a
range of materials and assistance with their usage fee. The instruments are therefore relatively
accessible to researchers with limited resources.

• Many of the instruments’ authors are choosing to remain involved in or apprised of other
researchers’ use of their instrument. Many authors requested to be notified of the use of their
instrument; a number request access to data collected with their instrument. Such involvement may
help to retain instrument integrity and to coordinate further instrument testing and development.

A review of the instruments also suggests a few critical next steps in the development of recovery instru-
ments and the recovery knowledge base as a whole. The following three points are not intended as a com-
prehensive list, but rather are offered as a starting point for individual thought and community dialogue
about future recovery research endeavors.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 97

 The Evaluation Center @ HSRI

Recommendations for future instrument development:

• The primacy of consumers’ roles in the development of the recovery knowledge base should
be guarded by all stakeholders. As Judi Chamberlin suggested in her reflections in Chapter
2, the invitational conference and, by extension, this volume, are steps towards establishing a
recovery-oriented knowledge base that will be widely recognized and accepted by traditional
mental health services researchers. As her reflections also suggest, the potential benefits of
such developments are considerable, particularly given the system’s current movement towards
evidence-based practices. A risk inherent in this process is the dilution of the original vision
and a weakening of consumer ownership of the concept and its development. The presence of a
number of established and acclaimed consumer researchers helps to offset this risk; also it would
be helpful to have a widespread commitment among non-consumer researchers to collaborate with
consumer-researchers whenever possible and among all researchers to combine “power-sharing”
research methodologies (such as qualitative or participatory approaches) with more traditional
ones. Ultimately all researchers, whether consumers or not, have a responsibility to ensure that the
development of the recovery knowledge base remains consumer-driven.

• Continued instrument development and use should both draw upon and reflect the
experiences of diverse populations. Some of the instruments reviewed in Volume II were created
by or with the input of people from ethnic or racial minority groups, much of the instrument testing
drew from diverse participant pools, and a number of the scales contain items relating specifically
to organizational cultural competency or to the experiences of respondents who are from racial
or ethnic minority groups. These are all strengths in the pool of available recovery measures but
should not be taken as indicators that cultural competence has been achieved. Rather, they should
be considered important initial steps towards building a knowledge base that reflects a diverse and
vibrant vision of recovery.

• Non-measures based criteria for the establishment of instrument validity should be developed
and implemented. Some of the instruments reviewed in this volume had not yet been developed
to the point of validity testing. Most of those that had were tested against other developing
measures of recovery or against measures thought to represent aspects of recovery (e.g., quality of
life, symptom distress). One of the next logical steps would be to test the instruments against other
“real world” criteria. This task will be a difficult one as it involves revisiting the process of defining
recovery and the even more problematic issue of defining who is recovered or in recovery. For
individual measures, one promising possibility is the use of a self-rating item - similar to the Stage
of Recovery item in the Recovery Enhancing Environment Measure (REE) - to allow respondents
to identify their level of recovery; the results for the overall measure or its domains could then
be validated using self-rating item responses. Clearly this is just one possible solution. Indeed,
the development of the recovery knowledge base will be best served by a variety of innovative,
consumer-defined solutions to validity testing.

As indicated earlier, this volume is not intended to be a static document. Rather, it will be made available
in an updatable format at the Evaluation Center Web site: http://tecathsri.org. We hope to use this format
H H

to document both instrument development and advancements of the recovery orientation in mental health
services research. Your comments and recommendations will better enable us to do that, and we welcome
them at contacttec@hsri.org.
H H

Measuring the Promise: A Compendium of Recovery Measures, Volume II 98

 The Evaluation Center @ HSRI

References:

New Freedom Commission on Mental Health, Achieving the Promise: Transforming Mental Health Care in America.
Final report. (DHHS Pub. No. SMA-03-3832). Rockville, MD: 2003.

United States Department of Health and Human Services, Substance Abuse and Mental Health Services
Administration. Transforming Mental Health Care in America, Federal Action Agenda: First Steps. . Retrieved
September 200, from the SAMHSA Web site: http://www.samhsa.gov/Federalactionagenda/NFC_
TOC.aspx.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 99

 The Evaluation Center @ HSRI

APPENDIXES

Appendix A: Participant list for Measuring the Promise: Assessing Recovery & Self- Determination
Instruments for Evidence-Based Practices

Appendix B: Instrument Description Form

Appendix C: Notes for Non-Researchers

Appendix D: Instruments and Materials

Measuring the Promise: A Compendium of Recovery Measures, Volume II 100

 The Evaluation Center @ HSRI

Appendix A: Participant list for Measuring the Promise: Assessing Recovery & Self-Determination
Instruments for Evidence-Based Practices

William. A. Anthony, Ph.D. Marsha Langer Ellison, Ph.D., M.S.W. Maria O’Connell, Ph.D.
Executive Director and Professor Senior Research Associate/Research Co-Director of Research and Evaluation
Boston University Center for Psychiatric Assistant Professor Yale Program for Recovery & Community
Rehabilitation Sargent College of Health and Health
Rehabilitation Sciences Boston University
Crystal R. Blyler, Ph.D. Steven J. Onken, Ph.D.
Social Science Analyst Daniel B. Fisher, M.D., Ph.D. Assistant Professor
SAMHSA Center for Mental Health Co-Director Columbia University School of Social Work
Services National Empowerment Center
Ruth O. Ralph, Ph.D.
Sarah R. Callahan, MHSA Vijay Ganju, Ph.D. Senior Research Associate (retired)
Deputy Director Director Edmund S. Muskie School of Public Service
NASMHPD National Technical Assistance Center for Mental Health Quality and
Center Accountability Priscilla Ridgway, Ph.D., M.S.W
NASMHPD Research Institute, Inc. University of Kansas School of Social
Teresita Camacho-Gonsalves, Ph.D. Welfare
Assistant Director & Project Manager Erin Garrett
The Evaluation Center@HSRI Research Assistant E. Sally Rogers, Sc.D.
The Evaluation Center@HSRI Director of Research
Jean Campbell, Ph.D. Boston University Center for Psychiatric
Research Associate Professor Kevin Ann Huckshorn RN, MSN, CAP Rehabilitation
Missouri Institute of Mental Health Director
NASMHPD National Technical Assistance Michelle P. Salyers, Ph.D.
Theodora Campbell-Orde, M.P.A. Center Co-Director, ACT Center of Indiana
Research Associate Assistant Scientist, Psychology Department
The Evaluation Center@HSRI Stephen Leff, Ph.D. Indiana University Purdue University at
Director & Principal Investigator Indianapolis
Judi Chamberlin The Evaluation Center@HSRI
Director of Education and Training Julie Silver
National Empowerment Center Ted Lutterman Project Director
Director of Data Analysis Human Services Research Institute
Larry Davidson, Ph.D. NASMHPD Research Institute
Director Stephen Strempek
Yale Program for Recovery & Community Wilma J. Lutz, RN, PhD Research Assistant Intern
Health Research Administrator The Evaluation Center@HSRI
Ohio Department of Mental Health
Jonathan Delman, JD, MPH Laura Van Tosh
Executive Director Anthony Mancini, Ph.D. Consultant
Consumer Quality Initiatives, Inc. Research Scientist
NYS Office of Mental Health
Douglas Dornan, M.S.
Research Scientist Lynne Mock, Ph.D.
NYS Office of Mental Health Assistant Professor
University of Illinois at Chicago
Jeanne Dumont, Ph.D. Center on Mental Health Services Research
Well Being Programs, Inc. & Policy

Measuring the Promise: A Compendium of Recovery Measures, Volume II 101

 The Evaluation Center @ HSRI

Appendix B: Instrument Description Form

Instrument Name:

Citation for the Instrument:

Introduction
1. Please briefly describe the theory behind your instrument and how it relates to the concept
of recovery.

2. For what purpose and use was the instrument developed?

3. Briefly describe the instrument’s development. (What method(s) was used to develop the
instrument?)

4. What stakeholders were involved in the instrument’s development? (Check all that apply)
___Consumer/survivors
___Family/friends of consumer/survivor
___Members of racial and ethnic minority groups
___Providers
___Researchers
___Advocates
___Administrators
___Other (specify):__________

4.1. Briefly describe the involvement of consumer/survivors in the instrument’s

development?

4.2. Briefly describe the involvement of members of racial and ethnic minority groups in
the instrument’s development?

Measuring the Promise: A Compendium of Recovery Measures, Volume II 102

 The Evaluation Center @ HSRI

I. Instrument Details
1. What is the total number of items in the instrument?_______
2. Domains
2.1. If your instrument is divided into domains, please list the domains and the number of
items in each domain (also, please attach latest version of instrument).

2.2. How were these domains developed (e.g., content analysis, factor analysis, concept
mapping)?

2.3. Which domains can be used as subscales?

3. Question types included in the instrument (check all that apply):

___Open-ended questions
___Close-ended questions
___Likert Scale
___Other:_____________

4. Reading level of the instrument:

___Reading level of the instrument is unknown.
___Informal feedback suggests the reading level is appropriate for respondents.
___Formal testing indicates that the reading level is appropriate for respondents. ___Specify
level:_______
___Not Applicable. Instrument is not administered in self-report format.

5. Scoring instrument responses:

___There are no explicit guidelines indicating how to score responses
___There are explicit guidelines indicating how to score responses
___There are explicit guidelines indicating how to score responses and norms with which to
compare data

6. Versions of the instrument (check all that apply):

___One version of the instrument
___Baseline/follow-up versions of the instrument
___Versions for different stakeholders groups (please list groups, e.g., consumer, family
member, etc.)

___Other (specify):______________

7. What translations of the instrument are available for non-English speakers?

7.1. Did the translations include any adaptation of the original instrument? (If so, briefly
describe the methods used to adapt the instrument and the adaptations to the
instrument)

Measuring the Promise: A Compendium of Recovery Measures, Volume II 103

 The Evaluation Center @ HSRI

II. Application
1. Populations for which the instrument is intended (check all that apply):
Age: ___Adult ___Adolescent ___Child

Persons who have the following diagnoses:

___Serious Mental Illness
___Dual Diagnosis
___Substance Abuse
___Other:_________________

Ethnicity/race
___American Indian or Alaska Native
___Asian
___Black or African American
___Native Hawaiian or Other Pacific Islander
___White
___Hispanic or Latino
___Not Hispanic or Latin
___Other (Specify):____________

2. Population(s) with which the instrument has been tested (check all that apply):
Age: ___Adult ___Adolescent ___Child

Persons who have the following diagnoses:

___Serious Mental Illness
___Dual Diagnosis
___Substance Abuse
___Other:_________________

Ethnicity/race
___American Indian or Alaska Native
___Asian
___Black or African American
___Native Hawaiian or Other Pacific Islander
___White
___Hispanic or Latino
___Not Hispanic or Latin
___Other (Specify):____________

3. In what setting(s) is the instrument intended for use? (Check all that apply)
___Criminal justice system
___Inpatient setting
___Outpatient setting
___Peer-run program
___Residential program
___Other:________________

4. In what setting(s) has the instrument been tested?(Check all that apply)
___Criminal justice system

Measuring the Promise: A Compendium of Recovery Measures, Volume II 104

 The Evaluation Center @ HSRI

___Inpatient setting
___Outpatient setting
___Peer-run program
___Residential program
___Other:________________

5. Intended administration method(s)? (Check all that apply)

Self administered:
___in person
___in a group
___by mail
___via the internet
Administered by an interviewer:
___in an individual interview
___in a group interview
___by a phone interview
___Observational method
___ Other (specify):_______________

6. Approximately how many minutes does it take to complete the instrument on-average?

7. What is the qualification and/or training requirement for individuals administering the
instrument?

8. What is the intended unit(s) of analysis for the instrument? (Check all that apply)
Program:
___Provider organization
___Specific Services
System:
___State mental health system
___Local mental health system
___Behavioral health care organization
___Multi-service agency
___Individual
___Other (specify):____________

Instrument Development and Evaluation

1. To what extent did cognitive testing inform the development of this instrument? Briefly
discuss.

2. Please describe the field test method(s), including information on the population(s),
setting(s), sample design, and sample size.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 105

 The Evaluation Center @ HSRI

3. Please report the psychometric properties appropriate to the instrument.

Reliability
3.1 Internal consistency reliability (describe how measured):

3.2 Test-retest reliability (describe how measured):

Interval between tests:

3.3 Interrater reliability (describe how measured):

Validity
3.4 Face validity (describe how measured):

3.5 Construct validity (describe how measured):

3.6 Concurrent validity (describe how measured):

3.7 Convergent validity (describe how measured):

3.8 Criterion validity (describe how measured):

3.9 Other (describe how measured):

Measuring the Promise: A Compendium of Recovery Measures, Volume II 106

 The Evaluation Center @ HSRI

4. What is the response rate for the instrument? How was the response rate calculated (e.g.,
define both the numerator and the denominator)? Discuss any follow-up involved in securing
responses?

5. Please specify the rates of missing data for the instrument

6. Please specify the rate of refusal for the instrument:

7. Describe any normality testing which has been done on the instrument (testing for floor and
ceiling effects).

Supporting Materials/Mechanisms
1. How can this instrument been used for quality improvement? Are training, materials, and
other supports for this available?

2. What other supporting material is available for the instrument?

___Informed consent material
___Information on administering the instrument
___Interviewer/Administration training
___Guidelines to scoring responses
___Guidelines to interpreting data scores
___Technical assistance
___Other (specify):_______________________

Permission to Use
1. Is permission required to use this instrument?
___The instrument can be used freely without contacting the author or listed contact ___
Permission is recommended but not required for use of the instrument
___Permission is required from the author or contact prior to using the instrument
___Other (specify):________________________________________________________

2. Copyright status of the instrument:

3. Contact information for the instrument:

4. Fees associated with instrument’s use:

Measuring the Promise: A Compendium of Recovery Measures, Volume II 107

 The Evaluation Center @ HSRI

Instrument Utilization
1. Descriptions of current and past uses:

2. Publication citations:

3. Plans for further testing and evaluation:

Summary
1. Strengths of the instrument:

Measuring the Promise: A Compendium of Recovery Measures, Volume II 108

 The Evaluation Center @ HSRI

Appendix C: Notes for Non-Researchers

Taken directly from Can We Measure Recovery? A Compendium of Recovery and Recovery-Related Instruments
(Volume I) (Ruth, Kidder, Phillips, 2000).

Notes for Non-Researchers

The language of measurement must be defined before one can understand what has been done to prove
that the instrument or measure developed does the job for which it was designed. Thus, we include some
definitions and discussions here, which may assist in understanding the summaries of instruments in this
collection and the articles or notes that accompany each measuring instrument.
There are two basic ways to study what people think about a certain topic or area. These are named
qualitative studies and quantitative studies.
In a qualitative study, the research uses what people say or write in words, for example, studies based on
short answers to open-ended questions, or on personal histories. Qualitative studies are sometimes used to
develop ideas and descriptions that can then be used to develop quantitative measures. Qualitative studies
may also assist in describing a program or a situation, providing a better picture than can be done with
only qualitative studies.
In a quantitative study or measure, questions are stated in such a way that the response can be numbered,
for example, 1. Strongly Agree, 2. Agree, 3. Disagree, 4. Strongly Disagree. Quantitative studies use a
number of terms and methods that are described and defined below.
Psychometrics, according to Webster’s II New Riverside University Dictionary, is “(1) measurement of
psychological variables, as intelligence, aptitude, and emotional disturbance; (2) mathematical, especially
statistical design of psychological tests and measures.” Psychometrics provide numbers that indicate how
consistent the measurement is from one time to another, and whether it measures what it is supposed to
be measuring.
The two major psychometric features used in the development and testing of measurement instruments
are reliability and validity. In general, reliability is the consistency of the measurement – that is, it will
measure the same thing each time it is used. Validity means that your measuring tool or scale is actually
measuring what you say it is. Both reliability and validity are determined in different ways.
There are several different types of reliability.
1. Test-retest reliability – a person gives the same answers today as they gave last week
(circumstances remaining the same). This is consistency over time, and is generally reported in a
correlation1 of time one with time two.
2. Internal consistency – the items or questions ask about the same things. This can be tested by
correlating one half of the items with the other half. A statistical method of averaging correlations of
a number of random selection of items was developed by a mathematician named Cronbach (1951)2,
and is called Cronbach’s Alpha, coefficient alpha, or sometimes just alpha.
3. Inter-rater reliability – when two or more people are interviewers, asking a number of people
the same questions, you want to be sure that the responses are the same, no matter who is asking
the questions. Again, a correlation is computed between responses when two people ask the same
people the same questions.

1
Correlation is a measure ranging from 0.00 to 1.00 of how well two or more things, e.g. item scores, change together. Both things may get higher at the same
time, or lower at the same time, or one may get higher while the other gets lower.
2
Cronbach L. (1951) Coefficient Alpha and the Internal Structure of Tests. Psychometrica, Vol. 16, pp. 297-334.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 109

 The Evaluation Center @ HSRI

There are also several types of validity:

1. Content Validity or Face Validity – when the items are all clearly related to the topic or title of the
test. When the items seem to make sense in relation to the topic being tested, it is also called face
validity.
2. Construct Validity – the measure of how well the instrument fits the ideas of a topic, theory, or
construct. It may be tested by finding out how two groups respond that are known to be at opposite
extremes about the topic or theory.
3. Concurrent Validity – when the test or measure is highly correlated with another known measure
of the same kind.
4. Predictive Validity – the test or measure can predict some practical result or some important
outcome.
After an instrument has been developed and tested, data analysis of responses must be done to determine
whether the responses are similar or different to what was expected. Sometimes, for example, a construct
or theory on which the test is built will appear to have several areas, and so items may be developed to test
for each hypothesized area. A statistical procedure called factor analysis will show which items are most
closely related to each other, and which are less closely related. Sometimes items will cluster together in
such a way that can be described as a test of one or more of these areas. These clusters of items are called
factors. If these factors can be named in a way that makes sense in relation to the theory, this is called fac-
tor validity.

These are the major terms used in the summary of the instruments in this compendium. For other research
terms, it is recommended that a glossary of research terms be found. Such a glossary which is valuable for
non-researchers is available from the Missouri Institute of Mental Health, and is cited as follows:

References

Rittenhouse, T., Cutler, S., & Campbell, J. (1999). Dressed-down research terms: A glossary for non-
researchers. St. Louis, MO: Missouri Institute of Mental Health.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 110

 The Evaluation Center @ HSRI

Appendix D: Instruments and Materials

I. Measures of Individual Recovery

1. Consumer Recovery Outcomes System (CROS 3.0)

2. Illness Management and Recovery (IMR) Scales

3. Mental Health Recovery Measure (MHRM)

4. Ohio Mental Health Consumer Outcomes System

5. Peer Outcomes Protocol (POP)

6. Reciprocal Support Scale

7. Recovery Assessment Scale (RAS)

8. Recovery Measurement Tool Version 4 (RMT)

9. Relationships and Activities that Facilitate Recovery Survey (RAFRS)

2. Measures of Recovery Promoting Environment

1. AACP ROSE - Recovery Oriented Services Evaluation

2. Recovery Enhancing Environment Measure (REE)

3. Recovery Oriented Systems Indicators Measure (ROSI)

4. Recovery Self-Assessment (RSA)

Measuring the Promise: A Compendium of Recovery Measures, Volume II 111

 The Evaluation Center @ HSRI

Consumer Recovery Outcomes System (CROS 3.0)

The Colorado Health Networks Partnership

For more information contact:

Anita Miller, Psy.D.

CROS, LLC
7150 Campus Drive, Suite 300
Colorado Springs, CO 80920
Phone: 800-804-5040 ext.1444
Email:anita.miller@valuesoption.com
Web site: www.crosllc.com

Permission to use:

The CROS is copyrighted by CROS,LLC. Permission is required from CROS, LLC prior to using the in-
strument. There is a user’s fee associated with the instrument. Prices vary depending whether the Com-
plete Processing Package Option or the Site License Option is selected.

The Complete Processing Package includes questionnaires, training, data processing and scoring, technical
support, and a variety of reports. Agencies pay a subscription fee for each consumer who will participate in
CROS. CROS is priced on a per user, per month (pupm) basis. Final pricing is determined by volume and
number of planned administrations per year. For 2 administrations per year, the price ranges from $7176.00
for 100 consumers to $10,800.00 for 500 consumers.

An agency choosing the Annual Site License option will get permission to reproduce and use the question-
naires. An administration manual and scoring instructions are included. Algorithms for spreadsheet scor-
ing and the production of the various reports are available for an additional $50.00. The Site License prices
range from $200.00 per year for use of the Consumer Questionnaire for 1-99 consumers to $400.00 per year
for use of the Consumer, Staff and VIP Questionnaires for 100+ consumers.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 112

 The Evaluation Center @ HSRI

EXECUTIVE SUMMARY

The Consumer Recovery Outcomes System (CROS 3.0):

Assessing Clinical Status and Progress in Persons with Severe and Persistent Mental Illness

Bernard L. Bloom, Ph.D., & Anita Miller, Psy.D.

The Consumer Recovery Outcomes System (CROS 3.0) consists of two brief integrated assessment ques-
tionnaires that are periodically completed by consumers with severe and persistent mental illnesses and
clinical staff responsible for their care in order to review clinical progress. The questionnaire set has two
principal goals – multidimensional assessment of clinical status and identification of areas of agreement
and disagreement between raters that may provide useful information for treatment planning. CROS 3.0
includes a “VIP” questionnaire, containing the same questions as the staff questionnaire, in order to record
the perceptions of a friend, acquaintance, or relative. This additional perspective is especially pertinent in
a treatment program that seeks to expand the individual’s self-identity and role beyond that of being only
a mental health consumer. Gathering enough data to study the psychometrics of the VIP form has been dif-
ficult but gaining the perspective of this third group of people remains an important endeavor.

In order to reflect the growing importance of the field of psychiatric rehabilitation and two relatively new
orientations to treatment – the “strengths perspective” and the “recovery-oriented perspective,” assess-
ment must be broadened beyond symptom status. In keeping with these concepts, CROS 3.0 assesses four
domains – Hope for the Future, Daily Functioning, Coping with Clinical Symptoms, and Quality of Life. In
addition, the consumer form includes an assessment of Treatment Satisfaction. The initial draft of CROS
was based on results of consumer and clinician focus group meetings designed to identify questionnaire
items that were strengths-based, clinically relevant to a recovery-oriented treatment approach, and that
avoided jargon and negative connotations.

Analysis of psychometric characteristics of CROS 3.0 is based on an initial sample of 585 consumers and
staff. The most common primary diagnosis (49%) was some type of schizophrenic disorder; 20% had a
primary diagnosis of depressive disorder; 17% had a primary diagnosis of bipolar disorder. Psychometrics
of the consumer and staff CROS 3.0 reveal the following: Scores on all nine scales are negatively skewed.
Consumer scores are consistently more skewed than staff scores. In addition, consumer scale score means
are consistently and significantly (p<. 01) higher than staff scale score means. The oblique factor analyses
of the 35 items on the CROS 3.0 consumer questionnaire and the 30 items on the CROS 3.0 staff ques-
tionnaire indicate a satisfactory relationship to the five conceptualized scales. Consumer and staff scale
score correlations are moderately high. It should also be noted that scores among the consumer scales and
among the staff scales are substantially correlated with each other. Demographic and diagnostic relation-
ships with CROS 3.0 scale scores were studied for subgroups separated by age, gender, ethnicity, home-
lessness, diagnosis, alcohol/street drug use, and medication adherence.

The reliability assessments of internal consistency and test-retest reliability for both staff and con-
sumer questionnaires are satisfactory. Staff inter-rater reliability measures were weaker but adequate.
Concurrent validity was also established for both the consumer and staff forms. Changes in CROS 3.0
scale scores over time were examined and scale scores between the first and second test administrations

Measuring the Promise: A Compendium of Recovery Measures, Volume II 113

 The Evaluation Center @ HSRI

were significantly correlated. All scores increased between the first and second test administrations and
were statistically significant in two scales on the consumer form and on all four scales of the staff form.

The psychometric analysis reveals a number of strengths; the factorial structure is sound, consumer and
staff scale scores are significantly correlated with each other without being redundant, measures of reli-
ability are quite satisfactory and initial measures of concurrent validity are encouraging, and, in spite of
the low ceiling, significant improvements over time were noted on six of the nine scales. Administration
takes only a few minutes and results can be presented in a variety of individualized and normative modes.
Careful and ongoing training to consumers and staff frequently remedies negatively skewed scores and
improves inter-rater reliability.

A nationwide study of CROS 3.0 with over 1100 consumer/staff dyads is in the planning stages. Advance-
ments to CROS 3.0 may subsequently be made. These data will provide additional rigorous scientific sup-
port for the use of CROS. CROS will be a component of a comprehensive approach to quality patient care,
as it is intended to inform patients and providers about deficits and strengths over the course of treatment
for serious mental illness.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 114

Please do not staple or photocopy these forms - the fax will not read them properly.

CROS L.L.C.
Consumer CROS 3.0
Consumer Recovery
Outcomes System
Questionnaire

Welcome! CROS is an outcomes system that

defines recovery as regaining meaning and purpose in one’s life.

How to complete these questions.

• Respond based on how you feel or what you think at the time you are
completing the questions.
• Use a black pen to mark your answers.
• Color in the circle that matches your answer to each question.

• If you make a mistake, draw a line through the circle; like this •
• Do not write over the black boxes in the corners.

In order for the questionnaire to be scored properly,

please complete all the questions.

Thank you for your cooperation!

Your answers to these questions will provide important information to your clinician
and influence the development of mental health programs.

CROS 3.0 is fully owned by CROS, L.L.C.

Copyright  2000. All rights reserved.
 Consumer CROS
This section is to be completed by staff. Month Day Year
Consumer Identifier 1 Date CROS is Filled Out -- --
Organization ID Agency ID Program ID Staff ID
¾ Think about the services you receive from this agency:
If you are new to this agency, start at Question 6.
Very Somewhat Somewhat Very
1. How do I feel about the amount of information I receive about my satisfied satisfied dissatisfied dissatisfied
mental illness? ○ ○ ○ ○
Very Somewhat Somewhat Very
2. How do I feel about the choices I get about my care? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
3. How do I feel about the amount of activities or groups I am offered to satisfied satisfied dissatisfied dissatisfied
help me cope with my mental illness? ○ ○ ○ ○
Very Somewhat Somewhat Very
4. How do I feel about the help I get finding services I need in the satisfied satisfied dissatisfied dissatisfied
community? ○ ○ ○ ○
Very Somewhat Somewhat Very
5. How do I feel about the availability of crisis services when I need them? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
¾ Think about your hopes for the future:
Believe strongly Believe Believe a little Do not believe
6. I believe I will handle my daily problems.
○ ○ ○ ○
Believe strongly Believe Believe a little Do not believe
7. I believe I will trust my thoughts and feelings.
○ ○ ○ ○
Believe strongly Believe Believe a little Do not believe
8. I believe I will feel alert and alive.
○ ○ ○ ○
Believe strongly Believe Believe a little Do not believe
9. I believe I will achieve goals I set for myself.
○ ○ ○ ○
Believe strongly Believe Believe a little Do not believe
10. I believe my life will have meaning.
○ ○ ○ ○
Believe strongly Believe Believe a little Do not believe
11. I believe I will have enjoyable experiences.
○ ○ ○ ○
Believe strongly Believe Believe a little Do not believe
12. I believe I will recover from mental illness.
○ ○ ○ ○
¾ Think about things you do in your daily life:
All the time Most of the time Sometimes Almost never
13. I plan and keep a daily routine on my own.
○ ○ ○ ○
All the time Most of the time Sometimes Almost never
14. I concentrate and finish tasks that I start.
○ ○ ○ ○
All the time Most of the time Sometimes Almost never
15. I make plans to do things with other people.
○ ○ ○ ○
16. I manage my residence (for example, do chores, pay bills, All the time Most of the time Sometimes Almost never
keep my room/house clean). ○ ○ ○ ○
All the time Most of the time Sometimes Almost never
17. I make my own decisions.
○ ○ ○ ○
18. I take care of my personal appearance (for example, bathe, All the time Most of the time Sometimes Almost never
wear clean clothes). ○ ○ ○ ○
19. I am involved in meaningful activity (for example, All the time Most of the time Sometimes Almost never
employment, school, volunteer work). ○ ○ ○ ○
20. How much progress am I making in recovering from my A great deal Some Very little None at all
mental illness? ○ ○ ○ ○

Consumer CROS
Copyright  2000 by CROS, L.L.C. All rights reserved. Page 1 of 2
 Consumer CROS
¾ Think about how well you are coping:
Very well - Fairly well - Not so well - Very poorly -
21. How well am I coping with feeling sad? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
22. How well am I coping with feeling tense or anxious? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
23. How well am I coping with feeling angry or hostile never a problem rarely a problem often a problem always a problem
toward others? ○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
24. How well am I coping with disturbing thoughts? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
25. How well am I coping with difficulty sleeping? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
26. How well am I coping with external stressors (for Very well - Fairly well - Not so well - Very poorly -
never a problem rarely a problem often a problem always a problem
example, health problems, family conflicts,
unemployment, tragedy in community)?
○ ○ ○ ○
¾ Think about how satisfied you are with your life:
Very Somewhat Somewhat Very
27. How do I feel about the amount of freedom I have where I live? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
28. How do I feel about having things to do that I enjoy (for example, satisfied satisfied dissatisfied dissatisfied
going to the movies, sports events, hobbies)? ○ ○ ○ ○
Very Somewhat Somewhat Very
29. How do I feel about the way I spend my free time? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
30. How do I feel about the amount of privacy I have where I live? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
31. How do I feel about the amount of comfort and security I have where I satisfied satisfied dissatisfied dissatisfied
live? ○ ○ ○ ○
Very Somewhat Somewhat Very
32. How do I feel about my access to health care? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
33. How do I feel about the neighborhood and residence where I live? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
34. How do I feel about my access to reliable transportation? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
35. How do I feel about my relationships with family/significant satisfied satisfied dissatisfied dissatisfied
others/friends? ○ ○ ○ ○
Believe Believe Do not
36. I believe my use of alcohol or street drugs interferes with my recovery from
strongly Believe a little believe
mental illness. I do not use alcohol or street drugs. ○
○ ○ ○ ○
All the Most of Some- Almost
37. I take my psychiatric medication as prescribed.
time the time times never
Psychiatric medication has not been prescribed for me. ○
○ ○ ○ ○
Believe Believe Do not
38. I believe taking my psychiatric medication helps with my recovery from
strongly Believe a little believe
mental illness. Psychiatric medication has not been prescribed for me. ○
○ ○ ○ ○
Did anyone help you complete this form? Yes ○ No ○
Was this questionnaire translated into another language for you? Yes ○ No ○

The End. Thank You!

Consumer ID 1
Consumer CROS
Copyright  2000 by CROS, L.L.C. All rights reserved. Page 2 of 2
Please do not staple or photocopy these forms - the fax will not read them properly.

CROS L.L.C.
Staff CROS 3.0
Consumer Recovery
Outcomes System
Questionnaire

Welcome! CROS is an outcomes system that

defines recovery as regaining meaning and purpose in one’s life.

How to complete these questions.

• Respond based on how you feel or what you think at the time you are completing the
questions.
• Use a black pen to mark your answers.
• Color in the circle that matches your answer to each question.

• If you make a mistake, draw a line through the circle; like this •
• Write numbers clearly; like this

1 2 3 4 5 6 7 8 9 Ø

• Do not write over the black boxes in the corners.

In order for the questionnaire to be scored properly,

please complete all the questions.

Thank you for your cooperation!

Your answers to these questions will provide important information
and influence the development of mental health programs.

CROS 3.0 is fully owned by CROS, L.L.C.

Copyright  2000. All rights reserved.
 Staff CROS
Month Day Year

Consumer Identifier 1 Date CROS is Filled Out -- --

Organization ID Agency ID Program ID Staff ID
1. Current Assessment (DSM-IV)

Primary . Secondary . Tertiary .

2. Last hospital discharge was: ○ Within last 6 months ○ Within 6 months to 1 year ago ○ Over 1 year ago
(Leave blank if never hospitalized)
3. Does the consumer have a representative payee? ○ Yes ○ No
4. Is the consumer chronically homeless? ○ Yes ○ No
5. Is the consumer new to this agency? ○ Yes ○ No
¾ Think about the consumer’s hopes for the future:
All Most of Some- Almost
6. To what extent does the consumer seem to believe he/she will handle daily problems? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
7. To what extent does the consumer seem to believe he/she will trust his/her thoughts and the time the time times never
feelings?
○ ○ ○ ○
All Most of Some- Almost
8. To what extent does the consumer seem to believe he/she will feel alert and alive? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
9. To what extent does the consumer seem to believe he/she will achieve goals he/she sets? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
10. To what extent does the consumer seem to believe his/her life will have meaning? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
11. To what extent does the consumer seem to believe he/she will have enjoyable experiences? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
12. To what extent does the consumer seem to believe he/she will recover from mental illness? the time the time times never
○ ○ ○ ○
¾ Think about things the consumer does in his/her daily life:
All Most of Some- Almost
13. How often does the consumer plan and keep a daily routine on his/her own? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
14. How often does the consumer concentrate and finish tasks that he/she starts? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
15. How often does the consumer make plans to do things with other people? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
16. How often does the consumer manage his/her residence (for example, do chores, pay bills, keep the time the time times never
his/her room/house clean)?
○ ○ ○ ○
All Most of Some- Almost
17. How often does the consumer make his/her own decisions? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
18. How often does the consumer take care of his/her personal appearance (for example, bathe, the time the time times never
wear clean clothes)?
○ ○ ○ ○
All Most of Some- Almost
19. How often is the consumer involved in meaningful activity (for example, employment, school, the time the time times never
volunteer work)?
○ ○ ○ ○
A great Very None
20. How much progress is the consumer making toward recovering from his/her mental illness? deal Some little at all
○ ○ ○ ○

Staff CROS
Copyright  2000 by CROS, L.L.C. All rights reserved. Page 1 of 2
 Staff CROS
¾ Think about how well the consumer is coping:
Very well - Fairly well - Not so well - Very poorly -
21. How well is the consumer coping with feeling sad? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
22. How well is the consumer coping with feeling tense or anxious? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
23. How well is the consumer coping with feeling angry or hostile never a problem rarely a problem often a problem always a problem
toward others?
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
24. How well is the consumer coping with disturbing thoughts? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
25. How well is the consumer coping with difficulty sleeping? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
26. How well is the consumer coping with external stressors (health Very well - Fairly well - Not so well - Very poorly -
problems, family conflicts, unemployment, tragedy in never a problem rarely a problem often a problem always a problem
community)? ○ ○ ○ ○
¾ Think about how satisfied the consumer is with his/her life:
Very Somewhat Somewhat Very
27. How does the consumer feel about the amount of freedom he/she has where he/she satisfied satisfied dissatisfied dissatisfied
lives?
○ ○ ○ ○
Very Somewhat Somewhat Very
28. How does the consumer feel about having things to do that he/she enjoys (for satisfied satisfied dissatisfied dissatisfied
example, going to the movies, sports events, hobbies)?
○ ○ ○ ○
Very Somewhat Somewhat Very
29. How does the consumer feel about the way he/she spends his/her free time? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
30. How does the consumer feel about the amount of privacy he/she has where he/she satisfied satisfied dissatisfied dissatisfied
lives?
○ ○ ○ ○
Very Somewhat Somewhat Very
31. How does the consumer feel about the amount of comfort and security he/she has satisfied satisfied dissatisfied dissatisfied
where he/she lives?
○ ○ ○ ○
Very Somewhat Somewhat Very
32. How does the consumer feel about his/her access to health care? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
33. How does the consumer feel about the neighborhood and residence where he/she satisfied satisfied dissatisfied dissatisfied
lives?
○ ○ ○ ○
Very Somewhat Somewhat Very
34. How does the consumer feel about his/her access to reliable transportation? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
35. How does the consumer feel about his/her relationships with family/significant satisfied satisfied dissatisfied dissatisfied
others/friends?
○ ○ ○ ○
36. To what extent do you believe the consumer’s use of alcohol or street drugs Strongly Believe a Do not
interferes with his/her recovery from mental illness? believe Believe little believe
The consumer does not use alcohol or street drugs. ○ ○ ○ ○ ○
All the Most of Almost
37. To what extent do you believe the consumer takes his/her psychiatric medication as
time the time Sometimes never
prescribed? Psychiatric medication has not been prescribed for the consumer. ○ ○ ○ ○ ○
38. To what extent do you believe the consumer’s psychiatric medication helps with Strongly Believe a Do not
his/her recovery from mental illness? believe Believe little believe
Psychiatric medication has not been prescribed for the consumer. ○ ○ ○ ○ ○

The End. Thank You!

Consumer ID 1
Staff CROS
Copyright  2000 by CROS, L.L.C. All rights reserved. Page 2 of 2
Please do not staple or photocopy these forms - the fax will not read them properly.

CROS L.L.C.
VIP CROS 3.0
Questionnaire for
Consumer Recovery
Outcomes System
__________________________
Consumer’s Name

Welcome! You have been chosen by a consumer of mental health services as

someone who is a Very Important Person (VIP) in his or her life. By completing this
questionnaire, your perspective on the consumer’s strengths, progress and treatment needs will
contribute to effective treatment planning. CROS defines
recovery as regaining meaning and purpose in one’s life.
How to complete this questionnaire.

• Respond based on how you feel or what you think at the time you are completing the questions.
• Use a black pen to mark your answers.
• Color in the circle that matches your answer to each question.
• If you make a mistake, draw a line through the circle; like this 
• Do not write over the black boxes in the corners.
• When you are done, return this entire questionnaire to the mental health center.

Please complete all the questions.

Consent Section
Before you complete this CROS questionnaire, there are some important things you should understand:
• Completing this CROS questionnaire is voluntary. If you do not participate, the mental health services the consumer
receives will not be affected, changed or reduced in any way. You also may refuse to answer specific questions.
• Completing this CROS questionnaire includes benefits and risks. Questions will lead you to think about areas of the
consumer’s life that are important in his or her recovery. Benefits include providing information to the consumer and
his or her counselor that will help them develop a treatment plan that builds upon the consumer’s strengths and
skills. A potential risk may be some distress to you or the consumer if the consumer’s perceptions are significantly
different from your perceptions.
• Confidentiality: By agreeing to complete this CROS questionnaire, you are agreeing to share information with the
consumer and his or her treatment team. Information from this questionnaire will be maintained in a database and
may be used for research. Your name will not be in this database or on this questionnaire when it is sent to the
database.
If you have questions or concerns about CROS, please talk with the consumer’s counselor.

________________________________ ________________________________ __________

VIP Name (print) VIP Signature Date
This consent is valid for this CROS questionnaire only.
* File this page in the consumer’s chart.

Thank you for your cooperation!

CROS 3.0 is fully owned by CROS, L.L.C.

Copyright  2000. All rights reserved.
 VIP CROS
Consumer Identifier 1 Date CROS is Filled Out -- --
Month Day Year

Organization ID Agency ID Program ID Staff ID

¾ Start here (completely fill only one circle for each of the following questions)
Please indicate your relationship to the consumer:
○ Spouse/Significant Other ○ Other Relative ○ Residential Facility Staff
○ Parent/Step-parent ○ Friend ○ Other
¾ Think about the consumer’s hopes for the future:
All Most of Some- Almost
1. To what extent does the consumer seem to believe he/she will handle daily problems? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
2. To what extent does the consumer seem to believe he/she will trust his/her thoughts and the time the time times never
feelings?
○ ○ ○ ○
All Most of Some- Almost
3. To what extent does the consumer seem to believe he/she will feel alert and alive? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
4. To what extent does the consumer seem to believe he/she will achieve goals he/she sets? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
5. To what extent does the consumer seem to believe his/her life will have meaning? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
6. To what extent does the consumer seem to believe he/she will have enjoyable experiences? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
7. To what extent does the consumer seem to believe he/she will recover from mental illness? the time the time times never
○ ○ ○ ○
¾ Think about things the consumer does in his/her daily life:
All Most of Some- Almost
8. How often does the consumer plan and keep a daily routine on his/her own? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
9. How often does the consumer concentrate and finish tasks that he/she starts? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
10. How often does the consumer make plans to do things with other people? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
11. How often does the consumer manage his/her residence (for example, do chores, pay bills, keep the time the time times never
his/her room/house clean)?
○ ○ ○ ○
All Most of Some- Almost
12. How often does the consumer make his/her own decisions? the time the time times never
○ ○ ○ ○
All Most of Some- Almost
13. How often does the consumer take care of his/her personal appearance (for example, bathe, the time the time times never
wear clean clothes)?
○ ○ ○ ○
All Most of Some- Almost
14. How often is the consumer involved in meaningful activity (for example, employment, school, the time the time times never
volunteer work)?
○ ○ ○ ○
A great Very None
15. How much progress is the consumer making toward recovering from his/her mental illness? deal Some little at all
○ ○ ○ ○

VIP CROS
Copyright  2000 by CROS L.L.C. All rights reserved. Page 1 of 2
 VIP CROS
¾ Think about how well the consumer is coping:
Very well - Fairly well - Not so well - Very poorly -
16. How well is the consumer coping with feeling sad? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
17. How well is the consumer coping with feeling tense or anxious? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
18. How well is the consumer coping with feeling angry or hostile never a problem rarely a problem often a problem always a problem
toward others?
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
19. How well is the consumer coping with disturbing thoughts? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
Very well - Fairly well - Not so well - Very poorly -
20. How well is the consumer coping with difficulty sleeping? never a problem rarely a problem often a problem always a problem
○ ○ ○ ○
21. How well is the consumer coping with external stressors (health Very well - Fairly well - Not so well - Very poorly -
problems, family conflicts, unemployment, tragedy in never a problem rarely a problem often a problem always a problem
community)? ○ ○ ○ ○
¾ Think about how satisfied the consumer is with his/her life:
Very Somewhat Somewhat Very
22. How does the consumer feel about the amount of freedom he/she has where he/she satisfied satisfied dissatisfied dissatisfied
lives?
○ ○ ○ ○
Very Somewhat Somewhat Very
23. How does the consumer feel about having things to do that he/she enjoys (for satisfied satisfied dissatisfied dissatisfied
example, going to the movies, sports events, hobbies)?
○ ○ ○ ○
Very Somewhat Somewhat Very
24. How does the consumer feel about the way he/she spends his/her free time? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
25. How does the consumer feel about the amount of privacy he/she has where he/she satisfied satisfied dissatisfied dissatisfied
lives?
○ ○ ○ ○
Very Somewhat Somewhat Very
26. How does the consumer feel about the amount of comfort and security he/she has satisfied satisfied dissatisfied dissatisfied
where he/she lives?
○ ○ ○ ○
Very Somewhat Somewhat Very
27. How does the consumer feel about his/her access to health care? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
28. How does the consumer feel about the neighborhood and residence where he/she satisfied satisfied dissatisfied dissatisfied
lives?
○ ○ ○ ○
Very Somewhat Somewhat Very
29. How does the consumer feel about his/her access to reliable transportation? satisfied satisfied dissatisfied dissatisfied
○ ○ ○ ○
Very Somewhat Somewhat Very
30. How does the consumer feel about his/her relationships with family/significant satisfied satisfied dissatisfied dissatisfied
others/friends?
○ ○ ○ ○
31. To what extent do you believe the consumer’s use of alcohol or street drugs Strongly Believe a Do not
interferes with his/her recovery from mental illness? believe Believe little believe
The consumer does not use alcohol or street drugs. ○ ○ ○ ○ ○
All the Most of Almost
32. To what extent do you believe the consumer takes his/her psychiatric medication as
time the time Sometimes never
prescribed? Psychiatric medication has not been prescribed for the consumer. ○ ○ ○ ○ ○
33. To what extent do you believe the consumer’s psychiatric medication helps with Strongly Believe a Do not
his/her recovery from mental illness? believe Believe little believe
Psychiatric medication has not been prescribed for the consumer. ○ ○ ○ ○ ○

The End. Thank You!

Consumer ID 1
VIP CROS
Copyright  2000 by CROS L.L.C. All rights reserved. Page 2 of 2
 The Evaluation Center @ HSRI

ILLNESS MANAGEMENT AND RECOVERY(IMR) SCALES

Mueser, K., & Gingerich, S.

For more information contact:

Kim T. Mueser, Ph.D.

New-Hampshire-Dartmouth Psychiatric Research Center
Main Building
105 Pleasant St.
Concord, NH 03301
Email: Kim.T.Mueser@Dartmouth.edu

Permission to use:

The IMR Scales are not copyrighted and can be used freely without contacting the author or listed con-
tact. There is not a user’s fee associated with the scales.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 124

 Illness Management and Recovery Scale:
Client Self-Rating
ID Number:__________________________ Date:_______

Please take a few minutes to fill out this survey. We are interested in the way things are
for you, so there is no right or wrong answer. If you are not sure about a question, just
answer it as best as you can.

Just circle the number of the answer that fits you best.

1. Progress towards personal goals: In the past 3 months, I have come up with…

1 2 3 4 5

A personal goal, A personal goal A personal goal A personal

No but have not done and made it a and have gotten goal and
personal anything to finish little way toward pretty far in have finished
goals my goal. finishing it. finishing my goal. it.

2. Knowledge: How much do you feel like you know about symptoms, treatment, coping
strategies (coping methods), and medication?

1 2 3 4 5

Not very much A little Some Quite a bit A great deal

3. Involvement of family and friends in my mental health treatment: How much are
family members, friends, boyfriend/girlfriend, and other people who are important to you
(outside your mental health agency) involved in your mental health treatment?

1 2 3 4 5

A lot of the time and

Not Only when Sometimes, like when Much of they really help me
at all there is a things are starting to the time with my mental
serious problem go badly health
4. Contact with people outside of my family: In a normal week, how many times do you
talk to someone outside of your family (like a friend, co-worker, classmate, roommate,
etc.)
1 2 3 4 5

0 times/ 1-2 times/ 3-4 times/ 6-7 times/ 8 or more times/

week week week week week

5. Time in Structured Roles: How much time do you spend working, volunteering, being
a student, being a parent, taking care of someone else or someone else’s house or
apartment? That is, how much time do you spend in doing activities for or with another
person that are expected of you? (This would not include selfcare or personal home
maintenance.)

1 2 3 4 5
2 hours or less/ 3-5 hours/ 6 to 15 hours/ 16-30 hours/ More than 30
week week week week hours/ week

6. Symptom distress: How much do your symptoms bother you?

1 2 3 4 5
My symptoms My symptoms My symptoms My symptoms My symptoms
really bother bother me quite bother me bother me very don’t bother me
me a lot. a bit. somewhat. little. at all.

7. Impairment of functioning: How much do your symptoms get in the way of you doing
things that you would like to or need to do?

1 2 3 4 5
My symptoms My symptoms My symptoms My symptoms My symptoms
really get in my get in my way get in my way get in my way don’t get in my
way a lot. quite a bit. somewhat. very little. way at all.
8. Relapse Prevention Planning: Which of the following would best describe what you
know and what you have done in order not to have a relapse?

1 2 3 4 5
I have several I have a
I don’t know I know a little, but I know 1 or 2
things that I can written plan
how to I haven’t made a things I can do,
do, but I don’t that I have
prevent relapse prevention but I don’t have
have a written shared with
relapses. plan. a written plan
plan others.

9. Relapse of Symptoms: When is the last time you had a relapse of symptoms (that is,
when your symptoms have gotten much worse)?

1 2 3 4 5
I haven’t had a
Within the In the past 2 to In the past 4 to In the past 7 to
relapse in the past
last month 3 months 6 months 12 months
year

10. Psychiatric Hospitalizations: When is the last time you have been hospitalized for
mental health or substance abuse reasons?

1 2 3 4 5
I haven’t been
Within the In the past 2 to In the past 4 to In the past 7 to
hospitalized in the past
last month 3 months 6 months 12 months
year

11. Coping: How well do feel like you are coping with your mental or emotional illness
from day to day?

1 2 3 4 5
Not well at all Not very well Alright Well Very well
12. Involvement with self-help activities: How involved are you in consumer run
services, peer support groups, Alcoholics Anonymous, drop-in centers, WRAP (Wellness
Recovery Action Plan), or other similar self-help programs?

1 2 3 4 5
I know about I’m interested in
I don’t know I participate in I participate
some self-help self-help activities,
about any self-help in self-help
activities, but but I have not
self-help I’m not activities activities
participated in the
activities interested occasionally. regularly.
past year

13. Using Medication Effectively: (Don’t answer this question if your doctor has not
prescribed medication for you). How often do you take your medication as prescribed?

1 2 3 4 5
Never Occasionally About half the time Most of the time Every day

14. Functioning affected by alcohol use. Drinking can interfere with functioning when it
contributes to conflict in relationships, or to money, housing and legal concerns, to
difficulty showing up at appointments or paying attention during them, or to increased
symptoms. Over the past 3 months, how much did drinking get in the way of your
functioning?

1 2 3 4 5
Alcohol use Alcohol use Alcohol use gets Alcohol use Alcohol use is not
really gets in gets in my in my way gets in my a factor in my
my way a lot way quite a bit somewhat way very little functioning

15. Functioning affected by drug use. Using street drugs, and misusing prescription or
over-the-counter medication can interfere with functioning when it contributes to conflict
in relationships, or to money, housing and legal concerns, to difficulty showing up at
appointments or paying attention during them, or to increased symptoms. Over the past 3
months, how much did drug use get in the way of your functioning?

1 2 3 4 5
Drug use Drug use Drug use gets Drug use Drug use is not
really gets in gets in my in my way gets in my a factor in my
my way a lot way quite a bit somewhat way very little functioning
 Illness Management and Recovery Scale:
Clinician Rating
Clinician/Team Name:_________________________ Date:__________
Study ID#: ___________

Please take a few moments to fill out the following survey regarding your perception of
your client’s ability to manage her or his illness, as well as her or his progress toward
recovery. We are interested in the way you feel about how things are going for your
client, so please answer with your honest opinion. If you are not sure about an item, just
answer as best as you can.

Please circle the answer that fits your client the best.

1. Progress toward goals: In the past 3 months, s/he has come up with…

1 2 3 4 5
A personal A personal goal A personal goal
goal, but has and made it a and has gotten A personal goal
No personal
not done little way pretty far in and has
goals
anything to toward finishing the finished it
finish the goal finishing it goal

2. Knowledge: How much do you feel your client knows about symptoms, treatment,
coping strategies (coping methods), and medication?

1 2 3 4 5
Not very much A little Some Quite a bit A great deal

3. Involvement of family and friends in his/her mental health treatment: How much are
people like family, friends, boyfriends/girlfriends, and other people who are important to
your client (outside the mental health agency) involved in his/her treatment?

1 2 3 4 5
Sometimes, A lot of the
Only when
like when time and they
there is a Much of the
Not at all things are really help with
serious time
starting to go his/her mental
problem
badly health
4. Contact with people outside of the family: In a normal week, how many times does
s/he talk to someone outside of her/his family (like a friend, co-worker, classmate,
roommate, etc.)?

1 2 3 4 5
0 times/ 1-2 times/ 3-4 times/ 6-7 times/ 8 or more times/
week week week week week

5. Time in Structured Roles: How much time does s/he spend working, volunteering,
being a student, being a parent, taking care of someone else or someone else’s house or
apartment? That is, how much time does s/he spend in doing activities for or with another
person that are expected of him/her? (This would not include self-care or personal home
maintenance.)

1 2 3 4 5
2 hours or less/ 3-5 hours/ 6 to 15 hours/ 16-30 hours/ More than 30
week week week week hours/ week

6. Symptom distress: How much do symptoms bother him/her?

1 2 3 4 5
Symptoms really Symptoms Symptoms Symptoms Symptoms
bother him/her a bother him/her bother him/her bother him/her don’t bother
lot quite a bit somewhat very little him/her at all

7. Impairment of functioning: How much do symptoms get in the way of him/her doing
things that s/he would like to do or needs to do?

1 2 3 4 5
Symptoms
Symptoms really Symptoms get Symptoms get Symptoms get
don’t get in
get in her/his in his/her way in his/her way in his/her way
his/her way at
way a lot quite a bit somewhat very little
all

8. Relapse Prevention Planning: Which of the following would best describe what s/he
knows and has done in order not to have a relapse?

1 2 3 4 5
Knows 1 or 2 Knows several
Knows a little, Has a written
Doesn’t know things to do, things to do,
but hasn’t plan and has
how to prevent but doesn’t but doesn’t
made a relapse shared it with
relapses have a written have a written
prevention plan others
plan plan
9. Relapse of Symptoms: When is the last time s/he had a relapse of symptoms (that is,
when his/her symptoms have gotten much worse)?

1 2 3 4 5
Hasn’t had a
Within the last In the past 2 to In the past 4 to In the past 7 to
relapse in the
month 3 months 6 months 12 months
past year

10. Psychiatric Hospitalizations: When is the last time s/he has been hospitalized for
mental health or substance abuse reasons?

1 2 3 4 5
No
Within the last In the past 2 to In the past 4 to In the past 7 to
hospitalization
month 3 months 6 months 12 months
in the past year

11. Coping: How well do feel your client is coping with her/his mental or emotional
illness from day to day?

1 2 3 4 5
Not well at all Not very well Alright Well Very well

12. Involvement with self-help activities: How involved is s/he in consumer run services,
peer support groups, Alcoholics Anonymous, drop-in centers, WRAP (Wellness
Recovery Action Plan), or other similar self-help programs?

1 2 3 4 5
Is interested in
Knows about self-help Participates in Participates in
Doesn’t know
some self-help activities, but self-help self-help
about any self-
activities, but hasn’t activities activities
help activities participated in
isn’t interested occasionally regularly
the past year

13. Using Medication Effectively: (Don’t answer this question if her/his doctor has not
prescribed medication). How often does s/he take his/her medication as prescribed?

1 2 3 4 5
About half the Most of the
Never Occasionally Every day
time time

_____ Check here if the client is not prescribed psychiatric medications.

14. Impairment of functioning through alcohol use: Drinking can interfere with
functioning when it contributes to conflict in relationships, or to financial, housing and
legal concerns, to difficulty attending appointments or focusing during them, or to
increases of symptoms. Over the past 3 months, did alcohol use get in the way of his/her
functioning?

1 2 3 4 5

Alcohol use Alcohol use Alcohol use Alcohol use is

Alcohol use
gets in his/her gets in his/her gets in his/her not a factor in
really gets in
way quite a bit way somewhat way very little his/her
her/his way a lot
functioning

15. Impairment of functioning through drug use: Using street drugs, and misusing
prescription or over-the-counter medication can interfere with functioning when it
contributes to conflict in relationships, or to financial, housing and legal concerns, to
difficulty attending appointments or focusing during them, or to increases of symptoms.
Over the past 3 months, did drug use get in the way of his/her functioning?

1 2 3 4 5

Drug use gets Drug use gets Drug use gets Drug use is not
Drug use really
in his/her way in his/her way in his/her way a factor in
gets in her/his
quite a bit somewhat very little his/her
way a lot
functioning
 The Evaluation Center @ HSRI

MENTAL HEALTH RECOVERY MEASURE (MHRM)

Young, S., & Bullock, W.

For more information contact:

Wesley A. Bullock, Ph.D.
Department of Psychology (#948)
University of Toledo
2801 W. Bancroft St.
Toledo, OH 43606-3390
Phone: 419-530-2719
Email: wesley.bullock@utoledo.edu

Permission to use:

The MHRM is copyrighted. The instrument may be reproduced freely as long as the author citation and
author contact information is retained on the form. Users are encouraged to contact the author for further
information on scoring and normative data for the MHRM.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 133

 Mental Health Recovery Measure (MHRM)©
(Young & Bullock, 2003)

Client’s Name: __________________________________________________ Date: _______________

The goal of this questionnaire is to find out how you view your own current recovery process.
The mental health recovery process is complex and is different for each individual. There are
no right or wrong answers. Please read each statement carefully, with regard to your own
current recovery process, and indicate how much you agree or disagree with each item by
filling in the appropriate circle.

SD = Strongly Disagree D = Disagree NS = Not Sure A = Agree SA = Strongly Agree

SD D NS A SA

1. I work hard towards my mental health recovery. O O O O O

2. Even though there are hard days, things are improving for me. O O O O O

3. I ask for help when I am not feeling well. O O O O O

________________________________________________________________________________________________

4. I take risks to move forward with my recovery. O O O O O

5. I believe in myself. O O O O O

6. I have control over my mental health problems. O O O O O

________________________________________________________________________________________________

7. I am in control of my life. O O O O O

8. I socialize and make friends. O O O O O

9. Every day is a new opportunity for learning. O O O O O

________________________________________________________________________________________________

10. I still grow and change in positive ways despite my mental health problems. O O O O O

11. Even though I may still have problems, I value myself as a person of worth. O O O O O

12. I understand myself and have a good sense of who I am. O O O O O

________________________________________________________________________________________________

13. I eat nutritious meals everyday. O O O O O

14. I go out and participate in enjoyable activities every week. O O O O O

15. I make the effort to get to know other people. O O O O O

_______________________________________________________________________________________________

Please continue on next page.

 SD = Strongly Disagree D = Disagree NS = Not Sure A = Agree SA = Strongly Agree

SD D NS A SA

16. I am comfortable with my use of prescribed medications. O O O O O

17. I feel good about myself. O O O O O

18. The way I think about things helps me to achieve my goals. O O O O O

________________________________________________________________________________________________
19. My life is pretty normal. O O O O O

20. I feel at peace with myself. O O O O O

21. I maintain a positive attitude for weeks at a time. O O O O O

________________________________________________________________________________________________

22. My quality of life will get better in the future. O O O O O

23. Every day that I get up, I do something productive. O O O O O

24. I am making progress towards my goals. O O O O O

________________________________________________________________________________________________

25. When I am feeling low, my religious faith or spirituality helps me feel better. O O O O O

26. My religious faith or spirituality supports my recovery. O O O O O

27. I advocate for the rights of myself and others with mental health problems. O O O O O
________________________________________________________________________________________________

28. I engage in work or other activities that enrich myself and the world around O O O O O
me.

29. I cope effectively with stigma associated with having a mental health O O O O O
problem.

30. I have enough money to spend on extra things or activities that enrich my O O O O O
life.

Thank you for completing this measure.

The MHRM© was developed with the help of mental health consumers by researchers at the
University of Toledo, Department of Psychology. This research was supported through a
grant from the Ohio Department of Mental Health, Office of Program Evaluation and
Research. For further information, please contact Wesley A. Bullock, Ph.D. at (419) 530-2721
or email: wesley.bullock@utoledo.edu.
 The Evaluation Center @ HSRI

OHIO MENTAL HEALTH CONSUMER OUTCOMES SYSTEM

Ohio Department of Mental Health Office of Program Evaluation and Research

For more information contact:

Dee Roth, M.A.
Ohio Department of Mental Health
Phone: 614-466-8651
Email: rothd@mh.state.oh.us

Web site: http://www.mh.state.oh.us/oper/outcomes/outcomes.index.html

Permission to use:

The OMHCOS is copyrighted. Permission is required for use of the instruments outside of the state of
Ohio. The Adult Instruments are free for use within Ohio, however, out-of-state parties must pay a small
usage fee.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 136

 Ohio Mental Health Consumer Outcomes System
Adult Consumer Form A A
Today’s Date Agency Use Only

Name Client’s Medical Record Number:

Date of Birth

Gender (check one): Male Female

We are very interested in how you are doing, and how our services may or may not be helping you. Please
answer all of the questions below, then give the questionnaire to your case manager or another staff person
at the mental health agency.
4. How much money you have to
Part 1 spend for fun?
Below are some questions about how satisfied you Terrible
are with various aspects of your life in the past 6 Mostly dissatisfied
Equally satisfied/dissatisfied
months. For each question, checkmark the Mostly satisfied
answer that best describes how you feel. Very pleased

How do you feel about: 5. The amount of meaningful activity in

your life (such as work, school,
volunteer activity, leisure activity)?
1. The amount of friendship in your life?
Terrible
Terrible Mostly dissatisfied
Mostly dissatisfied Equally satisfied/dissatisfied
Equally satisfied/dissatisfied Mostly satisfied
Mostly satisfied Very pleased
Very pleased
6. The amount of freedom you have?
2. The amount of money you get?
Terrible Terrible
Mostly dissatisfied Mostly dissatisfied
Equally satisfied/dissatisfied Equally satisfied/dissatisfied
Mostly satisfied Mostly satisfied
Very pleased Very pleased

3. How comfortable and well-off you are financially? 7. The way you and your family act
Terrible toward each other?
Mostly dissatisfied
Terrible
Equally satisfied/dissatisfied
Mostly dissatisfied
Mostly satisfied Equally satisfied/dissatisfied
Very pleased Mostly satisfied
Very pleased
Does not apply

Please turn to the next page

02/16/2000....................................................................................................................................................................................................... Page 1 of 6
 8. Your personal safety? 14. Concerns about my medications (such
as side effects, dosage, type of
Terrible medication) are addressed:
Mostly dissatisfied
Equally satisfied/dissatisfied Never
Seldom/rarely
Mostly satisfied
Sometimes
Very pleased
Often
9. The neighborhood in which you live? Always
Not applicable/no medications
Terrible
Mostly dissatisfied
Equally satisfied/dissatisfied The next two items deal with how you have
Mostly satisfied been treated by other people.
Very pleased

10. Your housing/living arrangements? 15. I have been treated with dignity and
respect at this agency.
Terrible
Mostly dissatisfied Never
Equally satisfied/dissatisfied Seldom/rarely
Mostly satisfied Sometimes
Very pleased Often
Always
11. Your health in general?
16. How often do you feel threatened by
Terrible people’s reactions to your mental health
Mostly dissatisfied problems?
Equally satisfied/dissatisfied
Mostly satisfied Never
Seldom/rarely
Very pleased
Sometimes
12. How often do you have the opportunity to Often
spend time with people you really like? Always

Never
Seldom/rarely Part 3
Sometimes The following questions ask you about how
Often
much you were distressed or bothered by
Always
some things during the last seven days.
Part 2 Please mark the answer that best describes

These next few items ask you about your health how you feel.

and medications within the past 6 months.

During the past 7 days, about how much
were you distressed or bothered by:
13. How often does your physical condition
interfere with your day-to-day functioning? 17. Nervousness or shakiness inside
Never
Not at all
Seldom/rarely
A little bit
Sometimes
Some
Often
Quite a bit
Always
Extremely

02/16/2000....................................................................................................................................................................................................... Page 2 of 6
18. Being suddenly scared for no reason 25. Feeling of worthlessness
Not at all Not at all
A little bit A little bit
Some Some
Quite a bit Quite a bit
Extremely Extremely

26. Feeling lonely even when you are

19. Feeling fearful with people
Not at all
Not at all
A little bit
A little bit
Some
Some
Quite a bit
Quite a bit
Extremely
Extremely
20. Feeling tense or keyed up 27. Feeling weak in parts of your body
Not at all Not at all
A little bit A little bit
Some Some
Quite a bit Quite a bit
Extremely Extremely
21. Spells of terror or panic 28. Feeling blue
Not at all
Not at all
A little bit
A little bit
Some
Some
Quite a bit
Quite a bit
Extremely
Extremely
22. Feeling so restless you couldn’t sit still
29. Feeling lonely
Not at all
Not at all
A little bit
A little bit
Some
Some
Quite a bit
Quite a bit
Extremely
Extremely
23. Heavy feelings in arms or legs 30. Feeling no interest in things
Not at all Not at all
A little bit A little bit
Some Some
Quite a bit Quite a bit
Extremely Extremely
24. Feeling afraid to go out of your home alone 31. Feeling afraid in open spaces
or on the streets
Not at all
A little bit Not at all
Some A little bit
Quite a bit Some
Extremely Quite a bit
Extremely
02/16/2000....................................................................................................................................................................................................... Page 3 of 6
 32. How often can you tell when mental or 37. Getting angry about something never helps.
emotional problems are about to occur?
Strongly agree
Never Agree
Seldom/rarely Disagree
Sometimes Strongly disagree
Often
Always 38. I have a positive attitude toward myself.
33. When you can tell, how often can you Strongly agree
take care of the problems before they Agree
become worse? Disagree
Never Strongly disagree
Seldom/rarely
Sometimes 39. I am usually confident about the
Often decisions I make.
Always
Strongly agree
Part 4 Agree
Disagree
Below are several statements relating to
Strongly disagree
one’s view about life and having to make
decisions. Please check the response that 40. People have no right to get angry just
because they don’t like something.
is closest to how you feel about the
statement. Check the word or words that Strongly agree
Agree
best describes how you feel now.
Disagree
Strongly disagree
34. I can pretty much determine what will
happen in my life.
41. Most of the misfortunes in my life were due
Strongly agree to bad luck.
Agree
Strongly agree
Disagree
Agree
Strongly disagree
Disagree
Strongly disagree
35. People are limited only by what they think
is possible.
42. I see myself as a capable person.
Strongly agree
Strongly agree
Agree
Agree
Disagree
Disagree
Strongly disagree
Strongly disagree
36. People have more power if they join
together as a group. 43. Making waves never gets you anywhere.
Strongly agree Strongly agree
Agree Agree
Disagree Disagree
Strongly disagree Strongly disagree

02/16/2000....................................................................................................................................................................................................... Page 4 of 6
44. People working together can have an effect 51. I am able to do things as well as
on their community. most other people.
Strongly agree
Strongly agree
Agree
Agree
Disagree
Disagree
Strongly disagree
Strongly disagree
52. I generally accomplish what I set out to do.
45. I am often able to overcome barriers.
Strongly agree
Strongly agree
Agree
Agree
Disagree
Disagree
Strongly disagree
Strongly disagree

46. I am generally optimistic about the future. 53. People should try to live their lives the way
they want to.
Strongly agree
Agree Strongly agree
Disagree Agree
Strongly disagree Disagree
Strongly disagree
47. When I make plans, I am almost certain to
make them work. 54. You can’t fight city hall (authority).
Strongly agree
Strongly agree
Agree
Agree
Disagree
Disagree
Strongly disagree
Strongly disagree
55. I feel powerless most of the time.
48. Getting angry about something is often the
first step toward changing it. Strongly agree
Agree
Strongly agree
Disagree
Agree
Strongly disagree
Disagree
Strongly disagree
56. When I am unsure about something, I
usually go along with the rest of the
49. Usually I feel alone. group.
Strongly agree Strongly agree
Agree Agree
Disagree Disagree
Strongly disagree Strongly disagree

50. Experts are in the best position to decide 57. I feel I am a person of worth, at least
what people should do or learn. on an equal basis with others.

Strongly agree Strongly agree

Agree Agree
Disagree Disagree
Strongly disagree Strongly disagree

02/16/2000....................................................................................................................................................................................................... Page 5 of 6
58. People have a right to make their own 64. What is your marital status?
decisions, even if they are bad ones.
Never married
Strongly agree Married
Agree Separated
Disagree Divorced
Strongly disagree Widowed
Living together
59. I feel I have a number of good qualities.
65. What is your current living
Strongly agree
situation?
Agree
Disagree Your own house/apartment
Strongly disagree Friend’s home
Relative’s home
60. Very often a problem can be solved by Supervised group living
taking action.
Supervised apartment
Strongly agree Boarding home
Agree Crisis residential
Disagree Child foster care
Strongly disagree Adult foster care
Intermediate care facility
61. Working with others in my community can
Skilled nursing facility
help to change things for the better.
Respite care
Strongly agree MR intermediate care facility
Agree Licensed MR facility
Disagree State MR institution
Strongly disagree State MH institution
Hospital
Part 5 Correctional facility
Homeless
Please tell us some things about yourself. Rest home
Other
62. What was the last school grade you completed?
66. What is your employment
Less than 1st grade 10th grade
status?
1st grade 11th grade
2nd grade High school diploma/GED Employed full time
3rd grade Trade/Tech school Employed part time
4th grade Some college Sheltered employment
5th grade 2 yr college/Associate degree Unemployed
6th grade 4 yr college/Undergraduate degree Homemaker
7th grade Graduate school courses Retired
8th grade Graduate degree Disabled
9th grade Post-graduate studies Inmate of institution
Further special studies
67. Are you in treatment because
63. Race (check all that apply): you want to be?
Yes
White Hispanic/Latino
No
Native American/Pacific Islander Asian
Black/African-American Other Please stop here. Thanks!!

02/16/2000....................................................................................................................................................................................................... Page 6 of 6
 Ohio Mental Health Consumer Outcomes System
Adult Consumer Form B B
Today’s Date
Agency Use Only
Name
Client’s Medical Record Number:
Date of Birth

Gender (check one): Male Female

We are very interested in how you are doing, and how our services may or may not be helping you. Please
answer all of the questions below, then give the questionnaire to your case manager or another staff person
at the mental health agency.

4. How much money you have to

Part 1 spend for fun?
Below are some questions about how satisfied you Terrible
are with various aspects of your life in the past 6 Mostly dissatisfied
Equally satisfied/dissatisfied
months. For each question, checkmark the Mostly satisfied
answer that best describes how you feel. Very pleased

How do you feel about: 5. The amount of meaningful activity in

your life (such as work, school,
volunteer activity, leisure activity)?
1. The amount of friendship in your life?
Terrible
Terrible Mostly dissatisfied
Mostly dissatisfied Equally satisfied/dissatisfied
Equally satisfied/dissatisfied Mostly satisfied
Mostly satisfied Very pleased
Very pleased
6. The amount of freedom you have?
2. The amount of money you get?
Terrible Terrible
Mostly dissatisfied Mostly dissatisfied
Equally satisfied/dissatisfied Equally satisfied/dissatisfied
Mostly satisfied Mostly satisfied
Very pleased Very pleased

3. How comfortable and well-off you are financially? 7. The way you and your family act
Terrible toward each other?
Mostly dissatisfied
Terrible
Equally satisfied/dissatisfied
Mostly dissatisfied
Mostly satisfied Equally satisfied/dissatisfied
Very pleased Mostly satisfied
Very pleased
Does not apply
Please turn to the next page

02/16/2000....................................................................................................................................................................................................... Page 1 of 4
 8. Your personal safety? 14. Concerns about my medications (such
as side effects, dosage, type of
Terrible medication) are addressed:
Mostly dissatisfied
Equally satisfied/dissatisfied Never
Seldom/rarely
Mostly satisfied
Sometimes
Very pleased
Often
9. The neighborhood in which you live? Always
Not applicable/no medications
Terrible
Mostly dissatisfied
Equally satisfied/dissatisfied The next two items deal with how you have
Mostly satisfied been treated by other people.
Very pleased

10. Your housing/living arrangements? 15. I have been treated with dignity and
respect at this agency.
Terrible
Mostly dissatisfied Never
Equally satisfied/dissatisfied Seldom/rarely
Mostly satisfied Sometimes
Very pleased Often
Always
11. Your health in general?
16. How often do you feel worried by people’s
Terrible reactions to the problems that brought you
Mostly dissatisfied to the agency?
Equally satisfied/dissatisfied
Mostly satisfied Never
Seldom/rarely
Very pleased
Sometimes
12. How often do you have the opportunity to Often
spend time with people you really like? Always

Never
Seldom/rarely Part 3
Sometimes The following questions ask you about how
Often
much you were distressed or bothered by
Always
some things during the last seven days.
Part 2 Please mark the answer that best describes

These next few items ask you about your health how you feel.

and medications within the past 6 months.

During the past 7 days, about how much
were you distressed or bothered by:
13. How often does your physical condition
interfere with your day-to-day functioning? 17. Nervousness or shakiness inside
Never
Not at all
Seldom/rarely
A little bit
Sometimes
Some
Often
Quite a bit
Always
Extremely

02/16/2000....................................................................................................................................................................................................... Page 2 of 4
18. Being suddenly scared for no reason 25. Feeling of worthlessness
Not at all Not at all
A little bit A little bit
Some Some
Quite a bit Quite a bit
Extremely Extremely

26. Feeling lonely even when you are

19. Feeling fearful with people
Not at all
Not at all
A little bit
A little bit
Some
Some
Quite a bit
Quite a bit
Extremely
Extremely
20. Feeling tense or keyed up 27. Feeling weak in parts of your body
Not at all Not at all
A little bit A little bit
Some Some
Quite a bit Quite a bit
Extremely Extremely
21. Spells of terror or panic 28. Feeling blue
Not at all
Not at all
A little bit
A little bit
Some
Some
Quite a bit
Quite a bit
Extremely
Extremely
22. Feeling so restless you couldn’t sit still
29. Feeling lonely
Not at all
Not at all
A little bit
A little bit
Some
Some
Quite a bit
Quite a bit
Extremely
Extremely
23. Heavy feelings in arms or legs 30. Feeling no interest in things
Not at all Not at all
A little bit A little bit
Some Some
Quite a bit Quite a bit
Extremely Extremely
24. Feeling afraid to go out of your home alone 31. Feeling afraid in open spaces
or on the streets
Not at all
A little bit Not at all
Some A little bit
Quite a bit Some
Extremely Quite a bit
Extremely
02/16/2000....................................................................................................................................................................................................... Page 3 of 4
 32. How often can you tell when mental or 36. What is your marital status?
emotional problems are about to occur?
Never married
Never Married
Seldom/rarely Separated
Sometimes Divorced
Often Widowed
Always Living together

37. What is your current living

situation?
33. When you can tell, how often can you
take care of the problems before they Your own house/apartment
become worse? Friend’s home
Never Relative’s home
Seldom/rarely Supervised group living
Sometimes Supervised apartment
Often Boarding home
Always Crisis residential
Child foster care
Adult foster care
Intermediate care facility
Part 4 Skilled nursing facility
Please tell us some things about yourself. Respite care
MR intermediate care facility
Licensed MR facility
State MR institution
34. What was the last school grade you completed? State MH institution
Less than 1st grade 10th grade Hospital
1st grade 11th grade Correctional facility
2nd grade Homeless
High school diploma/GED
3rd grade Rest home
Trade/Tech school
4th grade Some college Other
5th grade 2 yr college/Associate degree
6th grade 4 yr college/Undergraduate degree 38. What is your employment
7th grade Graduate school courses status?
8th grade Graduate degree Employed full time
9th grade Post-graduate studies Employed part time
Further special studies Sheltered employment
Unemployed
Homemaker
Retired
35. Race (check all that apply):
Disabled
White Hispanic/Latino Inmate of institution
Native American/Pacific Islander Asian
Black/African-American Other 39. Are you in treatment because
you want to be?
Yes
No
Please stop here. Thanks!!
02/16/2000....................................................................................................................................................................................................... Page 4 of 4
 Ohio Mental Health Consumer Outcomes System
Provider Form A P
Client’s Name Today’s Date

Client’s Medical Record Number

Please circle the appropriate response for each statement that corresponds with the client’s highest
level of functioning in the past 6 months.

1. Does the client initiate non-professional social contact or respond to others’ initiation of social
contact?

Withdrawn/isolated Minimal contact Moderate contact Optimal contact Unsure

2. How effectively does this client interact with others? NOTE: “Effectively” refers to how
successfully and appropriately the client behaves in social settings (i.e., how well she/he
minimizes interpersonal friction, meets personal needs, achieves personal goals in socially
appropriate manner).

Very ineffectively Ineffectively Mixed or dubious Effectively Very Effectively Unsure

effectiveness

3. How effective is the client’s social support network in helping the client meet his/her needs?
NOTE: A support network may consist of interested family, friends, acquaintances, coworkers,
peers, or social clubs, etc.

Very ineffective Ineffective Mixed or dubious Effective Very Effective Unsure

effectiveness

4. Please rate the client’s housing stability

Moved very frequently Moved often Moved a few times Moved once Did not move Unsure

5. Has the client been forced/compelled to move from his/her living arrangements?

Yes No Unsure

02/16/2000....................................................................................................................................................................................................... Page 1 of 3
6. How well does the client perform independently in the following day-to-day living activities?

Task is not Someone other Client needs extensive Client needs some Client acts Unsure
completed than the client supervision or supervision or independently or not
completes task assistance assistance applicable

A. Personal hygiene 1 2 3 4 5 ?
B. Dressing appropriately 1 2 3 4 5 ?
C. Obtaining regular nutrition 1 2 3 4 5 ?
D. Using public transportation 1 2 3 4 5 ?
E. Shopping 1 2 3 4 5 ?
F. Doing laundry 1 2 3 4 5 ?
G. Housekeeping 1 2 3 4 5 ?
H. Managing money 1 2 3 4 5 ?

7. To what extent has the client engaged in the following meaningful activities?

Almost Seldom Sometimes Often Almost always Unsure

Never (<1x / week) (1-2x / week) (3-4x / week) (>5x / week) or not
(<1x / mo.) applicable

A. Work 1 2 3 4 5 ?
B. School 1 2 3 4 5 ?
C. Volunteer activity 1 2 3 4 5 ?
D. Parenting 1 2 3 4 5 ?
E. Homemaking 1 2 3 4 5 ?
F. Leisure activity 1 2 3 4 5 ?

8. Of the roles listed above, in general how well is the client performing in his/her primary role?

Extremely poorly Poorly Satisfactorily Well Extremely well Unsure

9. How frequently is the client’s functioning compromised by addictive or compulsive behaviors

(e.g., alcohol abuse, drug abuse, gambling)?

Almost always Often Sometimes Seldom Almost never Unsure

(>5x / week) (3-4x / week) (1-2x / week) (<1x / week) (<1x / month)

02/16/2000....................................................................................................................................................................................................... Page 2 of 3
10. Has the client abided by the law sufficiently to avoid incarceration and/or criminal justice system
involvement?

No Yes Unsure

11. Has the client attempted to or actually physically harmed someone?

Yes No Unsure

12. Has the client been a victim of:

a) rape Yes No Unsure

b) assault Yes No Unsure

c) threats Yes No Unsure

d) exploitation Yes No Unsure

e) harassment Yes No Unsure

f) suicide attempt Yes No Unsure

g) other type of harm to self Yes No Unsure

h) hate crimes Yes No Unsure

i) theft, robbery, vandalism Yes No Unsure

Thank you!!

02/16/2000....................................................................................................................................................................................................... Page 3 of 3
 The Evaluation Center @ HSRI

THE PEER OUTCOMES PROTOCOL (POP)

Campbell, J., Cook, J.A., Jonikas, J.A., & Einspahr, K. 2004

For more information contact:

All components of the instrument (Administration Manual; A Question-by-Question Guide, Survey
Instrument; Response Cards, and Psychometric Report) are available for free download: http://www. H

psych.uic.edu/uicnrtc/popmanual.htm. A paper copy of the materials can be obtained for $20 from the
H

UIC Center.

UIC National Research and Training Center on Psychiatric Disability

Attention: Dissemination Coordinator
104 South Michigan Avenue, Suite 900
Chicago, IL 60603
Phone: 312-422-8180

Questions about the POP can be directed to its first author.

Jean Campbell, Ph.D.
Missouri Institute of Mental Health
Email: Jean.Campbell@mimh.edu
H H

Phone: 314-877-6457

Permission to use:

The POP is copyrighted by the University of Illinois at Chicago, 2004. People with psychiatric disabilities
and non-profit, mental health consumer-run programs/organizations may reproduce and use the research
protocol and documentation for their own personal use without permission. The authors would
appreciate being acknowledged in such instances. All other rights are reserved and written permission
must be obtained from the UIC Center. There are no user fees associated with the POP.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 150

 Peer Outcomes Protocol Project

Peer Outcomes Protocol (POP):

Questionnaire

Prepared by:

Jean Campbell, Ph.D.

Missouri Institute of Mental Health, University of Missouri-Columbia
Judith A. Cook, Ph.D.
UIC National Research & Training Center on Psychiatric Disability
Jessica A. Jonikas, M.A.
UIC National Research & Training Center on Psychiatric Disability
Kimberlee Einspahr, Ph.D.
Independent Consultant

This protocol was developed as a project of the University of Illinois at Chicago, National Research
and Training Center on Psychiatric Disability, directed by Judith A. Cook, Ph.D. The Center is
supported by the National Institute on Disability and Rehabilitation Research, U.S. Department of
Education, and the Center for Mental Health Services, Substance Abuse and Mental Health
Services Administration (Cooperative Agreement #H133B000700). The opinions expressed herein
do not necessarily reflect the position, policy, or views of either agency, and no official
endorsement should be inferred.
©Copyright, University of Illnois at Chicago
2004

The authors would like to acknowledge the POP Consumer/Survivor Advisory Board
members, who provided input on multiple drafts of the Protocol, as well as Richard
Evenson, Ph.D., Research Professor Emeritus at the Missouri Institute of Mental
Health, who analyzed the psychometric properties of the instrument. Also invaluable
was the assistance of Diane O’Rourke, MA, of the UIC Survey Research Laboratory,
who helped prepare the final versions of the Protocol and its companion pieces.

To contact Jean Campbell, Ph.D.:

Missouri Institute of Mental Health

University of Missouri-Columbia
5400 Arsenal Street
St. Louis, MO 63139
Phone: (314) 644-7829
FAX: (314) 644-7934
www.mimh.edu/mimh

To contact Judith A. Cook, Ph.D. and Jessica A. Jonikas, M.A., and/or receive additional
copies of the Protocol:

The UIC National Research and Training Center on Psychiatric Disability

104 South Michigan Avenue, Suite 900
Chicago, Illinois 60603
Phone: (312) 422-8180
FAX: (312) 422-0740
TDD: (312) 422-0706
www.psych.uic.edu/uicnrtc
 PEER OUTCOMES PROTOCOL
QUESTIONNAIRE

Respondent ID:___ ___ ___ ___ ___ ___ ___ ___ ___ ___

Site ID: ___ ___

Interviewer Number: ___ ___

Informed Consent Signed ___________________________(Interviewer Signature)

Date
___/___/__ Interview Session I: ___ ___ ___ minutes Start Time___ ___ End Time ___ ___
Date
Interview Session II: ___ ___ ___ minutes Start Time___ ___ End Time ___ ___
___/___/__
Date
Interview Session ___ ___ ___ minutes Start Time___ ___ End Time ___ ___
___/___/__
III:
Date
Interview Session ___ ___ ___ minutes Start Time___ ___ End Time ___ ___
___/___/__
IV:

Total Interview Time: ______hours and ______minutes

 4

POP Questionnaire
Table Of Contents
Interviewer Instructions to the Respondent ................................................................................5
Demographics Module...................................................................................................................6
Service Use Module......................................................................................................................11
Employment Module ...................................................................................................................17
Employment Satisfaction Scale ..........................................................................................18
Community Life Module .............................................................................................................19
Community Satisfaction Scale............................................................................................19
Quality of Life Scale (A. Lehman)......................................................................................23
Social Satisfaction Scale)...................................................................................................25
Discrimination Scale..........................................................................................................26
Crime Demographics .........................................................................................................27
Social Acceptance Scale ( J. Campbell & R. Schriaberl) ..................................................28
Quality of Life Module ................................................................................................................29
SF-12 Health Survey (J. Ware)..........................................................................................29
Quality of Life Scale (A. Lehman)......................................................................................31
Subjective Quality of Life...................................................................................................31
Program Quality of Life.....................................................................................................32
Well-Being Module ......................................................................................................................33
Personhood and Empowerment Scale ...............................................................................33
Recovery Scale ...................................................................................................................35
Program Satisfaction Module .....................................................................................................36
Program Satisfaction Scale................................................................................................36
Coercion Scale (J. Campbell; V. Wieselthier; K. Einspahr, & R. Evenson) .....................40
Conclusion ................................................................................................................................... 41

Peer Outcomes Protocol – Table of Contents

 5

Interviewer Instructions to the Respondent

The purpose of this interview is to learn more about how you feel about this peer
support program. I will ask you questions about yourself, such as your age and
where you live, as well as questions about how you are feeling right now, and your
opinions about this program. Some of the questions will ask you to give me some
detailed information. Other questions will ask how much you agree or disagree
with a statement. For some of the questions, I will show you a card with a list of
possible responses and ask you to select the one that best reflects how you feel. I
will write down your answers for each question, so we can combine your answers
with those given by other people to get an overall view of how this program is
doing.

Before we start, I’d like to remind you that your answers to these questions are
private. No one outside of the research staff will know about your answers to these
questions. Also, your participation is voluntary. That means you do not have to
answer any questions you do not want to answer. The interview takes about an
hour. If you need a break or want to stop, please let me know.

Peer Outcomes Protocol – Interviewer Instructions

 6

DEMOGRAPHICS MODULE
First, we would like to ask you some general questions about yourself.
Interviewer: Circle gender if known. Ask question only if necessary.

1. What is your gender?

Male .......................................1
Female....................................2

2. What is your date of birth? ______ month ______ day ______ year

3. Are you of Spanish or Hispanic origin? Please tell me the group or groups that represent
your national origin or ancestry. [Circle all that apply]

[Hand respondent response card 1]

No, not Spanish/Hispanic.......................................................1
Yes, Mexican, Mexican-American, Chicano.........................2
Yes, Puerto Rican...................................................................3
Yes, Cuban.............................................................................4
Yes, other Spanish/Hispanic ..................................................5
Please specify ____________________________________

Peer Outcomes Protocol - Demographics Module

 7

What ethnicity do you consider yourself to be? Please tell me the group or groups which
represent your race. [Circle all that apply]

[Hand respondent response card 2]

White..........................................................................1
Black or African American........................................2
American Indian/Native American ............................3
Eskimo .......................................................................4
Aleut...........................................................................5
Asian or Pacific Islander (API)
(including East Indian)............................................6
If Asian or API Ask:
Chinese...........................................................7
Japanese .........................................................8
Filipino...........................................................9
Asian Indian ...................................................10
Hawaiian ........................................................11
Samoan...........................................................12
Korean............................................................13
Guamanian .....................................................14
Vietnamese.....................................................15
Other API ......................................................16
Please specify ________________________
Other race...................................................................17
Please specify ______________________________

4. In what country were you born?

United States, including Puerto Rico .........................1 [Skip to Question 7]
Other ..........................................................................2 [Continue to Question 6]
Please specify ______________________________

Peer Outcomes Protocol - Demographics Module

 8

5. In what year did you come to the U.S. to stay?

(If came to stay more than once, ask): In what year did you come to the U.S. to stay the
last time? ______ (Year)

6. Is English your primary language?

Yes .............................................................................1 [Skip to Question 9]
No...............................................................................2

7. What is your primary language? ____________________

8. What is your current marital status?

[Hand respondent response card 3]

Now married ..............................................................1 [Skip to Question 11]
Widowed ....................................................................2
Divorced.....................................................................3
Separated....................................................................4
Never married ............................................................5

9. Are you living in a committed relationship, but not married? By committed relationship, I
mean sharing your life and housing with a partner?
Yes .............................................................................1
No...............................................................................2

10. Which of the following describes your sexual orientation?

Interviewer: Read list to respondent and circle category of response.
Heterosexual. By heterosexual, I mean, “straight” ...........................1
Gay male ............................................................................................2
Lesbian female ...................................................................................3
Bisexual. by “bisexual,” I mean both straight and lesbian or gay. ...4

11. (If female): How many children have you given birth to?
(If male): How many children have you fathered?
Interviewer: If none, write “0”.

______(number of children)

Peer Outcomes Protocol - Demographics Module

 9

12. How many children under the age of 18 live with you at least four days per week?
Interviewer: If none, write “0” and skip to Question 15.

______(number of children)

13. Are you a single parent? By “single parent,” I mean that you are the only adult living in the
household and all other people who live with you are under the age of 18.
Yes .............................................................................1
No...............................................................................2

14. Are you a veteran? By “veteran,” I mean, did you serve in the armed forces?
Yes .............................................................................1
No...............................................................................2

15. Do you have a physical and/or sensory disability? By “physical or sensory disability,” I
mean one that is not caused by a psychiatric disability.

Yes .............................................................................1
No...............................................................................2

16. What is the highest grade in school that you have completed?

[Hand respondent response card 4]

No formal schooling ..................................................1
Up to 8th grade............................................................2
Some high school.......................................................3
High school diploma/GED.........................................4
Some college or post-high school training.................5
2-year Associate degree .............................................6
4-year college degree .................................................7
Post-college graduate training....................................8

Peer Outcomes Protocol - Demographics Module

 10

17. In the past 30 days, have you had any financial support from the following sources? [Read
the list to the respondent and circle all that apply]

[Hand respondent response card 5]

Earned income .......................................................................1

Social Security Benefits (SSA)..............................................2
Social Security Disability Income (SSDI) .............................3
Supplemental Security Income (SSI).....................................4
Armed Service connected disability payments ......................5
Other Social Welfare benefits----state or county
(TANF, Aid to Families with Dependent Children) ....................6

Vocational program
(Comprehensive Employment and Training Act (CETA),
Vocational Rehabilitation, sheltered workshop, Goodwill)........... 7

Unemployment compensation ...............................................8

Retirement, investment or savings income ............................9
Rent supplements (including HUD, section 8 certificates,
living programs receiving public support) ................................10

Alimony or child support .......................................................11

Food stamps ...........................................................................12
Family and/or spouse contribution.........................................13
Other sources(s): ....................................................................14
Please Specify: ___________________________________

18. How much money did you receive during the past 30 days from all of these sources?
_____________________

19. What was your total personal income last year?

$________________

20. On the average, how much money do you have to spend on yourself each month, not counting
money for room and meals?

$________________

Peer Outcomes Protocol - Demographics Module

 11

SERVICE USE MODULE

Now I would like to ask you a few questions related to your status as a mental health consumer/survivor.

1. Have you been diagnosed with a major mental illness?

Yes .............................................................................1 [Continue to Question 2]
No...............................................................................2 [Skip to Question 3]
Not sure......................................................................3 [Continue to Question 2]

2. What have you been told is your psychiatric diagnosis? From the list, please pick all
diagnoses that you have been told, or tell me any other diagnosis that may not be on the list.
Interviewer: Read question and instruction and circle all the categories that apply or write
in exactly what the respondent says.

[Hand respondent response card 6]

Schizophrenia.....................................................................................1
Schizoaffective Disorder....................................................................2
Manic Depression, Bipolar, or Affective Disorder............................3
Major Depression...............................................................................4
Anxiety Disorder
(such as Panic Disorder, Obsessive Compulsive Disorder, etc.) ................ 5

Dissociative Disorder
(such as Multiple Personality, Dissociative Amnesia, etc.)......................... 6

Personality Disorder...........................................................................7
Substance Abuse ................................................................................8
Other ..................................................................................................9
Please Specify __________________________________________

Peer Outcomes Protocol – Service Use Module

 12

3. Are you currently taking any psychiatric medications?

Yes .............................................................................1
No...............................................................................2 [Skip to Question 5]

4. In the past 30 days, have you been bothered by any side effects from the psychiatric
medications you have taken?

[Hand respondent response card 7]

1 2 3 4
No side effects Mild side effects Moderate side Severe side effects
effects

5. Have you had any problems associated with alcohol use in your lifetime?
Yes .............................................................................1
No...............................................................................2

6. Have you had any problems associated with drug use in your lifetime?
Yes .............................................................................1
No...............................................................................2

7. Were you ever physically abused as a child?

Yes .............................................................................1
No...............................................................................2

8. Were you ever sexually abused as a child?

Yes .............................................................................1
No...............................................................................2

9. Have you ever been hospitalized for psychiatric reasons?

Yes .............................................................................1
No...............................................................................2 [Skip to Question 13]

Peer Outcomes Protocol – Service Use Module

 13

10. How old were you at your first psychiatric hospitalization?

______ (years of age at first hospitalization)

11. About how many times have you been hospitalized for psychiatric reasons in your lifetime?

________ (number of psychiatric hospitalizations)

12. About how many times have you been hospitalized for psychiatric reasons during the past 12
months?

________ (number of psychiatric hospitalizations)

Next, I would like to know about some of the services you have used in the past.

13. How long have you been attending this peer support program? If you can, please tell me the
date of when you first started coming here.

_____ Month _____ Day _____ Year [Code as MM/DD/YYYY]

(approximate)

14. During a typical week, how often do you attend this peer support program?

[Hand respondent response card 8]

1 2 3 4 5
Almost 2 or more times a About once a week About once a A few times
every Day week month a year

Peer Outcomes Protocol – Service Use Module

 14

15. I have a list of services that are available in the community. For each one please tell me if
you have received the service in the past 30 days. If you have, tell me if you received the
service at this peer support program, somewhere else, or here and somewhere else.
Interviewer: Read list to the respondent and circle all that apply.

[Hand respondent response card 9]

Community Services At Peer Support Outside of Peer At Peer Program
Program Program & Elsewhere

Self-Help Group, such as AA, NA, 1 2 3

DMDA, a sexual abuse survivors group

Medication management by a psychiatrist 1 2 3

or doctor
Therapy 1 2 3
Counseling 1 2 3
Case Management 1 2 3
Crisis Hotline 1 2 3
Crisis Intervention Service 1 2 3
Supervised or Supported Living Program 1 2 3
Drop-In Center 1 2 3

Homeless Shelter 1 2 3
Domestic Violence Shelter/Program 1 2 3

Legal Aid 1 2 3

Job Training or Vocational Program 1 2 3

Partial or day hospitalization services 1 2 3

Services for alcohol use or abuse 1 2 3

problems
Services for drug use or abuse problems 1 2 3

Alternative therapy or treatment, such as 1 2 3

body massage, herbs/homeopathic
Other, describe: 1 2 3

Peer Outcomes Protocol – Service Use Module

 15

In this section, I would like to know about recent psychiatric problems and hospitalizations you
may have had, and about you experiences with peer support during these times. This
information is strictly confidential.

16. Have you had any significant emotional difficulties in the past six months?
Yes .............................................................................1
No...............................................................................2 [SKIP to Question 24]

17. Do you feel that this program helped prevent these difficulties from turning into a psychiatric
crisis during the past 6 months?
Yes .............................................................................1
No...............................................................................2

18. Do you feel that this program helped you stay out of the hospital during the past 6 months?
Yes .............................................................................1
No...............................................................................2 [SKIP to Question 24]

I’d like to read a list of ways this program might have helped you stay out of the hospital. As I
read each one, tell me whether it was true or false for you:

True False
19. Did the program help you stay out of the hospital by offering 1 2
you another place to stay?
20. Did the program help you stay out of the hospital by 1 2
providing support whenever you needed it?
21. Did the program help you stay out of the hospital by giving 1 2
you someone to talk to?
22. Did the program help you stay out of the hospital by helping 1 2
you cope with symptoms?
23. Did the program help you stay out of the hospital by 1 2
involving other people in your life?

Peer Outcomes Protocol – Service Use Module

 16

Next, I’d like to ask you about any recent hospitalizations you may have had.

24. In the past 6 months, have you been hospitalized for psychiatric problems?
Yes .............................................................................1
No...............................................................................2 [SKIP to Next Module]

25. In the past 6 months, how many times were you in a psychiatric hospital?
[If none write "0"]
__________ # of times

26. In the past 6 months, how many of your hospitalizations were involuntary?
__________ # of times

27. In the past 6 months, approximately how many total days were you hospitalized for
psychiatric reasons? __________ # of days

28. Were you in this program at the time of your most recent hospitalization?

Yes .............................................................................1 [Continue to Question 29]

No...............................................................................2 [Skip to Next Module]

I’d like to know how much you agree or disagree with the following statements about this most
recent hospitalization.

[Hand respondent response card 10] 1 2 3 4

Disagree Somewhat Somewhat Agree
Disagree Agree
29. People from this program supported me 1 2 3 4
while I was in the hospital.
30. People from this program ignored me 1 2 3 4
while I was in the hospital.
31. People from this program visited me 1 2 3 4
while I was in the hospital.***
32. People from this program made me feel 1 2 3 4
like a failure for being in the
hospital.***

Peer Outcomes Protocol – Service Use Module

 17

EMPLOYMENT MODULE
In this section, I would like to ask about your work activities.

1. Are you currently working for pay?

Yes .............................................................................1
No...............................................................................2 [Skip to Question 6]

2. How many hours per week do you work?

_________(number of hours)

3. Some people have more than one paid job. How many paid jobs do you have?
_________(number of jobs)

4. What is your current hourly wage? Pick the highest hourly wage if you have more than one
job.
_________(dollars per hour)

5. Does your job offer health insurance to you?

Yes .............................................................................1 [Skip to Question 9]
No...............................................................................2 [Skip to Question 9]

[Interviewer: Ask question and circle response]

Yes No
6. Are you currently interested in working? 1 2
7. Have you been looking for work during the last 4 1 2
weeks?
8. Do you remain at home to care for others? 1 2
9. Have you attended school or a training program in the 1 2
past 6 months?
10. Are you retired? 1 2
11. Do you do any volunteer work or any other kind of 1 2
work for which you are not paid?

Peer Outcomes Protocol – Employment Module

 18

Now I would like to ask you about how you feel about work, whether or not you are employed.
Please tell me how much you agree or disagree with the following statements.
Interviewer: Show respondent card, read the instruction and question, and circle number
of response.

[Hand respondent response card 11]

Disagree Somewhat Somewhat Agree
Disagree Agree
12. In general, I am satisfied with my 1 2 3 4
employment status right now.

13. If I am having emotional problems, I am able 1 2 3 4

to put them aside when I work.

14. I know how to get a job. 1 2 3 4

15. I know how to keep a job once I am hired. 1 2 3 4

16. This peer support program has helped me to 1 2 3 4

improve my work situation.
17. I feel comfortable talking to people in this 1 2 3 4
program about losing SSI or SSDI as a result
of returning to work.
18. I feel comfortable talking to people in this 1 2 3 4
program about losing Medicaid or Medicare
as a result of returning to work.
19. This program inspires me to believe that 1 2 3 4
meaningful work is possible for me.
20. This program does not have enough resources 1 2 3 4
to help program members find jobs.
21. This program does not have enough resources 1 2 3 4
to help program members keep jobs.

22. I am comfortable discussing work issues with 1 2 3 4

my peers in this program.

Interviewer: If respondent is not working, ask: 1 2 3 4

23. I am not working, but I would like to be
working.

Interviewer: If respondent is working, ask: 1 2 3 4

24. I am working at a job that I want.

Peer Outcomes Protocol – Employment Module

 19

COMMUNITY LIFE MODULE

I would like to ask some questions about your housing situation and community life.

1. Where do you currently live?

[Hand respondent response card 12]

Apartment, condo, house, or trailer .......................................1

Transitional living center or half-way house .........................2 [SKIP to Question 3]

Group home/board and care...................................................3 [SKIP to Question 3]

Shelter ....................................................................................4 [SKIP to Question 3]

Hotel or Motel........................................................................5 [SKIP to Question 3]

Street ......................................................................................6 [SKIP to Question 3]

Other .....................................................................................7 [SKIP to Question 3]

Please Specify _______________________________

2. Is this apartment, condo, house, or trailer ….

Rented for cash?................................................................... 1

Occupied without payment of cash rent?............................. 2

Owned by you with a mortgage or loan? ............................. 3

Owned by you free and clear (without a mortgage)?........... 4

Peer Outcomes Protocol – Community Life Module

 20

3. Who currently lives in your residence with you? [Circle all that apply]

[Hand respondent response card 13]

Parents....................................................................................1

Spouse or partner ...................................................................2

Friends....................................................................................3

Other peers .............................................................................4

Minor children .......................................................................5

Adult children ........................................................................6

No one (respondent lives alone) ............................................7

Other .....................................................................................8
Please Specify _____________________________________

4. Do you receive any help in managing your money?

Yes .........................................................................................1
No...........................................................................................2 [SKIP to Question 6]

5. From whom do you receive help? [Circle all that apply]

[Hand respondent response card 14]

People at this peer program....................................................1

Staff from another program ...................................................2

Family ....................................................................................3

Friends....................................................................................4

Spouse or Partner ...................................................................5

Other .....................................................................................6
Please Specify _____________________________________

Peer Outcomes Protocol – Community Life Module

 21

6. Do you receive any help with cooking?

Yes .........................................................................................1
No...........................................................................................2 [SKIP to Question 8]

7. From whom do you receive help? [Circle all that apply]

[Hand respondent back response card 14]

People at this peer program....................................................1

Staff from another program ...................................................2

Family ....................................................................................3

Friends....................................................................................4

Spouse or Partner ...................................................................5

Other .....................................................................................6
Please Specify _____________________________________

8. Do you receive any help with housekeeping?

Yes .........................................................................................1
No...........................................................................................2 [SKIP to Question 10]

9. From whom do you receive help? [Circle all that apply]

[Hand respondent back response card 14]

People at this peer program....................................................1

Staff from another program ...................................................2

Family ....................................................................................3

Friends....................................................................................4

Spouse or Partner ...................................................................5

Other .....................................................................................6
Please Specify _____________________________________

Peer Outcomes Protocol – Community Life Module

 22

Now I would like to ask you about how you feel about your current living situation and the neighborhood in
which you live. Please tell me how much you agree or disagree with the following statements.

[Hand respondent response card 15]

Disagree Somewhat Somewhat Agree
Disagree Agree
10. In general, I am satisfied with the 1 2 3 4
neighborhood in which I live.

11. I live in this neighborhood because I want 1 2 3 4

to.

12. I live in this kind of housing because I want 1 2 3 4

to.

13. I am involved in neighborhood activities, 1 2 3 4

such as volunteer work, religious groups,
sports, or recreation activities, that are not
related to being a mental health consumer.

14. I feel rejected by people in my 1 2 3 4

neighborhood because I am diagnosed with
mental illness.***

15. I feel this program helps people find better 1 2 3 4

housing.

Peer Outcomes Protocol – Community Life Module

 23

In this section, I would like to know about the people in your life and how you feel about your
social relationships.

[Hand respondent response card 16]

Not at Once 2-3 times 4-6 Once a day
all times or more

16. During the past 7 days, how often did you 1 2 3 4 5

spend time with friends or family in
recreational activities? This does not
include mental health system sponsored
activities or activities at the peer support
program.
17. How often did you spend time alone in 1 2 3 4 5
recreational activities during the past 7 days?
18. During the past 7 days, how often did you 1 2 3 4 5
go to clubs, church, or other meetings in your
community? This does not include mental
health system sponsored activities or activities
at the peer support program.
19. During the past 7 days, how often did you 1 2 3 4 5
spend time with friends in recreational
activities at this peer support program?

Interviewer: Show respondent card, read the questions, and circle number of response.

[Hand respondent response card 17]

Not Less than At least At least At least
at all once a once a once a once a
month month week day
20. About how often do you visit with someone 1 2 3 4 5
who does not live with you?
21. About how often do you telephone someone 1 2 3 4 5
who does not live with you?
22. About how often do you do something with 1 2 3 4 5
another person that you planned ahead of
time?
23. About how often do you spend time with 1 2 3 4 5
someone you consider more than a friend,
like a boyfriend or girlfriend?

Peer Outcomes Protocol – Community Life Module

 24

Now I would like to know how you feel about the things you do with other people. Please look
at this card. This is called the Delighted-Terrible Scale. The scale goes from terrible which is the
lowest ranking of 1, to delighted, which is the highest ranking of 7. There are also points 2
through 6 with descriptions about them. For the next three questions, please tell me what on the
scale best describes how you feel.

24. How do you feel about the things you do with other people?

[Hand respondent response card 18]

1 2 3 4 5 6 7
Terrible Unhappy Mostly Mixed Mostly Pleased Delighted
Dissatisfied Satisfied

25. How do you feel about the amount of time you spend with other people?

1 2 3 4 5 6 7
Terrible Unhappy Mostly Mixed Mostly Pleased Delighted
Dissatisfied Satisfied

26. How do you feel about the people you see socially?

1 2 3 4 5 6 7
Terrible Unhappy Mostly Mixed Mostly Pleased Delighted
Dissatisfied Satisfied

Peer Outcomes Protocol – Community Life Module

 25

The following section is about your social relationships. Please tell me how much you agree or
disagree with the following statements.

[Hand respondent response card 19]

Disagree Somewhat Somewhat Agree
Disagree Agree
27. The social relationships that I have with
neighbors are what I want them to be. 1 2 3 4

28. The social relationships that I have with

my family members are what I want
1 2 3 4
them to be.

29. The social relationships that I have with

my friends are what I want them to be. 1 2 3 4

30. The social relationships that I have with

my peers in this program are what I want
1 2 3 4
them to be.

31. I often feel lonely. ***

1 2 3 4

32. I lack intimacy in my everyday life. ***

1 2 3 4

Peer Outcomes Protocol – Community Life Module

 26

In the next set of questions, I am going to ask you about discrimination. Discrimination means
that you are denied your rights to freedom of speech, or equal access, or equal opportunity
because you are of a particular gender, or race, or sexual orientation, or have a mental or physical
disability.

33. Have you ever experienced discrimination in this program?

Yes .........................................................................................1
No...........................................................................................2

Now I’d like you to tell me how much you agree or disagree with the following statements.

[Hand respondent back response card 19]

Disagree Somewhat Somewhat Agree

Disagree Agree

34. I know what to do if I experience

1 2 3 4
discrimination from staff at this program.

35. I know what to do if I experience

1 2 3 4
discrimination in the workplace.

36. I know what to do if I experience

1 2 3 4
discrimination from my landlord.

Peer Outcomes Protocol – Community Life Module

 27

Now I am going to ask a few questions about crime and violence in your life.

37. Have you been the victim of a violent crime, such as assault, robbery, rape, or abuse, in the
past six months, whether it was reported or not reported?
Yes .........................................................................................1
No...........................................................................................2

38. Have you been a victim of a nonviolent crime, such as theft, in the past six months, whether
it was reported or not reported?
Yes .........................................................................................1
No...........................................................................................2

39. Have you been arrested in the past six months?

Yes .........................................................................................1
No...........................................................................................2

40. Have you been in jail or prison in the past six months?
Yes .........................................................................................1
No...........................................................................................2

Interviewer: For the next two questions, read the question, and check the response. If the
respondent is experiencing physical or sexual abuse as reported in Q41 and Q42, stop the
interview and (1) give the person a list of the local services and support groups, and (2) ask
the respondent if they need assistance in making contact with a service or support group,
then (3) proceed with the interview.

41. Are you experiencing physical abuse in your life?

Yes .........................................................................................1 [See above instruction]
No...........................................................................................2

42. Are you experiencing sexual abuse in your life?

Yes .........................................................................................1 [See above instruction]
No...........................................................................................2

Peer Outcomes Protocol – Community Life Module

 28

SOCIAL ACCEPTANCE

[Hand respondent response card 20]

Most of the Seldom
No
Time Sometimes or Never
Opinion
Rarely

43. How often do people treat you differently 1 2 3 4 5

when they know you have a mental
diagnosis or have received mental health
services?

As an individual who has received mental health services, how often do you think others . . .

[Hand respondent response card 21]

All of the Most of the No
Sometimes Seldom Never
time time Opinion
44. …feel or treat you like you are
violent or dangerous. 1 2 3 4 5 6

45. ... feel you are a child or treat you

like a child?” 1 2 3 4 5 6

46. …feel or treat you like you are

unpredictable? 1 2 3 4 5 6

47. ....think that you do not know what

is in your own best interests? 1 2 3 4 5 6

48. ....think or treat you like you are

incapable of caring for children? 1 2 3 4 5 6

49. ....think or treat you like you are

incapable of holding a job? 1 2 3 4 5 6

50. …feel or treat you like you are

incapable of having a satisfying
relationship with another man or 1 2 3 4 5 6
woman?

Peer Outcomes Protocol – Community Life Module

 29

QUALITY OF LIFE MODULE

Now I would like to ask you a few questions about your health. If you are unsure about how to
answer, please give the best answer you can.

1. In general, would you say your health is

[Hand respondent response card 22]

5 4 3 2 1
excellent very good good fair poor

The following items are about activities you might do during a typical day. Does your health
now limit you in these activities? If so, how much?

2. Moderate activities, such as moving a table, pushing a vacuum cleaner, bowling, or playing
golf.

[Hand respondent response card 23]

1 2 3
Yes, limited a Yes, limited a No, not limited
lot little at all

3. Climbing several flights of stairs.

[Hand respondent back response card 23]

1 2 3
Yes, limited a Yes, limited a No, not limited
lot little at all

During the past 4 weeks, have you had any of the following problems with your work or other
regular daily activities as a result of your physical health?

4. Accomplished less than you would like

Yes .........................................................................................1
No...........................................................................................2

5. Were limited in the kind of work or other activities

Yes .........................................................................................1
No...........................................................................................2

Peer Outcomes Protocol – Quality of Life Module

 30

During the past 4 weeks have you had any of the following problems with your work or other regular
daily activities as a result of any emotional problems, such as feeling depressed or anxious?

6. Accomplished less than you would like

Yes .........................................................................................1
No...........................................................................................2

7. Didn’t do work or other activities as carefully as usual

Yes .........................................................................................1
No...........................................................................................2

8. During the past 4 weeks, how much did pain interfere with your normal work, including both
work outside the home and housework?

[Hand respondent response card 24]

5 4 3 2 1
Not at all A little bit Moderately Quite a bit Extremely

These questions are about how you feel and how things have been with you during the past 4
weeks. For each question, please give the one answer that comes closest to the way you have
been feeling. How much of the time during the past 4 weeks

[Hand respondent response card 25]

All of Most of A good bit Some A little None of
the time the time of the of the of the the time
time time time

9. Have you felt calm and peaceful?* 6 5 4 3 2 1

10. Did you have a lot of energy?*** 6 5 4 3 2 1
11. Have you felt downhearted 6 5 4 3 2 1
and blue?***

12. During the past 4 weeks, how much of the time has your physical health or emotional
problems interfered with your social activities, like visiting with friends, relatives, etc.?

[Hand respondent response card 26]

5 4 3 2 1
All of the time Most of the Some of the A little of the None of the
time time time time

Peer Outcomes Protocol – Quality of Life Module

 31

In this next section, I would like to know how you feel about the quality of your life. For the
next question, I will use the Delighted-Terrible Scale.

[Hand respondent response card 27]

Please look at this card. The scale goes from terrible which is the lowest ranking of 1, to
delighted, which is the highest ranking of 7. There are also points 2 through 6 with descriptions
below them. Please tell me what on the scale best describes how you feel.

13. How do you feel about your life in general?

1 2 3 4 5 6 7
Terrible Unhappy Mostly Mixed Mostly Pleased Delighted
Dissatisfied Satisfied

Now I am going to make a series of statements about how you view your life right now. Please
tell me how much you agree or disagree with the statement.

[Hand respondent response card 28]

Disagree Somewhat Somewhat Agree
Disagree Agree
14. In general, I am satisfied with my
1 2 3 4
physical health.

15. In general, I am satisfied with my

1 2 3 4
emotional health.

16. In general, I am satisfied with how things

1 2 3 4
are going in my life.

17. I often do things that are enjoyable. 1 2 3 4

18. I am hopeful about the conditions of my

1 2 3 4
life in general.
19. Currently I have sufficient resources to
live on, such as adequate housing, 1 2 3 4
clothing, and food.
20. I worry about not having sufficient
1 2 3 4
resources to live on in the future.

21. I have things to do each day that give

1 2 3 4
meaning to my life.

Peer Outcomes Protocol – Quality of Life Module

 32

Now, I would like to know what impact participating in this program has on your life. I’m going
to read a series of statements and ask you how much you agree or disagree with each one.

[Hand respondent response card 28]

Disagree Somewhat Somewhat Agree
Disagree Agree

22. This program helps me to improve the

1 2 3 4
quality of my life.

23. This program helps me to do things that

1 2 3 4
are enjoyable.

24. This program helps me to be hopeful

1 2 3 4
about the conditions of my life.

25. This program helps me to worry less

about having sufficient resources to live 1 2 3 4
on in the future.

26. This program helps to protect my basic

1 2 3 4
human rights.

27. This program helps protect my rights as a

1 2 3 4
mental health consumer.

28. This program helps me to have

1 2 3 4
meaningful activities in my life

Peer Outcomes Protocol – Quality of Life Module

 33

WELL-BEING MODULE
Next, I would like to know about your sense of identity and self-esteem. Please tell me how
much you agree or disagree with the following statements.

[Hand respondent response card 29]

Disagree Somewhat Somewhat Agree

Disagree Agree

1. In general, I am satisfied with who I am

1 2 3 4
as a person.

2. I feel that I get the respect that I deserve

1 2 3 4
from important people in my life.

3. I am comfortable asking people to take

1 2 3 4
me seriously.

4. I feel that my opinions count. 1 2 3 4

5. I feel that I can trust my own decisions. 1 2 3 4

6. I feel that I have contributions to make in

1 2 3 4
life.

7. This program enables me to make

1 2 3 4
contributions in life.

8. This program helps me to believe that

1 2 3 4
personal growth in my life is possible.

9. This program helps me get respect from

1 2 3 4
important people in my life.

10. Being with members at this program

1 2 3 4
helps me to have personal power.

Peer Outcomes Protocol – Well-Being Module

 34

Disagree Somewhat Somewhat Agree

Disagree Agree
11. I take an active role in decisions about
1 2 3 4
my mental health services.

12. I have control over my daily routine. 1 2 3 4

13. I can change the things about my life that

1 2 3 4
are important to me.

14. I am becoming self-sufficient in my life. 1 2 3 4

15. I am knowledgeable about mental health

1 2 3 4
issues.

16. This program helps me have more

1 2 3 4
choices in my life.

17. This program inspired me to believe that

1 2 3 4
I can live independently.

18. This program helps me make positive

1 2 3 4
changes in my life.

19. This program helps me have an active

role in decisions about my mental health 1 2 3 4
services.

20. This program helps me to have control

1 2 3 4
over my daily routine.
21. This program helps me make needed
changes in the things that are important 1 2 3 4
to me.
22. This program helps me become self-
1 2 3 4
sufficient in my life.

23. I feel that I can change things about this

1 2 3 4
program if I want to.

24. I feel that I am involved in the planning

1 2 3 4
for the future of this program.

Peer Outcomes Protocol – Well-Being Module

 35

I would like to know how you are doing in your efforts to heal and recover from mental illness,
be empowered, and build an identity for yourself. Please tell me how much you agree or
disagree with the following statements about recovery.

[Hand respondent scale card 29].

Disagree Somewhat Somewhat Agree
Disagree Agree
25. In general, I am satisfied with my
progress towards recovery from mental 1 2 3 4
illness.
26. I usually know if I am beginning to have
1 2 3 4
a psychiatric problem.

27. If I have a psychiatric problem, usually I

can do something about it before it 1 2 3 4
becomes severe.
28. I have hope that I will recover from
1 2 3 4
mental illness.
29. I understand what recovery involves for
1 2 3 4
me.
30. I believe that personal growth in my life
1 2 3 4
is possible.

31. I am making positive changes in my life. 1 2 3 4

32. I usually can handle life’s ups and

1 2 3 4
downs.
33. If I am having emotional problems,
1 2 3 4
usually I can cope.

34. In general, I am satisfied with the kinds

1 2 3 4
of choices I can make in my life.

35. This program gives me hope that I will

1 2 3 4
recover from mental illness.

36. This program helps me cope if I have

1 2 3 4
psychiatric problems.

37. This program helps me to understand

1 2 3 4
what recovery involves for me.

Peer Outcomes Protocol – Well-Being Module

 36

PROGRAM SATISFACTION MODULE

In this last section of the survey, I would like your opinion of this peer support program. Please
tell me how much you agree or disagree with the following statements.

[Hand respondent response card 30].

Disagree Somewhat Somewhat Agree
Disagree Agree

1. Overall, I am satisfied with this peer

1 2 3 4
support program.

2. I am satisfied with the peer program

facilities, such as the condition and 1 2 3 4
layout of the rooms and building.

3. I do not feel physically safe when I am at

4 3 2 1
this program. ***

4. Overall, the program services are useful

1 2 3 4
to me.

5. This program is helpful to me regarding

my employment needs, such as choosing 1 2 3 4
or keeping a job.

6. This program is helpful to me regarding

my housing needs, such as finding a 1 2 3 4
place to live that I like.

7. This program is helpful to me regarding

my educational needs, such as finishing a
1 2 3 4
degree, or getting into a training
program.
8. This program is helpful to me regarding
my recreational needs, such as being
1 2 3 4
involved in a hobby, playing games, or
watching movies.

9. This program is helpful to me regarding

my transportation needs, such as helping 1 2 3 4
me get to this program.

Peer Outcomes Protocol – Program Satisfaction Module

 37

Disagree Somewhat Somewhat Agree

Disagree Agree

10. At this program I get the kind of

1 2 3 4
information that I need.

11. At this program I get information when I

1 2 3 4
need it.

12. In general, I feel that program staff

actively promote my human rights. By
human rights, I mean my rights to
1 2 3 4
freedom of speech or access to legal
representation, or my rights as a mental
health consumer.
13. I feel program staff are respectful of my
1 2 3 4
racial or ethnic background.

14. I feel program members are respectful of

1 2 3 4
my racial or ethnic background.

15. I feel program staff are respectful of my

1 2 3 4
sexual orientation.

16. I feel program members are respectful of

1 2 3 4
my sexual orientation.

17. I feel program staff are respectful of my

1 2 3 4
gender.

18. I feel program members are respectful of

1 2 3 4
my gender.

19. I feel program staff respect my wishes

regarding the confidentiality of my 1 2 3 4
personal information.

20. In general, program staff are competent. 1 2 3 4

21. I feel that program staff ignore my

4 3 2 1
individual problems.***

Peer Outcomes Protocol –- Program Satisfaction Module

 38

Disagree Somewhat Somewhat Agree

Disagree Agree
22. I feel safe talking about personal matters
1 2 3 4
with program staff.

23. In general, members and staff do not get

4 3 2 1
along with each other at this program.

24. In general, members at this program are

1 2 3 4
considerate.

25. I would recommend this program to other

1 2 3 4
mental health consumers.

26. I am able to accept criticism about

1 2 3 4
myself from program staff.

27. I feel that program staff are able to see

1 2 3 4
me as a person who has strengths

28. I feel that there are few power struggles

between members and program staff in 4 3 2 1
this program.

29. I feel that program staff focus on my real,

1 2 3 4
concrete needs.

30. I feel comfortable voicing my positive

1 2 3 4
opinions of this program.

31. I feel comfortable voicing my negative

1 2 3 4
opinions of this program.

32. This program helps me become

knowledgeable about mental health 1 2 3 4
issues.

33. This peer support program is making a

positive difference in how I feel about 1 2 3 4
myself as a person.
34. This program helps me cope if I have an
1 2 3 4
emotional crisis.

Peer Outcomes Protocol –- Program Satisfaction Module

 39

Disagree Somewhat Somewhat Agree

Disagree Agree
35. This program helps me become self-
1 2 3 4
sufficient in my life.

36. Participation at this peer support program

is making a positive difference in my 1 2 3 4
social life.

37. I feel comfortable socializing with

1 2 3 4
members of this peer support program.
38. I feel that I do not have to hide my
diagnosis of mental illness from 1 2 3 4
members of this program.
39. I can turn to program members at this
peer support program if I need help in
doing things, such as moving, getting a 1 2 3 4
ride, baby sitting, or organizing a party
for someone.
40. I get the emotional support that I need
1 2 3 4
from members in this program.

Peer Outcomes Protocol –- Program Satisfaction Module

 40

The following items relate to your experience of coercion within the mental health programs you
attend. Please indicate how often you feel this way.

[Hand respondent response card 31]

Always Most of Some of the Rarely Never
the Time Time
41. I feel pressured by staff to do 1 2 3 4 5
what they want me to do.

42. I feel like staff will get back at 1 2 3 4 5

me if I do not do what they
want me to do.

43. I have to butter up to staff to 1 2 3 4 5

get what I want.

44. I have to butter up to staff to 1 2 3 4 5

get what I need.
45. I have to do something staff 1 2 3 4 5
wants to get something I want.

46. Staff threatens me with the 1 2 3 4 5

loss of my housing.

47. Staff threatens me with the 1 2 3 4 5

loss of my spending money.

48. Staff threatens me with 1 2 3 4 5

hospitalization.
49. Staff threatens to make me 1 2 3 4 5
take medication I do not want.

50. Staff threatens me in other 1 2 3 4 5

ways.

Peer Outcomes Protocol – Program Satisfaction Module

 41

CONCLUSION
Thank you very much for completing this questionnaire. Your input is very important in developing the
final questionnaire. We want to make it as useful to peer support programs as possible.

1. We’ve covered a lot of ground, are there any thoughts or issues that you’d like to talk about?

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

[INTERVIEWER: Record the time here that the interview ends and remember to record
the end time on the cover page.]

End Time ____________

Peer Outcomes Protocol – Conclusion

 The Evaluation Center @ HSRI

RECIPROCAL SUPPORT SCALE

Silver,T., Bricker, D., Pesta, Z., & Pugh, D.

For more information contact:

Thelma Silver, Ph.D., LISW
Email: doovil@aol.com
H H

Permission to use:

The Reciprocal Support Scale is not copyrighted and the instrument can be used freely. There is not a
user’s fee associated with the instrument.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 192

Reciprocal Support Scale items and response format

Responses are in Likert format:

1. Almost Never 2. Rarely 3. Sometimes 4. Often 5. Almost Always

1. I find it easy to communicate my needs to my recovery partner.

2. I value my recovery partner as a person.

3. My recovery partner values me as a person.

4. My recovery partner serves as a role model.

5. I serve as a role model to my recovery partner.

6. I am supportive of my recovery partner.

7. My recovery partner is supportive of me.

8. I trust my recovery partner.

9. I think my recovery partner trusts me.

10. My recovery partner helped me with problem-solving.

11. I helped my recovery partner with problem-solving.

12. We can count on each other for advice.

13. We help each other.

14. We respect each other.

 The Evaluation Center @ HSRI

RECOVERY ASSESSMENT SCALE (RAS)

Giffort, D., Schmook, A., Woody, C., Vollendorf, C., & Gervain, M.

For more information contact:

Patrick W. Corrigan, Psy.D.

Center for Psychiatric Rehabilitation
at Evanston Northwestern Healthcare
Email: p-corrigan2@northwestern.edu
Phone: (224) 364-7203

Permission to use:

The RAS is not copyrighted and can be used freely. There is not a user’s fee associated with the
instrument.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 194

 RECOVERY ASSESSMENT SCALE
Giffort D, Schmook A, Woody C, Vollendorf C, and Gervain M
Corrigan, P. W., Giffort, D., Rashid, F., Leary, M., & Okeke, I. (1999). Recovery as a
Psychological Construct. Community Mental Health Journal, 35(3), 231-239.

Instructions: Below is a list of statements that describe how people sometimes feel about
themselves and their lives. Please read each one carefully and circle the number to the right
that best describes the extent to which you agree or disagree with the statement. Circle only
one number for each statement and do not skip any items.

Strongly Dis- Not Strongly

Disagree agree Sure Agree Agree

1. I have a desire to succeed. 1 2 3 4 5

2. I have my own plan for how to stay or become 1 2 3 4 5

well.

3. I have goals in life that I want to reach. 1 2 3 4 5

4. I believe I can meet my current personal goals. 1 2 3 4 5

5. I have a purpose in life. 1 2 3 4 5

6. Even when I don't care about myself, other people 1 2 3 4 5

do.

7. I understand how to control the symptoms of my 1 2 3 4 5

mental illness.

8. I can handle it if I get sick again. 1 2 3 4 5

9. I can identify what triggers the symptoms of my 1 2 3 4 5

mental illness.

10. I can help myself become better. 1 2 3 4 5

11. Fear doesn't stop me from living the way I want 1 2 3 4 5

to.

12. I know that there are mental health services that 1 2 3 4 5

do help me.

13. There are things that I can do that help me deal 1 2 3 4 5

with unwanted symptoms.

14. I can handle what happens in my life. 1 2 3 4 5

15. I like myself. 1 2 3 4 5

 Strongly Dis- Not Strongly
Disagree agree Sure Agree Agree

16. If people really knew me, they would like me. 1 2 3 4 5

17. I am a better person than before my experience 1 2 3 4 5

with mental illness.

18. Although my symptoms may get worse, I know I 1 2 3 4 5

can handle it.

19. If I keep trying, I will continue to get better. 1 2 3 4 5

20. I have an idea of who I want to become. 1 2 3 4 5

21. Things happen for a reason. 1 2 3 4 5

22. Something good will eventually happen. 1 2 3 4 5

23. I am the person most responsible for my own 1 2 3 4 5

improvement.

24. I'm hopeful about the future. 1 2 3 4 5

25. I continue to have new interests. 1 2 3 4 5

26. It is important to have fun. 1 2 3 4 5

27. Coping with my mental illness is no longer the 1 2 3 4 5

main focus of my life.

28. My symptoms interfere less and less with my life. 1 2 3 4 5

29. My symptoms seem to be a problem for shorter 1 2 3 4 5

periods of time each time they occur.

30. I know when to ask for help. 1 2 3 4 5

31. I am willing to ask for help. 1 2 3 4 5

32. I ask for help, when I need it. 1 2 3 4 5

33. Being able to work is important to me. 1 2 3 4 5

34. I know what helps me get better 1 2 3 4 5

35. I can learn from my mistakes. 1 2 3 4 5

36. I can handle stress. 1 2 3 4 5

 Strongly Dis- Not Strongly
Disagree agree Sure Agree Agree

37. I have people I can count on. 1 2 3 4 5

38. I can identify the early warning signs of becoming 1 2 3 4 5

sick.

39. Even when I don't believe in myself, other people 1 2 3 4 5

do.

40. It is important to have a variety of friends. 1 2 3 4 5

41. It is important to have healthy habits. 1 2 3 4 5

 The Evaluation Center @ HSRI

THE RECOVERY MEASUREMENT TOOL VERSION 4 (RMT)

Ralph, R. O.

For more information contact:

Ruth O. Ralph, Ph.D.

Phone: 207- 934-0579
Email: ruth.ralph@maine.edu
H H

Permission to use:

The RMT is not copyrighted and can be used freely. There is not a user’s fee associated with the
instrument; however the author requests data from the instrument’s use.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 198

 Recovery Measurement Tool Version 4
Ruth O. Ralph, Ph.D. (ruth.ralph@maine.edu)
Below are a number of statements that have been developed by a group of consumers in relation to their own recovery. We are
asking you to help us to see if this will work as a measurement of recovery. If you are willing, please follow the directions given.
Thank you for your time and effort.
Please tell us the following about yourself

What is your age______? Are you Male or Female (circle one)? M F

What is your education? (circle highest) Grade: 1 2 3 4 5 6 7 8 9 10 11 12 : Trade school/Associate degree – or 2

years past H.S., Bachelor’s Degree – or 4 years past H.S., Graduate School – more than 4 years past H.S. Graduate degree

What is your race/ethnicity? ________________________________________

Where do you live? City_______________ State_________________________

DIRECTIONS: FOR EACH STATEMENT, PLEASE CIRCLE THE NUMBER THAT INDICATES WHAT IS MOST LIKE YOU. IF
YOU FEEL IT DOES NOT APPLY TO YOU, CIRCLE THE 9 UNDER NOT APPLICABLE

Item Not at all Not very Somewhat Quite a Very much Not
like me much like like me bit like like me Applicable
me me
1. I don’t think there is anything good in my life. 1 2 3 4 5 9

2. There is hope for me even when I do not feel well. 1 2 3 4 5 9

3. I manage my pain or physical difficulties 1 2 3 4 5 9

4. When I do creative things I feel better. 1 2 3 4 5 9

5. I need to believe in something to change. 1 2 3 4 5 9

6. I develop plans to take care of myself, e.g. eating well, 1 2 3 4 5 9

sleeping enough, exercising.
7. I visit a number of places to see where I can make 1 2 3 4 5 9
friends.
8. I have an active social support network. 1 2 3 4 5 9

9. I am in a great deal of pain and/or I experience physical 1 2 3 4 5 9

difficulties.
 Item Not at all Not very Somewha Quite a bit Very much Not
like me much like t like me like me like me Applicable
me
10. I will find places where I can make friends. 1 2 3 4 5 9

11. I live in a safe place. 1 2 3 4 5 9

12. I find ways to improve my financial situation. 1 2 3 4 5 9

13. I am using my mind to the best of my ability. 1 2 3 4 5 9

14. I feel rage. 1 2 3 4 5 9

15. The thought that there is nothing good in my life is 1 2 3 4 5 9

affecting my mental health.
16. I can find a spiritual path that works for me. 1 2 3 4 5 9

17. I am committed to respecting my body and taking care 1 2 3 4 5 9

of it.
18. I participate in meaningful activities. 1 2 3 4 5 9

19. There is no meaning or purpose to my life. 1 2 3 4 5 9

20. I want my suffering to end so I can move on with my 1 2 3 4 5 9

life.
21. With the help of others, I feel better. 1 2 3 4 5 9

22. I will ask for help from others. 1 2 3 4 5 9

23. I can plan for my own future. 1 2 3 4 5 9

24. I can get on with my life when I have hope. 1 2 3 4 5 9

25 I feel confused. 1 2 3 4 5 9

26. I want help to better my situation. 1 2 3 4 5 9

27. I will seek out activities I enjoy. 1 2 3 4 5 9

28. I won’t be disabled by hopelessness and suffering. 1 2 3 4 5 9

 Item Not at all Not very Somewha Quite a bit Very much Not
like me much like t like me like me like me Applicable
me
29. Helping others find meaning and purpose helps me 1 2 3 4 5 9
feel connected and empowered.
30. I don’t take care of myself in any way. 1 2 3 4 5 9

31. I recognize I need to be active. 1 2 3 4 5 9

32. There is a basis for my pain. 1 2 3 4 5 9

33. When I have difficult feelings, I surround myself with 1 2 3 4 5 9

positive experiences.
34. I can make my own decisions 1 2 3 4 5 9

35. I am taking care of myself on a daily basis 1 2 3 4 5 9

36. I feel isolated 1 2 3 4 5 9

37. I am learning how to manage my pain and physical 1 2 3 4 5 9

difficulties.
38. I am determined to make my plans for self care work. 1 2 3 4 5 9

39. I am homeless. 1 2 3 4 5 9

40. I live in poverty. 1 2 3 4 5 9

41. I have no one to help me. 1 2 3 4 5 9

42. I need to interact with people that nourish my well- 1 2 3 4 5 9

being
43. I look for something better through reading and talking 1 2 3 4 5 9
to people.
44. When I take care of myself I feel better 1 2 3 4 5 9

45. I am not able to work 1 2 3 4 5 9

46. I think there is something better for me. 1 2 3 4 5 9

47. I am determined to keep a positive outlook on life. 1 2 3 4 5 9

 Item Not at all Not very Somewha Quite a bit Very much Not
like me much like t like me like me like me Applicable
me
48. I will choose the people I want to be with. 1 2 3 4 5 9

49. I feel frightened a lot of the time. 1 2 3 4 5 9

50. I can maintain the housing of my choice 1 2 3 4 5 9

51. Being in distress affects my body and how I care for 1 2 3 4 5 9

myself.
52. I can find a purpose for living. 1 2 3 4 5 9

53. I need to have people in my life who are supportive, 1 2 3 4 5 9

accepting, and understanding.
54. Social supports can help change my situation. 1 2 3 4 5 9

55. I have financial stability. 1 2 3 4 5 9

56.I have lost friends or family members because of my 1 2 3 4 5 9

illness
57. I can find my way out of homelessness and poverty. 1 2 3 4 5 9

58. I will limit my interactions with people who do not 1 2 3 4 5 9

support my recovery
59. There are healthy ways to end my suffering. 1 2 3 4 5 9

60. I think I can build some friendships. 1 2 3 4 5 9

61. I look for social supports. 1 2 3 4 5 9

62. I am determined to keep my mind stimulated and 1 2 3 4 5 9

open to new ideas
63, I have a regular source of income. 1 2 3 4 5 9

64.I have no desire to do any activities 1 2 3 4 5 9

65.I need to reconnect with people in my past. 1 2 3 4 5 9

 Item Not at all Not very Somewha Quite a bit Very much Not
like me much like t like me like me like me Applicable
me
66. When I do creative things with other people I have 1 2 3 4 5 9
more energy
67.I follow through on my plans to care for myself. 1 2 3 4 5 9

68.I am determined to be open to meeting new people 1 2 3 4 5 9

69. I feel discriminated against by family or friends 1 2 3 4 5 9

because of my mental illness.
70. Having a safe and comfortable living situation helps 1 2 3 4 5 9
my recovery
71. People I know make friends at churches, social clubs, 1 2 3 4 5 9
drop-in centers, school or work so I can too
72. I focus on empowering thoughts that support my 1 2 3 4 5 9
mental health
73. I am committed to caring for myself 1 2 3 4 5 9

74. I am hurting for a reason 1 2 3 4 5 9

75. I recognize I can be active 1 2 3 4 5 9

76. No one would hire me to work for them. 1 2 3 4 5 9

77. I am committed to nurture my relationships. 1 2 3 4 5 9

78. I want my feelings to change 1 2 3 4 5 9

79. I want to learn about what will be helpful for me. 1 2 3 4 5 9

 Item Not at all Not very Somewha Quite a bit Very much Not
like me much like t like me like me like me Applicable
me
80. I will seek people to be with who support my recovery. 1 2 3 4 5 9

81. I want to explore choices about possible activities 1 2 3 4 5 9

82. If social supports I need are not available I will work 1 2 3 4 5 9

with others to create them.
83. I have physical well-being. 1 2 3 4 5 9

84. I am not able to do things to take care of myself, like 1 2 3 4 5 9

cooking or laundry.
85. I want to learn more about taking care of myself. 1 2 3 4 5 9

86. I will choose people to be in my life 1 2 3 4 5 9

87. I have an active and fulfilling social life. 1 2 3 4 5 9

88. Nothing is changing in my life. 1 2 3 4 5 9

89. I feel hopeless 1 2 3 4 5 9

90. I need to make connections with people. 1 2 3 4 5 9

91. My life has meaning and value. 1 2 3 4 5 9

 The Evaluation Center @ HSRI

RELATIONSHIPS AND ACTIVITIES THAT FACILITATE RECOVERY SURVEY (RAFRS)

Leavy, R.L., McGuire, A.B, Rhoades, C., & McCool, R.

For more information contact:

Richard Leavy, Ph.D.

Department of Psychology
Ohio Wesleyan University
Delaware, OH 43015
Email: rlleavy@owu.edu
H H

Permission to use:

The RAFRS is not copyrighted and can be used freely. There is not a user’s fee associated with the
instrument.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 205

 Relationships and Activities that Facilitate Recovery Survey (RAFRS)

We are interested in the relationships and activities that you feel have been helpful in
your own recovery from mental illness. By recovery, we mean the way you have learned to cope
with your mental illness and go forward with your life. Please answer all the questions, whether
or not you consider yourself to be in recovery right now.

Please read each of the statements and circle the rating that most closely matches your opinion.

1. In the last 6 months, my community support person (case manager) has been helpful in
my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

2. In the last 6 months, my parents have been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

3. In the last six months, my siblings (brothers and sisters) have been helpful in my
recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

4. In the last 6 months, my children have been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

5. In the last 6 months, my spouse or partner has been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

6. In the last 6 months, my best friend has been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

7. In the last 6 months, my pet has been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

8. In the last 6 months, staff members who work for the Mental Health Board have been
helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

9. If you were employed in the last 6 months, my boss or work supervisor has been helpful
in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse
10. In the last 6 months, attending mental health center groups has been helpful in my
recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

11. In the last 6 months, attending training session about the Recovery Model has been
helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

12. In the last 6 months, attending drop-in center and other self-help activities has been
helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

13. In the last 6 months, going to work has been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

14. In the last 6 months, taking medication has been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

15. In the last 6 months, talking with other people who have problems like mine has been
helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

16. In the last 6 months, talking with people who have a psychiatric history has been helpful
in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

17. In the last six months, prayer and worship services have been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

18. In the last 6 months, vigorous exercise has been helpful in my recovery.

No contact Yes, helped a lot Yes, helped a little No, didn’t help Made things worse

Please indicate any other people who you think have been helpful in your recovery.

Please indicate any other activities that you think have been helpful in your recovery.
Review all of the relationships and activities you rated about. Please indicate the TWO (2) that
you feel have been the most helpful in your recovery over the past six months:

1._________________________________________________________________________

2._________________________________________________________________________
 The Evaluation Center @ HSRI

AACP ROSE-RECOVERY ORIENTED SERVICES EVALUATION

American Association of Community Psychiatrists

For more information contact:

Wesley E. Sowers, M.D.

Allegheny County Office of Behavioral Health
304 Wood Street, 5th Floor
Pittsburgh, PA 15222
Phone: 412-350-3716
Email: WSowers@dhs.county.allegheny.pa.us
H H

Permission to use:

The AACP ROSE is copyrighted by the American Association of Community Psychiatrists but can be used
freely. There is not a user’s fee associated with the instrument.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 209

 Strongly Disagree

Mostly Disagree

Somewhat

Mostly Agree

Strongly Agree
AACP ROSE-
Recovery Oriented Services Evaluation

0 1 2 3 4
Administration
1. Promotion of recovery is included in organization's mission and vision
2. Service users are well represented in organization's internal review and
strategic planning processes
3. Stakeholders are recruited and retained to participate in organizational
oversight and development
4. Consumers are compensated for participation in administrative activities
(committees, CQI, etc)
5. Agency employs persons in recovery and persons with disabilities as
mentors and counselors
6. There are significant opportunities for service users and service
providers to interact outside clinical relationships
7. Service providers are knowledgeable about recovery principles and
recovery promotion
8. Service users are enlisted to participate in training of service providers.
9. Service users are well represented and respected in CQI processes
10. Outcome indicators are developed with service user participation
11. Outcome indicators are available to and make sense to service users
Treatment
12. There is comprehensive array of services available to meet all identified
needs.
13. All clinical services encourage the use of self-management principles
14. Advance directives/crisis plans are encouraged and respected by the
organization
15. A process is in place to assist service users to develop advance
directives
16. A process is in place to assure review and implement advance
directives during periods of incapacitation.
17. Organization is sensitive to cultural issues and provides services that
meet cultural needs
18. Staffing patterns reflect community’s ethnic/racial/linguistic profile.
19. Treatment planning is a collaborative process between service users
and providers
20. Service users are provided adequate information about service options
to make decisions regarding their service plans.
21. Choices made by service users are respected by providers
22. Recovery management plans are developed that emphasize individual
strengths and choice
23. Co-occurring disorders are treated at the same time and by the same
clinicians
24. A screening process is in place to assure detection of co-occurring
disorders
25. Organization meets competency standards for treating persons with co-
occurring disorders
 Strongly Disagree

Mostly Disagree

Somewhat

Mostly Agree

Strongly Agree
AACP ROSE-
Recovery Oriented Services Evaluation

0 1 2 3 4
26. Organization has program to reduce or eliminate the use of coercive
treatment
27. Attempts are made to engage and empower persons on involuntary
treatment status
28. Staff has been adequately trained to de-escalate volatile situations and
to avoid seclusion and restraint
29. Debriefing occurs following all episodes of seclusion or restraint if it
must be used.
Supports
30. Organization facilitates service user participation and leadership in
advocacy and peer support efforts/organizations
31. Organization has an active liaison with local advocacy and peer support
groups
32. Service users consistently indicate satisfaction with access to services.
33. Family members are engaged and educated to support recovery efforts.
34. Opportunities exist for family members to be involved in treatment
planning and organizational development
35. Family members are represented on committees and are involved in
staff training
36. Service users are encouraged and supported in pursuit of employment
and vocational skills.
37. Development of educational and employment goals are emphasized in
recovery plans
38. Individualized placement and support guides vocational activities
39. Tolerant housing is available to those who cannot maintain sobriety or
stable recovery.
40. Service users are satisfied with housing options available.
Organizational Culture
41. Service users feel respected by service providers
42. Service users feel welcome and valued
43. Providers communicate with service users honestly and sincerely.
44. Documentation is an open process that service users may have easy
access to if desired.
45. *Service users are informed of their rights and responsibilities.
46. There is an equitable process through which service users and
providers can resolve conflicts or disagreement
AACP ROSE Scoring Sheet

Rater Category:
Service User
Family Member of Service User
Service Provider – Clinician
Service Provider – Administrator
Stakeholder Advocate
Other

Overall /184

Administration /44

Treatment /72

Supports /44

Organizational Culture /24

Key for overall score:

≥ 160 points Excellent

≥ 140 points Good
≥ 125 points Fair
≥ 110 points Needs significant improvement
≤ 95 points Traditional Standards
 The Evaluation Center @ HSRI

RECOVERY ENHANCING ENVIRONMENT MEASURE (REE)

ALSO KNOWN AS THE DEVELOPING RECOVERY ENHANCING ENVIRONMENT MEASURE
(DREEM)
Ridgway, P.A.

For more information contact:

Priscilla Ridgway, Ph.D., M.S.W.

Assistant Professor
Yale Program for Recovery and Community Health
319 Peck Street
Building 6W Suite 1C
New Haven, CT 06513
Phone: 203-764-8667
Email: priscilla.ridgway@yale.edu

Permission to use:

The REE is copyrighted by Priscilla A. Ridgway, 2005. Permission is required from the author prior to
using the instrument. Fees associate with the instrument’s use have yet to be determined.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 213

 Ridgway Recovery-Enhancing Environment Measure
(REE)
WORD VERSION

This questionnaire explores the process of recovery from psychiatric disability, and the services
and supports that mental health consumers say help them achieve recovery. While recovery is
always a personal process based in self-responsibility, there are many things mental health
programs can do to support your progress or hold you back. This questionnaire looks at your
personal experience of recovery, and the services and supports that are available to you.

Your answers to these questions will be confidential. This means your answers will stay secret.
Your name will never be asked. Please do not write you name in the booklet. This study is
completely voluntary. You can skip any questions that you do not wish to answer. Other consumers
have said that this questionnaire is very interesting and they enjoyed filling it out. The survey takes
about 25 minutes to complete. Be sure to read the instructions below before you begin to answer.

Instructions:

1. This is not a test. There are no right answers or wrong answers on this survey. Answer each
question based on your personal opinions and beliefs.

2. All of the questions should be answered by marking one of the answer spaces that best fits your
opinion or situation. If you don’t find an answer that fits exactly, use one that come closest. If
any questions does not apply to you, or you are not sure of what it means, just leave it blank.

For more information contact:

THANK YOU FOR YOUR TIME AND ANSWERS!!!!

1
 A FEW QUESTIONS ABOUT YOU

1. What age group are you in (Check your current age group)?
___18-25
___26-35
___36-45
___46-55
___56 and over

2. What is your gender?

___Male
___Female

3. What is your racial or ethnic background?

___African-American
___Asian or Asian-American
___Caucasian/Non-Hispanic
___First Nations/Native American
___Hispanic/Latino
___Other ________________________

4. In total, how long have you received any form of mental health services?
___Less than 1 year
___One year or more but less than five years
___Between five and ten years
___More than ten years

YOUR INVOLVEMENT IN THE RECOVERY PROCESS

Which of the following statements is most true for you? (Check only one)

__ I have never heard of, or thought about, recovery from psychiatric disability.
__ I do not believe I have any need to recover from psychiatric problems.
__ I have not had the time to really consider recovery.
__ I've been thinking about recovery, but haven't decided to move on it yet.
__ I am committed to my recovery, and am making plans to take action very soon.
__ I am actively involved in the process of recovery from psychiatric disability.
__ I was actively moving toward recovery, but now I'm not because:
________________________________________________________
__ I feel that I am fully recovered; I just have to maintain my gains.
__Other (specify)_____________________________________________

2
 For the rest of the questions in this survey, answer only about what you experience in:

___________________________________________________________
(name of mental health program)

If no program is listed above, think about the mental health program you use the most and the
staff of that program. Write the name of the program in the line above. Answer each of the
following questions keeping that particular program in mind.

1. What kind of services are you currently receiving in that program? (check in all that apply)

___ self-help or consumer-run services

___ clubhouse
___ day treatment program
___ residential program
___ case-management
___ psychotherapy
___ medications/med. management
___ vocational/employment services
___ supported housing
___ other (describe)_____________

ELEMENTS OF RECOVERY AND RECOVERY-ENHANCING PROGRAMS

For each of the following questions you should circle one of these answers:
SA --If you strongly agree with the statement.
A --If you agree with the statement
N --If you are not sure, or neither agree nor disagree, or you are neutral.
D --If you disagree with the statement.
SD --If you strongly disagree with the statement.
Strongly Agree Neutral Disagree Strongly
Agree Disagree
1. Having a positive sense of personal identity beyond SA A N D SD
my psychiatric disorder is important to my recovery.

a) Staff view me as more than a "case" or a diagnosis; SA A N D SD

they want to know me as a person.

b) The program offers individualized services to meet SA A N D SD

my unique needs.
SA A N D SD
c) Staff treat me as a whole person with a body, mind,
emotions, important relationships and spirit.

3
 Strongly Agree Neutral Disagree Strongly
Agree Disagree
2. Having a sense of meaning in life is important to my SA A N D SD
recovery.

a) Staff help me make sense out of what is happening SA A N D SD

in my life.

b) Staff ask me what is meaningful to me. SA A N D SD

c) This program encourages me do things that give my SA A N D SD

life meaning.

3. Having hope is important to my recovery. SA A N D SD

a) Staff believe I have a positive future. SA A N D SD

b) Staff encourage me to feel hopeful again when I'm SA A N D SD

discouraged or have a setback.
SA A N D SD
c) Staff tell me most people do recover from
psychiatric problems over time.

4. Having up-to-date knowledge about psychiatric SA A N D SD

disorders and the most effective treatments is
important in my recovery.

a) Staff teaches me about my psychiatric disorder and SA A N D SD

symptoms.

b) The program provides me up-to-date information SA A N D SD

about effective treatments.
SA A N D SD
c) Staff gives me enough information about my
treatment options and their risks and benefits, for
me to give informed consent for treatment.

5. Being able to self-manage symptoms and avoid SA A N D SD

relapse is important to my recovery.

a) This program helps me identify and monitor SA A N D SD

triggers/early signs of relapse.

b) This program helps me develop personalized coping

SA A N D SD
skills so I can manage stress well.

c) This program teaches me ways to self-monitor and SA A N D SD

self-control psychiatric symptoms.

4
 Strongly Agree Neutral Disagree Strongly
Agree Disagree
6. Improving my general health and wellness is SA A N D SD
important to my recovery.

a) Staff pay careful attention to my physical health. SA A N D SD

b) This program encourages me to achieve a higher SA A N D SD

level of wellness.
SA A N D SD
c) This program offers wellness programming such as
nutrition, movement, relaxation.

7. Being an active consumer and directing my own SA A N D SD

recovery is important to my recovery.

a) Staff assist me to explore options and set my own SA A N D SD

personal goals.

b) Staff treat me as a responsible partner in decision- SA A N D SD

making.
SA A N D SD
c) I direct my own treatment in this program.

8. Having my rights respected and upheld is important SA A N D SD

to my recovery.

a) Staff inform me of my rights. SA A N D SD

b) There is a clear grievance policy if any of my rights SA A N D SD

are violated.
SA A N D SD
c) Staff uphold my rights.

9. Mutual self-help/ peer support is important to my SA A N D SD

recovery.

a) This program encourages consumers to help and SA A N D SD

support one another.
SA A N D SD
b) Self-help groups and peer support opportunities are
available in this program.
SA A N D SD
c) This program actively links me to self-help groups
and self-help resources in the community.

5
 Strongly Agree Neutral Disagree Strongly
Agree Disagree
10. Being involved in meaningful activities is important SA A N D SD
to my recovery.

a) Staff encourages me to get involved in meaningful SA A N D SD

activities.

b) Program activities are meaningful. SA A N D SD

c) The program assists me to become involved in SA A N D SD

meaningful activities (such as working, furthering
my education, creativity, volunteerism).

11. Being involved in, and a part of, the larger SA A N D SD

community is important to my recovery.

a) Staff helps me find and use community resources. SA A N D SD

b) Staff help me gain individualized supports so I can

live, learn & work in the community. SA A N D SD

c) I don't feel cut-off from the "real world" in this SA A N D SD

program.

12. Having positive relationships is important to my SA A N D SD

recovery.

a) Staff assist me in having positive relationships with SA A N D SD

my peers.

b) Staff support me in building or rebuilding positive SA A N D SD

relationships with family members.
SA A N D SD
c) Staff assist me in forming friendships with people
outside the mental health system.

13. Identifying and building on my personal strengths is SA A N D SD

important to my recovery.

a) Staff recognize and focus on my positive attributes SA A N D SD

and talents.

b) Staff help me explore my dreams, values and goals. SA A N D SD

c) Staff link me to opportunities and resources that SA A N D SD

build on and reflect my strengths.

6
 Strongly Agree Neutral Disagree Strongly
Agree Disagree
14. Developing new skills is important to my recovery. SA A N D SD

a) Staff help me assess how I am functioning and SA A N D SD

identify skills I need to develop.

b) This program teaches me the skills I want and need. SA A N D SD

c) This program connects me to places and people who SA A N D SD

help me build important skills.

15. Having my basic needs met is important to my SA A N D SD

recovery.

a) This program assists me to get a basic income SA A N D SD

and/or benefits.

b) This program helps me get decent, affordable

SA A N D SD
housing and/or rent subsidies.

c) This program helps me gain access to health care. SA A N D SD

16. Having a sense of control over my life and feeling SA A N D SD

empowered is important to my recovery.

a) Staff encourage and support my sense of SA A N D SD

empowerment.

b) Staff assist me to gain or maintain control over

SA A N D SD
important decisions in my life.
SA A N D SD
c) Staff do not try to maintain power and control over
me.

17. Spirituality is important to my recovery. SA A N D SD

a) Staff ask me about my spiritual beliefs. SA A N D SD

b) Staff helps me connect with spiritual resources and SA A N D SD

groups, if I so desire.
SA A N D SD
c) Staff encourage me to explore spiritual practices
such as prayer or meditation that can support well-
being.

7
 Strongly Agree Neutral Disagree Strongly
Agree Disagree
18. Taking on, and succeeding in, normal social roles is SA A N D SD
important to my recovery.

a) Staff would help me get a real job and succeed as an SA A N D SD

employee.

b) Staff would assist me to return to school and be a

SA A N D SD
successful student.

c) Staff would help me get housing and be a successful SA A N D SD

tenant.

19. Challenging stigma and discrimination is important SA A N D SD

to my recovery.

a) This program helps me overcome internalized SA A N D SD

stigma (feeling badly about myself because of my
psychiatric label).
SA A N D SD
b) This program raises my awareness of stigma and
discrimination.

c) This program teaches me to be an effective self- SA A N D SD

advocate for my civil, human and personal rights.

20. Taking on new challenges and moving out of my SA A N D SD

comfort zone is important to my recovery.

a) Staff encourage me to take on new challenges. SA A N D SD

b) I feel supported when I try new things that seemed SA A N D SD

out of my reach before.

c) Staff encourage me to stretch myself and grow. SA A N D SD

21. Having positive role models is important to my SA A N D SD

recovery.

a) This program employs people who are positive role SA A N D SD

models of recovery.

b) Staff help me learn from others who have

SA A N D SD
successfully recovered (e.g. share consumer life
stories, internet sites, speakers, mentors).

c) I have opportunities to become a provider or role SA A N D SD

model in the program, if I so choose.

8
 Strongly Agree Neutral Disagree Strongly
Agree Disagree
22. Having assistance when I am in crisis is important to SA A N D SD
my recovery.

a) This program has help available immediately if I am SA A N D SD

in crisis.

b) Staff hang in with me through hard times, they help SA A N D SD

me see setbacks are a part of recovery.

c) This program has good options if I am in crisis that SA A N D SD

help me avoid involuntary treatment and
hospitalization.

23. Intimacy and sexuality are important to my SA A N D SD

recovery.

a) This program supports me in forming and SA A N D SD

succeeding in intimate relationships.

b) This program adequately addresses my sexuality. SA A N D SD

c) This program provides information on sexuality, SA A N D SD

such as safe sex, and med side effects and sexuality.

24. Having helpers who really care about me and my SA A N D SD

recovery is important to my recovery.

a) The staff here really listen to me. SA A N D SD

b) Staff here spend enough quality time with me on

activities that promote my recovery. SA A N D SD

c) Staff encourage, motivate and support me to move SA A N D SD

toward recovery.

9
SPECIAL NEEDS
These questions relate to specific groups of people. If you are not a member of the specific
special needs group being asked about, place a check mark beside the question and go onto the
next question.

____ 1. If you are not a member of a minority group check here and skip to question 2.
Strongly Agree Neutral Disagree Strongly
Agree Disagree
Having my ethnic & cultural background respected is SA A N D SD
important to my recovery
a) Staff here are respectful to me as a person of a racial, SA A N D SD
ethnic, or cultural minority

b) This program understands and supports my cultural SA A N D SD

values/language/customs.

c) Staff are aware of, and sensitive to my cultural heritage SA A N D SD

and needs.

_____ 2. If you do not have both psychiatric problems and substance abuse check here and
skip to all of question 3.

Strongly Agree Neutral Disagree Strongly

Agree Disagree
Having help with alcohol or drug problems is important SA A N D SD
to my recovery.

a) This program has resources to help me with both SA A N D SD

alcohol and psychiatric problems.

b) This program has resources to help me with both SA A N D SD

drug and psychiatric problems.

c) This program links me to self-help groups that deal SA A N D SD

with dual diagnoses/ substance abuse.

10
_____ 3. If you do not have a history of abuse and/or trauma check here and skip to question 4.

Strongly Agree Neutral Disagree Strongly

Agree Disagree
Healing trauma, including sexual abuse and/or physical SA A N D SD
abuse, is important to my recovery.
a) This program has resources to help me heal from abuse SA A N D SD
and/or trauma.

b) It feels safe to open up about abuse or trauma in this SA A N D SD

program.

c) Staff deal effectively with abuse and trauma. SA A N D SD

____ 4. If you are not lesbian, gay, or bi-sexual put a check here and go to question 5.

Strongly Agree Neutral Disagree Strongly

Agree Disagree
Having support for my sexual orientation is important to SA A N D SD
my recovery.
a) Staff of this program are not homophobic (very SA A N D SD
negative about gay, lesbian or bi-sexual people).

b) Staff of the program are respectful to me as a lesbian, SA A N D SD

gay or bi-sexual person.

c) Staff deal effectively with issues of sexual preference . SA A N D SD

_____ 5. If you are not a parent put a check here and go on to the next section.

Strongly Agree Neutral Disagree Strongly

Agree Disagree
Having support as a parent is important to my SA A N D SD
recovery.
a) Staff support me in my role as parent. SA A N D SD

b) Staff assist me to be an effective parent. SA A N D SD

c) Staff help me uphold my rights in custody disputes. SA A N D SD

11
ORGANIZATIONAL CLIMATE

Circle the answer that best describes whether your organization has the quality we are asking
about. These qualities support resilience or the ability to rebound from adversity.

For each of the following questions you should circle one of these answers:
SA --If you strongly agree with the statement.
A --If you agree with the statement
N --If you are not sure, or neither agree nor disagree, or you are neutral.
D --If you disagree with the statement.
SD --If you strongly disagree with the statement.

Strongly Agree Neutral Disagree Strongly

Agree Disagree
1. The program promotes learning striving, and growth. SA A N D SD

2. The program is a hopeful environment that promotes SA A N D SD

positive expectations.

3. The program is inspiring and encouraging. SA A N D SD

4. Staff of this program are caring and compassionate. SA A N D SD

5. The program has enough resources to meet peoples' SA A N D SD

needs.

6. The program provides opportunities for meaningful SA A N D SD

participation and contribution.

7. The program helps people feel valued, respected and SA A N D SD

powerful.

8. The program helps people feel connected to others in SA A N D SD

positive ways.

9. The program is safe and attractive. SA A N D SD

10. All levels of staff are welcoming. SA A N D SD

11. There are creative and interesting things going on in the SA A N D SD

program.

12. The program provides real choices, desirable options, SA A N D SD

and opportunities.

13. The program asks for consumer feedback. SA A N D SD

14. The program makes changes based on consumer SA A N D SD

satisfaction.

12
RECOVERY MARKERS

For each of the following questions you should circle one of these answers that is true for you now.
SA --If you strongly agree with the statement.
A --If you agree with the statement
N --If you are not sure, or neither agree nor disagree, or you are neutral.
D --If you disagree with the statement.
SD --If you strongly disagree with the statement.

Strongly Agree Neutral Disagree Strongly

Agree Disagree
1. My living situation is safe and feels like home to me. SA A N D SD

2. I have trusted people I can turn to for help. SA A N D SD

3. I have at least one close mutual (give-and-take) SA A N D SD

relationship.

4. I am involved in meaningful productive activities. SA A N D SD

5. My psychiatric symptoms are under control. SA A N D SD

6. I have enough income to meet my needs. SA A N D SD

7. I'm not working, but see myself working within 6 SA A N D SD

months.

8. I am learning new things that are important to me. SA A N D SD

9. I am in good physical health. SA A N D SD

10. I have a positive spiritual life/connection to a higher SA A N D SD

power.

11. I like and respect myself. SA A N D SD

12. I'm using my personal strengths, skills or talents. SA A N D SD

13. I have goals I'm working to achieve. SA A N D SD

14. I have reasons to get out of bed in the morning. SA A N D SD

15. I have more good days than bad. SA A N D SD

16. I have a decent quality of life. SA A N D SD

17. I control the important decisions in my life. SA A N D SD

13
 Strongly Agree Neutral Disagree Strongly
Agree Disagree
18. I contribute to my community. SA A N D SD

19. I am growing as a person. SA A N D SD

20. I have a sense of belonging. SA A N D SD

21. I feel alert and alive. SA A N D SD

22. I feel hopeful about my future. SA A N D SD

23. I am able to deal with stress. SA A N D SD

24. I believe I can make positive changes in my life. SA A N D SD

Check the box that is true for you now. YES NO

25. I am working part time (less than 35 hours a week).

26. I am working full time (35 or more hours per week).

27. I am in school.

14
FINAL QUESTIONS

1. What are one or two of the most important things a mental health program and its staff can do to
support people with psychiatric disabilities in their mental health recovery?
_________________________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________

2. What are one or two of the most important thing you have learned so far on your journey of
recovery?
_________________________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________

3. What one or two things would you want to say to a person who is just beginning his or her journey
of recovery from psychiatric disability?
_________________________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________

4. Are there any other comments or ideas that could improve the program that you want to include in
the survey?
_________________________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________

THANK YOU!

15
 The Evaluation Center @ HSRI

RECOVERY ORIENTED SYSTEMS INDICATORS MEASURE (ROSI)

Dumont, J. M., Ridgway, P., Onken, S. J., Dornan, D. H., & Ralph, R. O.

For more information contact:

Steven J. Onken, Ph.D.

Columbia University School of Social Work
Email: so280@columbia.edu
Phone: (212) 851-2243

or

Jeanne M. Dumont, Ph.D.

Consultant in Mental Health Services Research
E-mail: jdumont@lightlink.com
Phone (607) 273-8021

Permission to use:

The ROSI will be in the public domain. Permission is recommended but not required for use of the
instrument. Fees associated with the instrument will include any needed or requested technical assistance
or training.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 229

 The Evaluation Center @ HSRI

Excerpted from the ROSI Research Team’s 2005 handout entitled Piloting the Recovery Oriented
System Indicators (ROSI) Administrative Data Profile and Consumer Self-Report Survey. Please note
additional use guidelines in the ROSI instrument description section

USING THE ROSI

The Research Team makes the following requests of any person or agency that chooses to move forward on
using the ROSI in the near future:

First, inform the Research Team of your wish to use the ROSI. This notification can be done by
contacting the Research Team through either Steven Onken <so280@columbia.edu> or Jeanne Dumont
<jdumont@lightlink.com>.

Second, use the measures as currently developed, do not shift the items around, change the wording of any
of the items, or shorten the measures by only gathering data on a subset of items.

Third, design your use in such a way that the data could be shared with the Research Team. The local site
would continue to ‘own’ the data, but would share the data set in aggregate form with the Research Team.
The Research Team’s request will be subject to approval by the local site’s research review, confidentiality
and IRB processes.

Fourth, gather a small set of additional data that includes self-report survey respondent demographic
variables, agency/authority-level descriptors, and methods of data collection.

By agreeing to these conditions, those using the ROSI measure will help advance recovery research in
several ways. The data gathered will be added to the data from other pilot sites to: 1) improve the analysis
of the statistical properties of the measure (psychometric testing); 2) improve the field’s understanding
of how program-/site-/systems-level variables influence findings; 3) build a database on how differing
sub-populations may differ in their responses to the ROSI; and 4) create a set of national norms that will
help in setting benchmarks for improvements in programs and systems. The larger the database that the
Research Team can acquire, the better the chances of conducting a thorough and sound analysis.

GUIDELINES FOR THE ROSI

The ROSI is developed from and grounded in the lived experiences of adults with serious and prolonged
psychiatric disorders. Thus, the ROSI consumer self-report survey and administrative profile are designed
to assess the recovery orientation of community mental health systems for adults with serious and
prolonged psychiatric disorders.

Using the 42-item ROSI consumer self-report survey without the allied use of the ROSI administrative
profile is not recommended. The 42-item consumer self-report survey is complemented by the administrative
data profile. Data that are generated by doing the self-report survey alone are incomplete. The
administrative profile gathers data on important indicators of the recovery orientation of a system that are
not covered on the consumer survey.

The ROSI consumer self-report survey currently does not have sub-scales and thus all 42 items should be
administered.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 230

 The Evaluation Center @ HSRI

It is important that you follow your process of human subject review in regards to securing approval for
conducting the ROSI consumer self-report survey and for being in compliance with HIPAA regulations. As
you determine the level of human subject review to complete, you will need to identify whether you need a
written or verbal consent, what are the risks and benefits for participants, and what participant incentive,
if any, you will provide.

You will need to develop a definition sheet for some of the terms used in the 42 items of the ROSI
consumer self-report survey. In this sheet, you will explain or define for the participants what and whom
you are asking them to evaluate. Thus, the definition sheets needs to be tailored to your specific mental
health service delivery system. What do you mean when an item uses the term “program” (see item #21
for example). Do you mean programs operated by the local public mental heath center or all local mental
health programs regardless who operates them? Or are you limiting it to one program? A similar set of
questions also applies to the term “staff.” Finally, how do you want to define “mental health services?” The
clearer you are in your definition sheet, the easier it is for participants to complete the survey (and the
easier for the survey administrator to answer their questions).
When administering the ROSI consumer self-report survey, please point out to the participants that some
of the items are negatively worded, for example, “Staff do not understand my experience as a person with
mental health problems.” Please instruct the participants to read each item carefully in order to answer the
negatively worded items accurately.

While the Research Team retained consumer’s phrasing in some individual items, as well as reduced
the average reading level for the 42-item ROSI consumer self-report survey; some of the individual
items require a high reading level. Some consumers may not have the literacy level needed to read or to
understand some items. The Research Team strongly recommends that someone (such as a volunteer or
peer specialist) be available to respondents during administration of the measure. This person can provide
reading support and assistance, as well as answer questions.

The NY Office of Mental Health has translated the 42-item ROSI consumer self-report survey into
Spanish. Because of differences in regional Spanish dialects and respondent literacy levels, the Research
Team strongly recommends that an interpreter be available to Spanish speaking respondents during the
administration of the survey. The 42-item ROSI consumer self-report survey is not available in other
languages at this time, but the Research Team is open to working with interested parties in such efforts.

Please record how you administered the ROSI using the ROSI Process Form, noting any variations that
occurred (e.g., “x” number were completed in a group setting, “x” number were completed one-on-one, an
English translator was available, etc.).

If you have questions, please contact the Research Team through either Steven Onken <so280@columbia.
edu> or Jeanne Dumont <jdumont@lightlink.com>. Thank you!

Measuring the Promise: A Compendium of Recovery Measures, Volume II 231

 Piloting the ROSI Measure Handout, August 03 2005, page 1
RECOVERY ORIENTED SYSTEM INDICATORS (ROSI) PROCESS FORM

Administering Entity:
Address:

1. ROSI measures completed:

a. Consumer Self-Report Survey b. Administrative Data Profile

2. Date data collection began: (day/month /year) / / /-/ / /-/ / /

Date data collection ended: (day/month /year) / / /-/ / /-/ / /

3. Type of process used to collect consumer self-report data (check all that apply and include the
response rate, i.e., ___ %, if available)

a. Consumer Self-Administered (___ %) g. Program Staff Interviewers (___ %)

b. Mail Administration (___ %) h. Consumer Interviewers (___ %)
c. Phone Administration (___ %) i. On-Line Data Collection (___ %)
d. Face To Face Administration (___ %) j. Quality Assurance Interviewers (___ %)
e. Individual Data Collection (___ %) k. External Evaluation Interviewers (___ %)
f. Group Data Collection (___ %) l. Other: (___ %)

4. If a sample was used, what sample methodology was involved?

a. Convenience Sample c. Stratified Sample

b. Random Sample d. Other:

5. Purpose for utilizing ROSI (check all that apply)

a. Quality Assurance Activity d. Research

b. Program Audit e. Other:
c. Program Evaluation

6. Provide any important feedback concerning the performance, usefulness, process, and findings based
upon your use of the ROSI measures

7. Contact Information for a person knowledgeable about the survey process

Thank you!
 Piloting the ROSI Measure Handout, August 03 2005, page 3

Recovery Oriented System Indicators (ROSI) Consumer Survey

Purpose: To provide the best possible mental health services, we want to know what things helped or hindered your
progress during the past six (6) months. Please follow the directions and complete all four sections.

Section One Directions: Please read each statement and then circle the response that best represents your situation
during the past six months. These responses range from “Strongly Disagree” to “Strongly Agree.” If the statement was
about something you did not experience, circle the last response “Does Not Apply To Me.”

Strongly Strongly Does Not

1. There is at least one person who believes in me. Disagree Disagree Agree Agree Apply To Me
Strongly Disagree Agree Strongly Does Not
2. I have a place to live that feels like a comfortable home to Disagree Agree Apply To Me
me.
Strongly Disagree Agree Strongly Does Not
3. I am encouraged to use consumer-run programs (for Disagree Agree Apply To Me
example, support groups, drop-in centers, etc.).
Strongly Strongly Does Not
4. I do not have the support I need to function in the roles I Disagree Disagree Agree Agree Apply To Me
want in my community.
Strongly Disagree Agree Strongly Does Not
5. I do not have enough good service options to choose from. Disagree Agree Apply To Me
Strongly Strongly Does Not
6. Mental health services helped me get housing in a place I Disagree Disagree Agree Agree Apply To Me
feel safe.
Strongly Strongly Does Not
7. Staff do not understand my experience as a person with Disagree Disagree Agree Agree Apply To Me
mental health problems.
Strongly Disagree Agree Strongly Does Not
8. The mental health staff ignore my physical health. Disagree Agree Apply To Me
Strongly Strongly Does Not
9. Staff respect me as a whole person. Disagree Disagree Agree Agree Apply To Me
Strongly Strongly Does Not
10. Mental health services have caused me emotional or Disagree Disagree Agree Agree Apply To Me
physical harm.
Strongly Disagree Agree Strongly Does Not
11. I cannot get the services I need when I need them. Disagree Agree Apply To Me
 Piloting the ROSI Measure Handout, August 03 2005, page 4

Please circle the response that best represents your situation during the past six months.
Strongly Strongly Does Not
12. Mental health services helped me get medical benefits that Disagree Disagree Agree Agree Apply To Me
meet my needs.
Strongly Strongly Does Not
13. Mental health services led me to be more dependent, not Disagree Disagree Agree Agree Apply To Me
independent.
Strongly Strongly Does Not
14. I lack the information or resources I need to uphold my Disagree Disagree Agree Agree Apply To Me
client rights and basic human rights.
Strongly Disagree Agree Strongly Does Not
15. I have enough income to live on. Disagree Agree Apply To Me
Strongly Disagree Agree Strongly Does Not
16. Services help me develop the skills I need. Disagree Agree Apply To Me

Section Two Directions: Please read each statement and then circle the response that best represents your situation
during the past six months. The responses range from “Never/Rarely” to “Almost Always/Always.” If the statement was
about something you did not experience, circle the last response “Does Not Apply To Me.”

Almost Always/ Does not

17. I have housing that I can afford. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
18. I have a chance to advance my education if I want to. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
19. I have reliable transportation to get where I need to go. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
20. Mental health services helped me get or keep employment. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
21. Staff see me as an equal partner in my treatment program. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
22. Mental health staff support my self-care or wellness. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
23. I have a say in what happens to me when I am in crisis. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
24. Staff believe that I can grow, change and recover. Never/Rarely Sometimes Often
Always apply to me
 Piloting the ROSI Measure Handout, August 03 2005, page 5

Please circle the response that best represents your situation during the past six months.

Almost Always/ Does not

25. Staff use pressure, threats, or force in my treatment. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
26. There was a consumer peer advocate to turn to when I Never/Rarely Sometimes Often
Always apply to me
needed one.
Almost Always/ Does not
27. There are consumers working as paid employees in the Never/Rarely Sometimes Often
Always apply to me
mental health agency where I receive services.
Almost Always/ Does not
28. Staff give me complete information in words I understand Never/Rarely Sometimes Often
Always apply to me
before I consent to treatment or medication.
Almost Always/ Does not
29. Staff encourage me to do things that are meaningful to me. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
30. Staff stood up for me to get the services and resources I Never/Rarely Sometimes Often
Always apply to me
needed.
Almost Always/ Does not
31. Staff treat me with respect regarding my cultural Never/Rarely Sometimes Often
Always apply to me
background (think of race, ethnicity, religion, language, age,
sexual orientation, etc).
Almost Always/ Does not
32. Staff listen carefully to what I say. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
33. Staff lack up-to-date knowledge on the most effective Never/Rarely Sometimes Often
Always apply to me
treatments.
Almost Always/ Does not
34. Mental health staff interfere with my personal relationships. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
35. Mental health staff help me build on my strengths. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
36. My right to refuse treatment is respected. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
37. My treatment plan goals are stated in my own words. Never/Rarely Sometimes Often
Always apply to me
Almost Always/ Does not
38. The doctor worked with me to get on medications that were Never/Rarely Sometimes Often
Always apply to me
most helpful for me.
 Piloting the ROSI Measure Handout, August 03 2005, page 6

Please circle the response that best represents your situation during the past six months.

Almost Always/
39. I am treated as a psychiatric label rather than as a person. Never/Rarely Sometimes Often
Always
Does not
apply to me
Almost Always/
40. I can see a therapist when I need to. Never/Rarely Sometimes Often
Always
Does not
apply to me
Almost Always/
41. My family gets the education or supports they need to be Never/Rarely Sometimes Often
Always
Does not
apply to me
helpful to me.
Almost Always/
42. I have information or guidance to get the services and Never/Rarely Sometimes Often
Always
Does not
apply to me
supports I need, both inside and outside my mental health
agency.

Section Three Directions: Are there other issues related to how services help or hinder your recovery? Please explain.
 Piloting the ROSI Measure Handout, August 03 2005, page 7

Section Four Directions: We are asking you to provide the following information in order for us to be able to have a
general description of participants taking this survey. Please check the answer that best fits your response to the question
or write in the answer in the line provided. Only answer those items you wish to answer. Please do not write your name
or address on this survey. This keeps your identity confidential.

1. What is your gender? a. Female b. Male

2. What is your age? (Write your current age in the two boxes.)

3. What is your racial or ethnic background? (Check the one that applies best.)
a. American Indian/ Alaska Native d. Native Hawaiian/ Other Pacific Islander f. More than one race
b. Asian e. White/Caucasian g. Other: ____________________________
c. Black or African American

Do you consider yourself Hispanic or Latino/a? a. Yes b. No

4. Your level of education is: (Check the highest level you reached or currently are in.)
a. Less than High School c. College/Technical Training e. Other: ____________________________
b. High School/GED d. Graduate School

5. How long have your been receiving mental health services?

a. Less than 1 year c. 3 to 5 years
b. 1 to 2 years d. More than 5 years

6. Which services have you used in the past six months? (Check all that apply.)
a. Counseling/Psychotherapy e. Assertive Community Treatment (ACT) i. Case Management
b. Housing/Residential Services f. Psychosocial Rehabilitation j. Clubhouse
c. Medication Management g. Employment/Vocational Services k. Other: ____________________________
d. Self-help/Consumer Run Service h. Alcohol/ Drug Abuse Treatment
 Piloting the ROSI Measure Handout, August 03 2005, page 8

[To survey administrator: Please collect this additional background information (if possible).]

7. The town, city or community you live in is mostly:

a. Urban c. Rural
b. Suburban d. Remote/Frontier

8. What type of place do you live in?

a. Living in my own home or apartment
b. Living in supervised/supported apartment
c. Living in a residential facility
d. Living in a boarding house
e. Homeless or homeless shelter
f. Other: ___________________________________

9. Are you a person who currently has both mental health and substance abuse (alcohol, drug addition) problems?
a. Yes b. No
 Piloting the ROSI Measure Handout, August 03 2005, page 9

ROSI Administrative-Data Profile: Authority Characteristics

Authority: Date ___________

1. What is your organization’s legal structure?

a. Public c. Private for Profit

b. Private Nonprofit d. Other:

2. Geographic Location:

Country:
State/ Province:

3. What geographic area do your cover?

4. Geographic Setting (check all that apply):

a. Urban d. Rural
b. Small City e. Remote/Frontier
c. Suburban

5. How many providers of mental health services are in your network (unduplicated)?

_______________

6. How many providers of mental health services are in your network provided data for this
ROSI Administrative-Data Profile?

_______________

7. What populations do you serve? (Check all that apply.)

a. Children General Mental Health f. Elderly Serious Mental Illness

b. Adult General Mental Health g. Children Substance Abuse
c. Elderly General Mental Health h. Adult Substance Abuse
d. Children Serious Emotional Disord i. Other: ____________________________
e. Adult Serious Mental Illness
 Piloting the ROSI Measure Handout, August 03 2005, page 10

ROSI Administrative-Data Profile: Mental Health Provider Characteristics

Provider Organization Date ___________

1. What is your organization’s legal structure?

a. Public c. Private for Profit

b. Private Nonprofit d. Other:

2. Geographic Location:

Country:
State/Province:
County:

3. Geographic Setting (check all that apply):

a. Urban d. Rural
b. Small City e. Remote/Frontier
c. Suburban

4. How many consumers does your organization serve in mental health services each year
(unduplicated)?

_______________

5. How many full time equivalents (FTEs) do you have on staff who directly provide mental
health services at this time?

_______________

6. Which mental health services do you provide at this time? (Check all that apply.)

a. Counseling/Psychotherapy g. Assertive Community Treatment (ACT)

b. Case Management h. Clubhouse
c. Housing/Residential Services i. Alcohol/ Drug Abuse Treatment
d. Medication Management j. Employment/Vocational Services
e. Self-help/Consumer Run Service k. Other: _____________________________
f. Psychosocial Rehabilitation
 Piloting the ROSI Measure Handout, August 03 2005, page 11

Recovery Oriented System Indicators (ROSI) Administrative-Data Profile

Recovery Theme: Peer Support (involves the findings that peer support and consumer operated
services in a myriad of forms facilitates recovery).

Performance Indicator: Free Standing Peer/Consumer Operated Programs

Authority Measure 1: The percentage of mental health catchment or service areas that
have free standing peer/consumer operated programs.
Numerator: Total number of mental health catchment or service areas that have free
standing peer/consumer operated programs.
Denominator: Total number of mental health catchment or service areas.
Provider Version of Measure 1: There is at least one free standing peer/consumer
operated program in our community. (Yes/No)

Performance Indicator: Peer/Consumer Operated Services Funding

Authority Measure 2: The percentage of state program funds allocated for peer/consumer
operated services.
Numerator: The amount of program funds in the state mental health budget allocated
for peer/consumer operated services during the reporting period.
Denominator: The total amount of program funds in state mental health budget during
the reporting period.

Authority Measure 3: The percentage of Medicaid funding reimbursed for peer/consumer

delivered services.
Numerator: The amount of Medicaid reimbursement for services delivered in
peer/consumer operated programs and by peer specialists during the reporting
period.
Denominator: The total amount of Medicaid reimbursement for behavioral health care
during the reporting period.

Performance Indicator: Consumer Employment in Mental Health Systems

Authority Measure 4: The number of annual slots specifically funded for training
consumers in relevant educational and training programs and institutes to become
mental health providers.

Authority Measure 5: The percentage of local mental health provider agencies that have
an affirmative action hiring policy regarding consumers.
Numerator: The number of local mental health provider agencies that have an
affirmative action hiring policy regarding consumers.
Denominator: The total number of local mental health provider agencies.
Provider Version of Measure 5: Our agency has an affirmative action hiring policy
regarding consumers. (Yes/No)

Recovery Theme: Choice (involves the findings that having choices, as well as support in the
process of making choices, regarding housing, work, social, service, treatment as well as other
areas of life facilitate recovery).
 Piloting the ROSI Measure Handout, August 03 2005, page 12

Performance Indicator: Advance Directives

Authority Measure 6: The percentage of local mental health provider agencies that have
an established mechanism to help clients develop advance directives.
Numerator: The number of local mental health provider agencies that have an
established mechanism to help clients develop advance directives.
Denominator: The total number of local mental health provider agencies.
Provider Version of Measure 6: Our agency has an established mechanism to help clients
develop advance directives. (Yes/No)

Recovery Theme: Formal Service Staff (involves the findings as to the critical roles formal
service staff play in helping or hindering the recovery process).

Performance Indicator: Direct Care Staff to Client Ratio

Authority Measure 7: The ratio of direct care staff to clients in each local mental health
provider agency.
Numerator: The total number of direct care staff (unduplicated) during the reporting
period.
Denominator: The total number of clients (unduplicated) during the reporting period.
Provider Version of Measure 7: The ratio of direct care staff to clients in the provider
agency.
Numerator: The total number of direct care staff (unduplicated) during the reporting
period.
Denominator: The total number of clients (unduplicated) during the reporting period.

Recovery Theme: Formal Services (involves the findings that formal service systems’ culture,
organization, structure, funding, access, choice, quality, range, continuity and other
characteristics can help or hinder the process of recovery).

Formal Services Sub-Theme: Helpful System Culture and Orientation (involves the
finding that a formal service system’s culture and orientation that is holistic and consumer
oriented facilitates recovery).

Performance Indicator: Recovery Oriented Mission Statement

Authority Measure 8: The state mental health authority’s mission statement explicitly
includes a recovery orientation. (Yes/No).

Authority Measure 9: The percentage of local mental health provider agencies whose
mission statements explicitly include a recovery orientation.
Numerator: The number of local mental health provider agencies whose mission
statement includes a recovery orientation.
Denominator: The total number of local mental health provider agencies.
Provider Version of Measure 9: Our agency’s mission statement explicitly includes a
recovery orientation. (Yes/No)

Performance Indicator: Consumer Involvement in Provider Contract Development

 Piloting the ROSI Measure Handout, August 03 2005, page 13

Authority Measure 10: The percentage of provider agency performance contracts that
have primary consumer involvement in their development/yearly review
(specifying services, outcomes, target numbers, etc).
Numerator: The number of provider agency performance contracts documenting
primary consumer involvement in their development/yearly review.
Denominator: The total number of provider agency performance contracts.

Performance Indicator: Office of Consumer Affairs

Authority Measure 11: The percentage of staff in the state office of consumer affairs
who are former or current consumers.
Numerator: The number state office of consumer affairs staff (unduplicated) who
are disclosed consumers (former or current) during the reporting period.
Denominator: The total number of state office of consumer affairs staff
(unduplicated) during the reporting period.

Authority Measure 12: The percentage of regional mental health offices/local mental
health authorities (or equivalent) that have an office of consumer affairs.
Numerator: The number of regional mental health offices/local mental health
authorities (or equivalent) that have an office of consumer affairs during the
reporting period.
Denominator: The total number of regional mental health offices/local mental
health authorities (or equivalent) during the reporting period.

Performance Indicator: Consumer Inclusion in Governance and Policy

Authority Measure 13: The percentage of state mental health authority planning
council members who are primary consumers.
Numerator: The number of primary consumers (unduplicated) who are state
planning council members during the reporting period.
Denominator: The total number state planning council members (unduplicated)
during the reporting period.

Authority Measure 14: The percentage of local mental health provider agency board
membership that are primary consumers.
Numerator: The number of primary consumers (unduplicated) who serve on local
mental health provider agency boards during the reporting period.
Denominator: The total number local mental health provider agency board
members (unduplicated) during the reporting period.
Provider Version of Measure 14: The percentage of our agency’s board membership
that are primary consumers.
Numerator: The number of primary consumers (unduplicated) who serve on our
board during the reporting period.
Denominator: The total number board members (unduplicated) during the
reporting period.

Formal Services Sub-Theme: Coercion (involves the finding that coercion in formal
service systems hinders recovery).
 Piloting the ROSI Measure Handout, August 03 2005, page 14

Performance Indicator: Involuntary Inpatient Commitments

Authority Measure 15: The percentage of clients under involuntary commitments in
public and private inpatient units.
Numerator: The number of clients who received involuntary inpatient
commitments during the reporting period.
Denominator: The total number of clients who received inpatient services during
the reporting period.
Provider Version of Measure 15: The percentage of clients under involuntary
commitments in inpatient units.
Numerator: The number of clients who received involuntary inpatient
commitments during the reporting period.
Denominator: The total number of clients who received inpatient services during
the reporting period.

Performance Indicator: Involuntary Outpatient Commitments

Authority and Provider Measure 16: The percentage of clients under involuntary
outpatient commitments.
Numerator: The number of clients who received involuntary outpatient
commitments during the reporting period.
Denominator: The total number of clients who received outpatient services during
the reporting period.

MHSIP’s Indicators on Seclusion

Authority Measure 17: Hours of seclusion as a percentage of client hours
Numerator: The total number of hours that all clients spent in seclusion.
Denominator: Sum of the daily census (excluding clients on leave status) for each
day (client days) multiplied by 24 hours.

Authority Measure 18: Percentage of clients secluded at least once during a reporting
period
Numerator: The total number of clients (unduplicated) who were secluded at least
once during a reporting period.
Denominator: The total number of unduplicated clients who were inpatients at the
facility during a reporting period.

MHSIP’s Indicators on Restraints

Authority Measure 19: Hours of restraint as a percentage of client hours
Numerator: The total number of hours that all clients spent in restraint during a
reporting period.
Denominator: Sum of the daily census (excluding clients on leave status) for each
day in a reporting period (client days) multiplied by 24 hours.

Authority Measure 20: Percentage of clients restrained at least once during the
reporting period
 Piloting the ROSI Measure Handout, August 03 2005, page 15

Numerator: The total number of clients (unduplicated) who were restrained at

least once during a reporting period.
Denominator: The total number of unduplicated clients who were inpatients at the
facility during the reporting period.

Formal Services Sub-Theme: Access to Services (involves the findings as to getting the
formal services that consumers feel they need and find helpful facilitates recovery).

MHSIP’s Proposed Indicator on Involvement in the Criminal/Juvenile Justice System

Add Authority Measure 21: The percentage of mental health catchment or service
areas that have jail diversion services.
Numerator: Total number of mental health catchment or service areas that have
jail diversion services.
Denominator: Total number of mental health catchment or service areas.
Provider Version of Measure 21: Jail diversion services are available in our
community for mental health consumers. (Yes/No)

MHSIP’s Proposed Indicator on Reduced Substance Abuse Impairment

Add Authority Measure 22: The percentage of mental health catchment or service
areas that have integrated substance abuse and mental health services.
Numerator: Total number of mental health catchment or service areas that have
integrated substance abuse and mental health services.
Denominator: Total number of mental health catchment or service areas.
Provider Version of Measure 22: Integrated substance abuse and mental health
services are available in our community for mental health consumers. (Yes/No)

Performance Indicator: Trauma Service Provision

Authority Measure 23: The percentage of mental health catchment or service areas
that have trauma services.
Numerator: Total number of mental health catchment or service areas that have
trauma services.
Denominator: Total number of mental health catchment or service areas.
Provider Version of Measure 23: Trauma services are available in our community for
mental health consumers. (Yes/No)
 The Evaluation Center @ HSRI

RECOVERY SELF-ASSESSMENT (RSA)

O’Connell, M., Tondora, J., Croog, G., Evans, A., & Davidson, L.

For more information contact:

Maria O’Connell, Ph.D.

Yale University School of Medicine
319 Peck Street, Building 6, Suite 1C
New Haven, CT 06519
Phone: (203) 764-7593
Email: maria.oconnell@yale.edu

Permission to use:
The RSA is not copyrighted. Permission is recommended but not required for use of the instrument. There is
not a user’s fee associated with the instrument.

Measuring the Promise: A Compendium of Recovery Measures, Volume II 244

 Code________________
Recovery Assessment: Person in Recovery Version

Please indicate the degree to which you feel the following items reflect the activities, values, and practices of your agency.

1 2 3 4 5
Strongly Disagree Strongly Agree

1. Staff focus on helping me to build connections in my neighborhood and community. 1 2 3 4 5 N/A

2. This agency offers specific services and programs to address my unique 1 2 3 4 5 N/A
culture, life experiences, interests, and needs.

3. I have access to all my treatment records. 1 2 3 4 5 N/A

4. This agency provides education to community employers about employing 1 2 3 4 5 N/A

people with mental illness and/or addictions.

5. My service provider makes every effort to involve my significant others (spouses, 1 2 3 4 5 N/A
friends, family members) and other sources of natural support (i.e., clergy, neighbors,
landlords) in the planning of my services, if this is my preference.

6. I can choose and change, if desired, the therapist, psychiatrist, or other service provider 1 2 3 4 5 N/A
with whom I work.

7. Most of my services are provided in my natural environment (i.e., home, community, 1 2 3 4 5 N/A
workplace).

8. I am given the opportunity to discuss my sexual and spiritual needs and interests. 1 2 3 4 5 N/A

9. Staff of this agency regularly attend trainings on cultural competency. 1 2 3 4 5 N/A

10. Staff at this agency listen to and follow my choices and preferences. 1 2 3 4 5 N/A

11. Staff at this agency help to monitor the progress I am making towards my 1 2 3 4 5 N/A
personal goals on a regular basis.

12. This agency provides structured educational activities to the community about mental 1 2 3 4 5 N/A
illness and addictions.

13. Agency staff do not use threats, bribes, or other forms of coercion to influence 1 2 3 4 5 N/A
my behavior or choices.

14. Staff at this agency encourage me to take risks and try new things. 1 2 3 4 5 N/A

15. I am/can be involved with facilitating staff trainings and education programs at this agency. 1 2 3 4 5 N/A

16. Staff are knowledgeable about special interest groups and activities in the community. 1 2 3 4 5 N/A

17. Groups, meetings, and other activities can be scheduled in the evenings or on weekends 1 2 3 4 5 N/A
so as not to conflict with other recovery-oriented activities such as employment or school.

Please continue on backÆ

O’Connell, Tondora, Evans, Croog, & Davidson (2003)
 1 2 3 4 5
Strongly Disagree Strongly Agree

18. This agency actively attempts to link me with other persons in recovery 1 2 3 4 5 N/A
who can serve as role models or mentors by making referrals to self-help, peer support,
or consumer advocacy groups or programs.

19. I am able to chose from a variety of treatment options at this agency (i.e., individual, 1 2 3 4 5 N/A
group, peer support, holistic healing, alternative treatments, medical).

20. The achievement of my goals is formally acknowledged and celebrated by the agency. 1 2 3 4 5 N/A

21. I am/can be routinely involved in the evaluation of the agency’s 1 2 3 4 5 N/A

programs, services, and service providers.

22. Staff use a language of recovery (i.e. hope, high expectations, respect) in everyday 1 2 3 4 5 N/A
conversations.

23. Staff play a primary role in helping me to become involved in non-mental health/addiction 1 2 3 4 5 N/A
related activities, such as church groups, special interest groups, and adult education.

24. If the agency can not meet my needs, procedures are in place to refer me to other 1 2 3 4 5 N/A
programs and services.

25. Staff actively assist me with the development of career and life goals that go beyond 1 2 3 4 5 N/A
symptom management and stabilization.

26. Agency staff are diverse in terms of culture, ethnicity, lifestyle, and interests. 1 2 3 4 5 N/A

27. I am/can be a regular member of agency advisory boards and management meetings. 1 2 3 4 5 N/A

28. At this agency, participants who are doing well get as much attention as those who are 1 2 3 4 5 N/A
having difficulties.

29. Staff routinely assist me in the pursuit of my educational and/or employment goals. 1 2 3 4 5 N/A

30. I am/can be involved with agency staff on the development and provision of 1 2 3 4 5 N/A
new programs and services.

31. Agency staff actively help me become involved with activities that give back to 1 2 3 4 5 N/A
my community (i.e., volunteering, community services, neighborhood watch/cleanup).

32. This agency provides formal opportunities for me, my family, service providers, 1 2 3 4 5 N/A
and administrators to learn about recovery.

33. The role of agency staff is to assist me, and other people in recovery with 1 2 3 4 5 N/A
fulfilling my individually-defined goals and aspirations.

34. Criteria for exiting or completing the agency were clearly defined and discussed 1 2 3 4 5 N/A
with me upon entry to the agency.

35. The development of my leisure interests and hobbies is a primary focus of my services. 1 2 3 4 5 N/A

36. Agency staff believe that I can recover and make my own treatment and life choices. 1 2 3 4 5 N/A

Thank You
 Code________________
Recovery Assessment: Family/Significant Other/Advocate Version
Please indicate the degree to which you feel the following items reflect the activities, values, and practices of the agency
from which you received this assessment.
1 2 3 4 5
Strongly Disagree Strongly Agree

1. Staff focus on helping people in recovery to build connections in their neighborhood 1 2 3 4 5 N/A
and community.

2. This agency offers specific services and programs to address the unique 1 2 3 4 5 N/A
culture, life experiences, interests, and needs of people in recovery.

3. People in recovery have access to all of their treatment records. 1 2 3 4 5 N/A

4. This agency provides education to community employers about employing 1 2 3 4 5 N/A

people with mental illness and/or addictions.

5. Service providers at this agency make every effort to involve significant others 1 2 3 4 5 N/A
(spouses, friends, family members) and other sources of natural support (i.e., clergy,
neighbors, landlords) in the planning of a person's services, if this is his/her preference.

6. People in recovery can choose and change, if desired, the therapist, psychiatrist, or other 1 2 3 4 5 N/A
service provider with whom they work.

7. Most services are provided in a person in recovery's natural environment (i.e., home, 1 2 3 4 5 N/A
community, workplace).

8. People in recovery are given the opportunity to discuss sexual and spiritual 1 2 3 4 5 N/A
needs and interests.

9. The staff of this agency regularly attend trainings on cultural competency. 1 2 3 4 5 N/A

10. Staff at this agency listen to and follow the choices and preferences expressed 1 2 3 4 5 N/A
by people in recovery.

11. Staff at this agency help to monitor the progress made towards a person in recovery's 1 2 3 4 5 N/A
personal goals on a regular basis.

12. This agency provides structured educational activities to the community about mental 1 2 3 4 5 N/A
illness and addictions.

13. Agency staff do not use threats, bribes, or other forms of coercion to influence 1 2 3 4 5 N/A
the behavior or choices of people in recovery.

14. Staff at this agency encourage people in recovery to take risks and try new things. 1 2 3 4 5 N/A

15. People in recovery are/can be involved with facilitating staff trainings and education 1 2 3 4 5 N/A
programs at this agency.

16. Staff are knowledgeable about special interest groups and activities in the community. 1 2 3 4 5 N/A

17. Groups, meetings, and other activities can be scheduled in the evenings or on weekends 1 2 3 4 5 N/A
so as not to conflict with other recovery-oriented activities such as employment or school.

18. This agency actively attempts to link people in recovery with others in recovery 1 2 3 4 5 N/A
who can serve as role models or mentors by making referrals to self-help, peer support,
or consumer advocacy groups or programs.

Please continue on backÆ

O’Connell, Tondora, Evans, Croog, & Davidson (2003)
 1 2 3 4 5
Strongly Disagree Strongly Agree

19. People in recovery can chose from a variety of treatment options at this agency 1 2 3 4 5 N/A
(i.e., individual, group, peer support, holistic healing, alternative treatments, medical).

20. The achievement of a person in recovery's goals is formally acknowledged and celebrated 1 2 3 4 5 N/A
by the agency.

21. People in recovery are/can be routinely involved in the evaluation of the agency’s 1 2 3 4 5 N/A
programs, services, and service providers.

22. Staff use a language of recovery (i.e. hope, high expectations, respect) in everyday 1 2 3 4 5 N/A
conversations.

23. Staff play a primary role in helping people in recovery to become involved in non- 1 2 3 4 5 N/A
mental health/addiction related activities, such as church groups, special interest groups,
and adult education.

24. If the agency can not meet a person in recovery's needs, procedures are in place to 1 2 3 4 5 N/A
refer him/her to other programs and services.

25. Staff actively assist people in recovery with the development of career and life 1 2 3 4 5 N/A
goals that go beyond symptom management and stabilization.

26. Agency staff are diverse in terms of culture, ethnicity, lifestyle, and interests. 1 2 3 4 5 N/A

27. People in recovery are/can be regular members of agency advisory boards 1 2 3 4 5 N/A
and management meetings.

28. At this agency, participants who are doing well get as much attention as 1 2 3 4 5 N/A
those who are having difficulties.

29. Staff routinely assist people in recovery with the pursuit of educational and/or 1 2 3 4 5 N/A
employment goals.

30. People in recovery can work along side agency staff on the development 1 2 3 4 5 N/A
and provision of new programs and services.

31. Agency staff actively help people become involved with activities that give back 1 2 3 4 5 N/A
to their community (i.e., volunteering, community services, neighborhood watch/cleanup).

32. This agency provides formal opportunities for people in recovery, family and significant 1 2 3 4 5 N/A
others, service providers, and administrators to learn about recovery.

33. The role of agency staff is to assist people in recovery with fulfilling their individually- 1 2 3 4 5 N/A
defined goals and aspirations.

34. Criteria for exiting or completing the programs are clearly defined and discussed 1 2 3 4 5 N/A
with people in recovery upon entry to the agency.

35. The development of a person in recovery's leisure interests and hobbies is a primary 1 2 3 4 5 N/A
focus of services.

36. Agency staff believe that people can recover and make their own treatment and life 1 2 3 4 5 N/A
choices.

Thank You
 Code________________
Recovery Self-Assessment: Provider Version

Please indicate the degree to which you feel the following items reflect the activities, values, and practices of your agency.

1 2 3 4 5
Strongly Disagree Strongly Agree

1. Helping people build connections with their neighborhoods and communities 1 2 3 4 5 N/A
is one of the primary activities in which staff at this agency are involved.

2. This agency offers specific services and programs for individuals with different 1 2 3 4 5 N/A
cultures, life experiences, interests, and needs.

3. People in recovery have access to all their treatment records. 1 2 3 4 5 N/A

4. This agency provides education to community employers about employing 1 2 3 4 5 N/A

people with mental illness and/or addictions.

5. Every effort is made to involve significant others (spouses, friends, family 1 2 3 4 5 N/A
members) and other natural supports (i.e., clergy, neighbors, landlords) in the planning
of a person's services, if so desired.

6. People in recovery can choose and change, if desired, the therapist, psychiatrist, or 1 2 3 4 5 N/A
other service provider with whom they work.

7. Most services are provided in a person's natural environment (i.e., home, community, 1 2 3 4 5 N/A
workplace).

8. People in recovery are given the opportunity to discuss their sexual and spiritual 1 2 3 4 5 N/A
needs and interests.

9. All staff at this agency regularly attend trainings on cultural competency. 1 2 3 4 5 N/A

10. Staff at this agency listen to and follow the choices and preferences of participants. 1 2 3 4 5 N/A

11. Progress made towards goals (as defined by the person in recovery) is monitored 1 2 3 4 5 N/A
on a regular basis.

12. This agency provides structured educational activities to the community about mental 1 2 3 4 5 N/A
illness and addictions.

13. Agency staff do not use threats, bribes, or other forms of coercion to influence 1 2 3 4 5 N/A
a person's behavior or choices.

14. Staff and agency participants are encouraged to take risks and try new things. 1 2 3 4 5 N/A

15. Persons in recovery are involved with facilitating staff trainings and education programs 1 2 3 4 5 N/A
at this agency.

16. Staff are knowledgeable about special interest groups and activities in the community. 1 2 3 4 5 N/A

17. Groups, meetings, and other activities can be scheduled in the evenings or on weekends 1 2 3 4 5 N/A
so as not to conflict with other recovery-oriented activities such as employment or school.

Please continue on backÆ

O’Connell, Tondora, Evans, Croog, & Davidson (2003)
 1 2 3 4 5
Strongly Disagree Strongly Agree
18. This agency actively attempts to link people in recovery with other persons in recovery 1 2 3 4 5 N/A
who can serve as role models or mentors by making referrals to self-help, peer support,
or consumer advocacy groups or programs.

19. This agency provides a variety of treatment options (i.e., individual, group, 1 2 3 4 5 N/A
peer support, holistic healing, alternative treatments, medical) from which agency
participants may choose.

20. The achievement of goals by people in recovery and staff are formally acknowledged 1 2 3 4 5 N/A
and celebrated by the agency.

21. People in recovery are routinely involved in the evaluation of the agency’s programs, 1 2 3 4 5 N/A
services, and service providers.

22. Staff use a language of recovery (i.e. hope, high expectations, respect) in everyday 1 2 3 4 5 N/A
conversations.

23. Staff play a primary role in helping people in recovery become involved in non- 1 2 3 4 5 N/A
mental health/addiction related activities, such as church groups, special interest groups, and
adult education.

24. Procedures are in place to facilitate referrals to other programs and services if the 1 2 3 4 5 N/A
agency cannot meet a person's needs.

25. Staff actively assist people in recovery with the development of career and life 1 2 3 4 5 N/A
goals that go beyond symptom management and stabilization.

26. Agency staff are diverse in terms of culture, ethnicity, lifestyle, and interests. 1 2 3 4 5 N/A

27. People in recovery are regular members of agency advisory boards and management 1 2 3 4 5 N/A
meetings.

28. At this agency, participants who are doing well get as much attention as those who are 1 2 3 4 5 N/A
having difficulties.

29. Staff routinely assist individuals in the pursuit of educational and/or employment goals. 1 2 3 4 5 N/A

30. People in recovery work along side agency staff on the development and 1 2 3 4 5 N/A
provision of new programs and services.

31. Agency staff actively help people become involved with activities that give back to 1 2 3 4 5 N/A
their communities (i.e., volunteering, community services, neighborhood watch/cleanup).

32. This agency provides formal opportunities for people in recovery, family members 1 2 3 4 5 N/A
service providers, and administrators to learn about recovery.

33. The role of agency staff is to assist a person with fulfilling their individually-defined 1 2 3 4 5 N/A
goals and aspirations.

34. Criteria for exiting or completing the agency are clearly defined and discussed 1 2 3 4 5 N/A
with participants upon entry to the agency.

35. The development of a person's leisure interests and hobbies is a primary focus of services. 1 2 3 4 5 N/A

36. Agency staff believe that people can recover and make their own treatment and life choices. 1 2 3 4 5 N/A

Thank You
 Code________________
Recovery Self-Assessment: CEO/ Agency Director Version

Please indicate the degree to which you feel the following items reflect the activities, values, and practices of your agency.

1 2 3 4 5
Strongly Disagree Strongly Agree

1. Helping people build connections with their neighborhoods and communities 1 2 3 4 5 N/A
is one of the primary activities in which staff at this agency are involved.

2. This agency offers specific services and programs for individuals with different 1 2 3 4 5 N/A
cultures, life experiences, interests, and needs.

3. People in recovery have access to all their treatment records. 1 2 3 4 5 N/A

4. This agency provides education to community employers about employing 1 2 3 4 5 N/A

people with mental illness and/or addictions.

5. Every effort is made to involve significant others (spouses, friends, family 1 2 3 4 5 N/A
members) and other natural supports (i.e., clergy, neighbors, landlords) in the planning
of a person's services, if so desired.

6. People in recovery can choose and change, if desired, the therapist, psychiatrist, or 1 2 3 4 5 N/A
other service provider with whom they work.

7. Most services are provided in a person's natural environment (i.e., home, community, 1 2 3 4 5 N/A
workplace).

8. People in recovery are given the opportunity to discuss their sexual and spiritual 1 2 3 4 5 N/A
needs and interests.

9. All staff at this agency regularly attend trainings on cultural competency. 1 2 3 4 5 N/A

10. Staff at this agency listen to and follow the choices and preferences of participants. 1 2 3 4 5 N/A

11. Progress made towards goals (as defined by the person in recovery) is monitored 1 2 3 4 5 N/A
on a regular basis.

12. This agency provides structured educational activities to the community about mental 1 2 3 4 5 N/A
illness and addictions.

13. Agency staff do not use threats, bribes, or other forms of coercion to influence 1 2 3 4 5 N/A
a person's behavior or choices.

14. Staff and agency participants are encouraged to take risks and try new things. 1 2 3 4 5 N/A

15. Persons in recovery are involved with facilitating staff trainings and education programs 1 2 3 4 5 N/A
at this agency.

16. Staff are knowledgeable about special interest groups and activities in the community. 1 2 3 4 5 N/A

17. Groups, meetings, and other activities can be scheduled in the evenings or on weekends 1 2 3 4 5 N/A
so as not to conflict with other recovery-oriented activities such as employment or school.

Please continue on backÆ

O’Connell, Tondora, Evans, Croog, & Davidson (2003)
 1 2 3 4 5
Strongly Disagree Strongly Agree

18. This agency actively attempts to link people in recovery with other persons in recovery 1 2 3 4 5 N/A
who can serve as role models or mentors by making referrals to self-help, peer support,
or consumer advocacy groups or programs.

19. This agency provides a variety of treatment options (i.e., individual, group, 1 2 3 4 5 N/A
peer support, holistic healing, alternative treatments, medical) from which agency
participants may choose.

20. The achievement of goals by people in recovery and staff are formally acknowledged 1 2 3 4 5 N/A
and celebrated by the agency.

21. People in recovery are routinely involved in the evaluation of the agency’s programs, 1 2 3 4 5 N/A
services, and service providers.

22. Staff use a language of recovery (i.e. hope, high expectations, respect) in everyday 1 2 3 4 5 N/A
conversations.

23. Staff play a primary role in helping people in recovery become involved in non- 1 2 3 4 5 N/A
mental health/addiction related activities, such as church groups, special interest groups, and
adult education.

24. Procedures are in place to facilitate referrals to other programs and services if the 1 2 3 4 5 N/A
agency cannot meet a person's needs.

25. Staff actively assist people in recovery with the development of career and life 1 2 3 4 5 N/A
goals that go beyond symptom management and stabilization.

26. Agency staff are diverse in terms of culture, ethnicity, lifestyle, and interests. 1 2 3 4 5 N/A

27. People in recovery are regular members of agency advisory boards and management 1 2 3 4 5 N/A
meetings.

28. At this agency, participants who are doing well get as much attention as those who are 1 2 3 4 5 N/A
having difficulties.

29. Staff routinely assist individuals in the pursuit of educational and/or employment goals. 1 2 3 4 5 N/A

30. People in recovery work along side agency staff on the development and 1 2 3 4 5 N/A
provision of new programs and services.

31. Agency staff actively help people become involved with activities that give back to 1 2 3 4 5 N/A
their communities (i.e., volunteering, community services, neighborhood watch/cleanup).

32. This agency provides formal opportunities for people in recovery, family members 1 2 3 4 5 N/A
service providers, and administrators to learn about recovery.

33. The role of agency staff is to assist a person with fulfilling their individually-defined 1 2 3 4 5 N/A
goals and aspirations.

34. Criteria for exiting or completing the agency are clearly defined and discussed 1 2 3 4 5 N/A
with participants upon entry to the agency.

35. The development of a person's leisure interests and hobbies is a primary focus of services. 1 2 3 4 5 N/A

36. Agency staff believe that people can recover and make their own treatment and life choices. 1 2 3 4 5 N/A
Thank You