Hunsley 2005
Hunsley 2005
The goal of this special section is to encourage greater awareness of evidence-based assessment (EBA)
in the development of a scientifically supported clinical psychology. In this introductory article, the
authors describe the elements that authors in this special section were asked to consider in their focused
reviews (including the scope of available psychometric evidence, advancements in psychopathology
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
research, and evidence of attention to factors such as gender, age, and ethnicity in measure validation).
This document is copyrighted by the American Psychological Association or one of its allied publishers.
The authors then present central issues evident in the articles that deal with anxiety, depression,
personality disorders, and couple distress and in the accompanying commentaries. The authors conclude
by presenting key themes emerging from the articles in this special section, including gaps in psycho-
metric information, limited information about the utility of assessment, the discrepancy between recom-
mended EBAs and current training and practice, and the need for further data on the process of clinical
assessment.
The past decade has witnessed a rapidly growing emphasis on assessment (EBA) has been largely overlooked. Even when as-
evidence-based approaches to the provision of services within sessment guidelines are presented side by side with those for
health and human service systems such as medical health care, treatment, they tend to be brief and underdeveloped relative to the
mental and behavioral health care, social work, education, and treatment guidelines (e.g., National Institute for Clinical Excel-
criminal justice (Barlow, 2004; Mullen & Streiner, 2004). Simply lence, 2004). Indeed, without scientifically sound assessment data,
put, evidence-based practice involves the use of an amalgamation it is impossible to determine whether a treatment, patient charac-
of systematically collected data, clinical expertise, and patient teristic, or therapeutic relationship variable has any impact on
preferences by decision makers (including practitioners, managers, patient functioning. To address this issue and to promote the
and policy makers) when considering services options for, at one dissemination of evidence-based treatments, some psychologists
extreme, individual patients or, at the other extreme, for national have argued that assessment data from measures with established
populations (e.g., Institute of Medicine, 2001; Sackett, Rosenberg, reliability and validity must be used in the evaluation of conditions
Gray, Haynes, & Richardson, 1996). Within American profes- for which treatment is sought and in the evaluation of the outcome
sional psychology, the move toward encouraging the explicit con- of treatment (e.g., Chambless & Hollon, 1998; Kazdin, Kratoch-
sideration of empirical evidence in service provision activities is will, & VandenBos, 1986; Ollendick, 2003). Taking this a step
best typified by the treatment-oriented task force initiatives devel- further, Weisz, Chu, and Polo (2004) recently recommended that
oped by several divisions of the American Psychological Associ-
evidence-based practice should be considered as an assessment–
ation (e.g., Chambless & Ollendick, 2001; Norcross, 2001).
intervention dialectic involving the accurate identification of initial
Given the long-standing emphasis on the centrality of accurate
treatment targets, the selection of the most appropriate evidence-
measurement for any scientific endeavor in psychology, it is sur-
based treatment for these targets, and periodic assessment of the
prising that evidence-based initiatives within psychology began
treatment to determine whether any adjustments to treatment are
with a focus on treatment rather than on assessment. This is most
necessary (i.e., assess–treat–reassess–adjust treatment).
likely due to the greater relative involvement by psychologists in
The first goal of this special section is to highlight the need to
offering therapeutic services and the greater relative professional
valuing of treatment over assessment activities. In the pursuit of explicitly consider the role of EBA in the larger context of
enhancing the quality of psychological interventions through the evidence-based psychological practice. Later in this introduction to
conscious application of empirical evidence, it is rather ironic that the special section, we provide an overview of the excellent
the importance of defining what might constitute evidence-based contributions in this special section that further our knowledge of
EBAs for adult disorders. These articles and commentaries, in
conjunction with a parallel special section on the EBAs for child
and adolescent disorders (Mash & Hunsley, 2005a), provide thor-
John Hunsley, School of Psychology, University of Ottawa, Ottawa,
ough summaries of the scientific literature for a number of disor-
Ontario, Canada; Eric J. Mash, Department of Psychology, University of
Calgary, Calgary, Alberta, Canada. ders and problems and pose several challenges that must be ad-
Correspondence concerning this article should be addressed to John dressed if clinical psychology is to have a truly evidence-based set
Hunsley, School of Psychology, University of Ottawa, Ottawa, Ontario of assessment practices. At this point, though, we turn to a
K1N 6N5, Canada. E-mail: hunch@uottawa.ca consideration of the second goal of the special section: to focus
251
252 SPECIAL SECTION: INTRODUCTION
attention on the need to explicitly define what constitutes an at some future point, the scientific standing of the assessment
EBA. process itself. Further details on issues related to our conceptual-
ization of EBA can be found in Hunsley, Crabb, and Mash (2004)
EBA and Mash and Hunsley (2005b)
The term evidence-based assessment is used in the scientific The Special Section: Elements of EBAs
literature in a variety of ways. The term has been used in the
context of evaluating the appropriateness or quality of routinely Given the nascent state of efforts to operationalize EBA, we
provided health care treatments in comparison with the empirical believed that it was premature for authors contributing to this
evidence regarding the best treatment options for a specific con- special section to follow strict, predetermined criteria in consider-
dition. For example, Rascol, Goetz, Koller, Poewe, and Sampaio ing the nature of EBA in their respective areas. We did not, for
(2002) described their systematic review of the efficacy and safety example, set out criteria for the extent and type of validity evi-
of different interventions available for the management of Parkin- dence necessary for a measure to be deemed to be valid for a
son’s disease as an EBA. Likewise, Ansell, Watson, and Fogelman specific assessment purpose. We did, however, provide authors for
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
(1999) described as an EBA their efforts to compare established this special section with a set of elements to consider in preparing
practice guidelines for the treatment of high cholesterol with data their reviews. These included the following points.
available from large-scale clinical trials conducted after the dis- First, psychometric evidence for a measure or an assessment
semination of the guidelines. In contrast, Corrigan (2002) used the strategy is always conditional, as reliability, validity, and utility
same term to describe the strategy of diagnosing gastroesophageal indices are dependent on the nature of the clinical sample and
reflux disease with an assessment protocol that had solid evidence criterion variable under consideration (cf. Hunsley & Meyer,
of reliability and validity. This is closer to the meaning that most 2003). Accordingly, EBAs should be considered to be disorder or
psychologists would typically understand of EBA and is similar to problem specific. This presents somewhat of a conundrum in that
the term empirically based measures, adopted by authors of recent one important purpose of assessment is to identify the nature of the
volumes on the assessment of anxiety and depression (Antony, problem(s). Thus, as has been recommended by many others, it
Orsillo, & Roemer, 2001; Nezu, McClure, Ronan, & Meadows, may be necessary to conceptualize multiple stages wherein guide-
2000). lines for initial assessments are problem nonspecific but would
There is yet another manner in which the term has been defined, become increasingly problem specific as the assessment focus is
one that includes the standard psychometric indices of reliability refined. Thus, EBAs need to be embedded in the purposes of
and validity but goes beyond to include utility considerations (e.g., assessment, including screening, diagnosis, prognosis, treatment
Cohen & Parkman, 1998). The clinical utility of assessment en- planning, treatment monitoring, and treatment evaluation. The
compasses treatment utility (i.e., the degree to which clinical criteria for the empirical support necessary for a measure to be
assessment data contribute to positive treatment outcomes; considered an EBA might differ as a function of purpose.
Nelson-Gray, 2003), diagnostic utility (i.e., the degree to which the Second, for each disorder or problem, there needs to be a clear
assessment data contribute to the formulation of an accurate and statement of the state of the psychopathology literature. This
complete diagnosis), and a range of additional factors such as includes indications of the main constructs that theory and research
assessment-related costs, the improvement in typical clinical de- have established as important for a particular disorder or problem,
cision making due to the assessment, alterations in the rates of key symptoms to assess, as well as common comorbid conditions
false positives and false negatives associated with the assessment and associated features that need to be examined. It should also
(on the basis of sensitivity and specificity indices), and the eco- include indications and methods for assessing broader life context
nomic and psychological costs associated with these errors (Hun- factors consistently found to be relevant to the functioning of
sley, 2003). It is this broadened definition of EBA, including individuals with a particular disorder (e.g., quality of life, family
reliability, validity, and utility considerations, that we adopted for factors, academic or work functioning, health care utilization,
this special section. relational adjustment). Assessing these areas allows for a compre-
In defining EBA, it is also important to acknowledge that hensive evaluation that could guide clinical service decisions and
psychological assessment is an iterative decision-making process serve as a baseline for determining treatment effectiveness. Of
that goes beyond the simple utilization of a set of assessment course, in attempting to obtain a comprehensive set of assessment
measures. Ideally, then, EBA should target integrated assessment data, psychologists must recognize that the amount and extent of
activities, such as formal or informal guidelines for obtaining and such information needed for providing appropriate services is,
integrating data from multiple informants and multiple measures, ultimately, an empirical question (cf. Garb, 1998).
not just individual tests or interview protocols. To our knowledge, Third, EBAs need to be sensitive to gender, ethnicity, and
though, there are no integrated assessment strategies that could be cultural factors. Scientific evidence for the applicability of assess-
considered to be evidence-based. There are, of course, assessment ment tools needs to be demonstrated, not simply assumed, on the
guidelines that are empirically derived (for an example, see Sny- basis of generalizations from nonrepresentative samples.
der, Heyman, & Haynes, 2005); however, at present, there are no Fourth, there must be psychometric evidence that the assessment
data that address whether the guidelines themselves are psycho- instruments used as part of a more general assessment strategy are
metrically sound and have appreciable clinical utility. Accord- reliable and valid (concurrent and discriminant validity especially).
ingly, at this point in time, a pragmatic approach to EBA involves, In developing criteria for EBAs, specific required values need to
of necessity, a focus on discrete assessment tools. It is critical, be proposed for various types of assessment instruments, and
though, that psychologists remain cognizant of the need to address, evidence needs to be presented from at least two independent
SPECIAL SECTION: INTRODUCTION 253
published studies (i.e., requires at least some replication). In pre- Joiner, Walker, Pettit, Perez, and Cukrowicz (2005) address the
senting psychometric evidence, it is important to recognize that issue of assessing depression in adult patients. Like Antony and
formal instruments are only one component of the assessment Rowa (2005), these authors touch on the minimum criteria neces-
process and that other components, and the entire process itself, sary for EBAs and how the purpose of assessment (e.g., screening
need to be evaluated as well. vs. treatment monitoring) influences the choice of preferred as-
Fifth, because of the present state of the research literature, sessment methods. With respect to the assessment of depressive
evidence for the utility and incremental validity of assessment symptoms, Joiner et al. stress that the evaluation of depressed
tools and strategies is valuable but not compulsory. Currently there mood, anhedonia, and suicidality (including distinguishing be-
is insufficient research on these elements in most clinical problem tween suicidal ideation and suicidal plans) are critical. Further-
areas. However, our hope is that the eventual sine qua non for any more, on the basis of extensive research evidence, these authors
assessment tool strategy is in its utility in helping to bring about make a convincing case for the need to also assess subtypes of
clinical change. depression, the chronicity and course of the depression, and com-
Finally, there needs to be an acknowledgment that although each mon comorbid conditions (especially bipolar disorder, anxiety
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
strategy may be empirically supported, there may be little evidence disorders, eating disorders, substance use disorders, and personal-
This document is copyrighted by the American Psychological Association or one of its allied publishers.
on the reliability and validity of clinician judgment for how to ity disorders). Despite the voluminous literature on the psycholog-
organize the resulting data into a coherent and clinically useful ical assessment of depression, they also emphasize that much more
evaluation. Long-standing questions regarding the reliability and research is needed to establish the utility of psychological assess-
validity of clinical decision-making algorithms in the field of ment efforts in the context of depression.
assessment will likely require discussion in the current context (cf. Widiger and Samuel’s (2005) contribution addresses the chal-
Garb, 1998). When the primary assessment focus is the pretreat- lenges associated with the EBA of personality disorders. A central
ment identification of problem areas and the establishment of focus in their article is the need to balance a thorough and accurate
baselines, this may not be a major concern because the psycholo- assessment with the limited time most psychologists are likely to
gist has opportunities to refine and correct any errors in inference. have to conduct such an assessment. Accordingly, after reviewing
It is more of a concern, however, when the entire clinical service the psychometric status of major self-report measures and semi-
is the assessment itself. In such circumstances, the duration of structured interviews, they recommend a two-stage assessment
clinical contact with the patient is relatively brief, which limits the process be adopted. The first stage involves the administration of
opportunities to obtain further information that could lead to a self-report inventory; if maladaptive personality traits are iden-
alterations in the psychologist’s clinical formulation. Without an tified, this should be followed with a semistructured interview to
appreciation of the limitations to clinical judgment, it is possible determine the presence, nature, and severity of any possible per-
that the conclusions drawn from the assessment by the psycholo- sonality disorders. As they sagely note, though, the use of estab-
gist may become reified when used by other professionals. lished semistructured interviews must involve careful attention to
the influence of age of onset of the personality problem, gender
biases, cultural and ethnic factors, and probable inaccuracies in
Contributions to the Special Section
patient self-perception and presentation. A significant gap in the
The special section begins with Antony and Rowa’s (2005) EBA of personality disorders that Widiger and Samuels identify is
contribution on the assessment of adult anxiety disorders. The that existing assessment tools were not designed to be sensitive to
authors present two main lines of argument in their article. The changes in personality functioning. A direct implication is that it
first deals specifically with the domains of assessment that are should be a priority for personality disorder researchers to develop
critical in assessing anxiety disorders. As they note, most patients new tools for the monitoring and evaluation of treatments for these
with anxiety symptoms present with many problems and, as a common but vexing clinical conditions.
result, may either partially or fully meet criteria for several diag- Evidence-based approaches to assessing couple distress are the
noses. To facilitate a comprehensive assessment that is optimally focus of the article by Snyder et al. (2005). On the basis of
useful for treatment planning and implementation, Antony and extensive research on couple functioning and distress, the authors
Rowa strongly recommend that much more than the patient’s propose a conceptual framework for couple-based assessment
diagnostic status must be evaluated. Specifically, they recommend strategies that included both individual and dyadic characteristics
that assessment incorporate an evaluation of anxiety triggers and that have been found to be implicated in relationship problems. On
cues (situational, interoceptive, and cognitive), avoidance behav- the basis of this framework, they describe assessment tools for
iors, compulsions and overprotective behaviors, physical symp- assessing variables such as each partner’s distress and each part-
toms and responses, skill deficits, associated distress and func- ner’s relationship-related behaviors, cognitions, and affect. Cutting
tional impairment, development and course of the clinical across several assessment methods, including interviews, observa-
problems, treatment history, environmental and family factors, tions, self-report, and informant report, Snyder et al. highlight both
general medical and health issues, and common comorbid disor- the availability of existing measures and the gaps in our knowledge
ders (including mood disorders and personality disorders). Beyond of the psychometric adequacy of these measures. They conclude
the issue of what constitutes EBA for anxiety disorders, Antony their article with several important evidence-based recommenda-
and Rowa also address the issue of what exactly constitutes EBA. tions for assessing patients who are in intimate relationships,
Their insightful comments on this, along with their cautions about including (a) the routine assessment of couple functioning when
possible obstacles to the dissemination of EBA, do much to elu- treating individuals and (b) the routine assessment of domains that
cidate the challenges associated with developing EBAs. are known to be strongly linked to relationship conflict and those
254 SPECIAL SECTION: INTRODUCTION
domains that have particularly adverse effects on couple function- Murray, 2003). Even if such information is available, there is
ing (such as physical aggression and substance abuse). almost no literature that considers the extent to which inclusion of
To round out this special section, there are commentaries from a measure consistently improves upon clinical decision making
two respected, well-informed experts on psychological assess- and/or the outcome of any clinical service. Such data are relevant
ment. Drawing on the perspective of international efforts to pro- to all measures but are particularly germane to decisions to use
mote evidence-based health care practice, Barlow (2005) examines time-consuming or costly assessment tools. For example, if re-
the role EBA should have in modern health care services. He also search demonstrated that the early accurate identification of Axis
emphasizes the ways in which EBA is associated with current II disorders with a semistructured interview led to better treatment
integrated models of psychopathology, which are also directly outcomes and less premature termination of services, then a very
responsible for recent developments in psychological treatments strong case could be made for the use of such an interview with
that are both efficacious and effective. In his commentary, McFall patients who, after screening, are suspected of having a personality
(2005) examines EBA from the perspective of conceptual factors disorder.
that underlie measurement efforts in psychology in both clinical
and research domains. The importance of (a) understanding the
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
presented in test manuals are all that are necessary for assessment logical testing and assessment: Conceptual, methodological, and statis-
to be evidence-based is akin to, as the character Gollum does in the tical issues. Psychological Assessment, 15, 446 – 455.
movie The Lord of the Rings: The Two Towers (Jackson, 2002), Institute of Medicine. (2001). Crossing the quality chasm: A new health
covering our ears and repeatedly saying “Not listening, not listen- system for the 21st century. Washington, DC: National Academy Press.
Jackson, P. (Director). (2002). The lord of the rings: The two towers
ing!” to drown out our doubts and confusion.
[Motion picture]. United States: New Line Cinema.
Establishing the utility of an assessment measure, for example,
Johnston, C., & Murray, C. (2003). Incremental validity in the psycholog-
may not be as glamorous as evaluating a new and promising ical assessment of children and adolescents. Psychological Assessment,
intervention, but it is just as critical to the delivery of high-quality 15, 496 –507.
clinical services. Without attention to EBA, the promotion and Joiner, T. E., Walker, R. L., Pettit, J. W., Perez, M., & Cukrowicz, K. C.
dissemination of evidence-based treatments has been likened to (2005). Evidence-based assessment of depression in adults. Psycholog-
building a magnificent house without bothering to construct the ical Assessment, 17, 267–277.
foundation (Achenbach, 2005). With a century of conceptual and Kazdin, A. E., Kratochwill, T. R., & VandenBos, G. R. (1986). Beyond
practical developments in psychological assessment behind us, we clinical trials: Generalizing from research to practice. Professional Psy-
have the requisite scientific knowledge and sophistication to en- chology: Research and Practice, 17, 391–398.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Mash, E. J., & Hunsley, J. (Eds.). (2005a). Developing guidelines for the
This document is copyrighted by the American Psychological Association or one of its allied publishers.