AJSLP

Viewpoint

Patient-Reported Outcomes and

Evidence-Based Practice in
Speech-Language Pathology
Matthew L. Cohena and William D. Hulab

Purpose: The patient’s perspective of their health is a core raises questions related to PRO-informed clinical practice
component of evidence-based practice (EBP) and person- in speech-language pathology. To address some of these
centered care. Patient-reported outcomes (PROs), captured questions, the article explores previous research to provide
with PRO measures (PROMs), are the main way of formally suggestions for clinical administration, interpretation, and
soliciting and measuring the patient’s perspective. Currently, future research.
however, PROs play a relatively small role in mainstream Conclusion: More routine measurement of subjective health
speech-language pathology practice. The purpose of this constructs via PROMs—for example, constructs such as
article is to raise important questions about how PROs effort, participation, self-efficacy, and psychosocial functioning
could be applied to EBP in speech-language pathology —may improve EBP. More routine use of PROMs could
for individuals with communication disorders and to significantly expand the information that is available to
propose preliminary approaches to address some of these clinicians about individual clients and add to the evidence
questions. base for the profession of speech-language pathology.
Method: Based on a narrative review of the literature, this However, careful consideration and more research are needed
article introduces relevant terminology and broadly describes on how to capture and interpret PROs from individuals with
PRO applications in other health care fields. The article also cognitive and language disorders.

When you can measure what you are speaking about about a particular client (“internal evidence”) and his or
and express it in numbers, you know something about her preferences. Data may be collected to inform the pro-
it. When you cannot express it in numbers, your cess of EBP, to evaluate the ultimate outcome of it, or both.
knowledge is of a meagre and unsatisfactory kind. Evidence is not limited to those things that can be quanti-
Therefore, “to measure is to know” and “if you fied although quantifiable data from standardized measures
cannot measure it, you cannot improve it.” are a relatively rigorous form of evidence.
-Lord Kelvin, as cited by Mayo et al.(2017) The 2013 Health Care Survey by the American
Speech-Language-Hearing Association (ASHA) revealed

E
vidence-based practice (EBP) requires the speech-
language pathologist (SLP) to gather, interpret, that only 40% of health care–based SLPs use standardized
integrate, and act on data. These data may include assessments as “internal evidence” of clinical outcomes
generalized information about clinical populations, which (ASHA, 2013). The most commonly used measures (84.7%
Dollaghan (2007) calls “external evidence,” as well as data of responses) were the Functional Independence Measure
(FIM; Granger et al., 1986) and ASHA’s Functional Com-
munication Measures (FCMs), the latter of which are part
of the National Outcomes Measurement System (NOMS;
a
Department of Communication Sciences and Disorders and Center ASHA, n.d.). Notably, the FIM and the FCMs are clinician-
for Health Assessment Research and Translation, University of reported outcome measures. The 2013 survey respondents
Delaware, Newark reported using a wide range of other measures as well, 29%
b
Geriatric Research, Education, and Clinical Center, VA Health Care of which had a performance-based component, 40% of which
System, and Department of Communication Sciences and Disorders,
had a clinician-reported component, 29% of which had an
University of Pittsburgh, PA
observer-reported component, and 17% of which had a
Correspondence to Matthew Cohen: mlcohen@udel.edu
patient-reported component.
Editor-in-Chief: Julie Barkmeier-Kraemer
Received June 26, 2019
Revision received August 16, 2019
Accepted October 21, 2019 Disclosure: The authors have declared that no competing interests existed at the time
https://doi.org/10.1044/2019_AJSLP-19-00076 of publication.

American Journal of Speech-Language Pathology • Vol. 29 • 357–370 • February 2020 • Copyright © 2020 The Authors 357
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions
 In general, performance-based and clinician-reported for Medicare & Medicaid Services; Rao, 2015) for health
outcome measures capture “behavior”—the external and care professionals to demonstrate the value of their services.
observable manifestations of conditions—and only the sub- Indeed, 65% of SLPs who responded to ASHA’s 2015
set of behaviors that are observable by the clinician in his Health Care Survey reported that they felt more pressure
or her particular treatment setting. Of course, for many of than in the previous year to demonstrate the value of
the conditions that SLPs treat, observable behavior only their services to administrators, other clinicians, or payers.
reflects part of the communication condition. For example, When asked, “Which of the following resources [choose all
a cognitive disorder may result in observable communica- that apply] would assist you in demonstrating your value?”
tion “errors,” but likely also involves unobservable elements respondents most often (62.8%) endorsed “outcomes data
such as communication “effort,” “difficulty,” and “confi- that demonstrate improvement resulting from speech-
dence” (see Figure 1). Similarly, a swallowing condition language pathology services.” In contrast, only 33.4%
may result in observable kinematics, but may also be asso- endorsed “educational sessions on how to negotiate with
ciated with unobservable elements such as fear of choking payers/administrators.” This suggests that SLPs want new
and social embarrassment. These unobservable outcomes and better evidence more than they want to improve the
can be measured with patient-reported outcome measures application of existing evidence. Accordingly, ASHA re-
(PROMs). ASHA’s 2013 Health Care Survey indicated cently reported that one of their strategic initiatives until
that very few SLPs are using PROMs, which means that at least 2025 is to “expand data available for quality im-
unobservable treatment effects are generally not being provement and demonstration of value” (ASHA, 2018),
measured. As an illustrative example, a 2016 survey found particularly through the addition of PROMs to the NOMS
that most SLPs used participation-related themes when to complement the FCMs (Rogers, 2018; Yorkston, 2015).
describing their rationale for treatment in simulated case Addition of PROMs will lead to not just “more” data but
scenarios. However, few of those SLPs “measured” partici- also “different” data that are useful as internal evidence for
pation, mainly because they did not know which measures EBP and for demonstrating the value of speech-language
to use and because they encountered logistical constraints pathology services.
such as time limitations (Torrence et al., 2016). The FCMs have been very useful as evidence for the
Measurement is important for EBP because it pro- profession, but they are of limited use as internal evidence
vides a rigorous, common language for thinking and com- for EBP. In addition to being limited in scope by measur-
municating about important health concepts, particularly ing only what is observable to the clinician, the FCMs are
with regard to identification/diagnosis of health conditions, psychometrically limited because each FCM comprises a
comparing health states across individuals, matching indi- single-item ordinal scale. Among other things, this means a
viduals to treatments, monitoring progress, and evaluating change from Level 5 to Level 6 is not necessarily the same
outcomes. “Outcomes should reflect the reason that an in- as from Level 6 to Level 7. The interpretation of changing
dividual seeks healthcare services” (National Quality Forum, ranks is therefore of minimal clinical usefulness to the
2009), and many individuals seek speech-language therapy clinician or the client. Furthermore, single-item scales typi-
for desired outcomes that are unobservable. PROMs are the cally provide lower measurement precision than multi-
most rigorous way of measuring those outcomes. item scales, and methods for quantifying the precision of
Failure to measure important variables is a signifi- individual score estimates generally require multiple items.
cant missed opportunity, not only for the delivery of EBP In contrast, PROMs are more sophisticated psychometri-
but also for the clinician and the profession. Although the cally, produce scores on an interval measurement scale,
full effectiveness of speech-language therapy is largely un- and are better suited for evaluating change. The reader is
documented, there is rising pressure (e.g., from the Centers referred to Yorkston and Baylor (2019) for an excellent
overview of PROMs for clinicians, including steps involved
in developing modern PROMs and examples of PROMs
intended for use with individuals with communication
Figure 1. Examples of observable and unobservable outcomes. disorders.
Many health disorders that are treated by speech-language
pathologists have both observable and unobservable components. The field of speech-language pathology is on the cusp
The unobservable components are most directly assessed by of embracing and integrating patient-reported outcomes
patient-reported outcome (PRO) measures. (PROs) in a significant way, led in large part by ASHA’s
interest in bolstering evidence for the profession through
the NOMS. However, there is also a significant opportunity
for PROs to strongly inform the process of EBP by pro-
viding more and different internal evidence about particular
clients. The remainder of this article provides clarifying
terminology surrounding PROs, a brief description of
modern PRO assessment and cross-discipline measurement
initiatives, and several guiding questions to be considered
as the field of speech-language pathology integrates PRO
assessment into its EBP.

358 American Journal of Speech-Language Pathology • Vol. 29 • 357–370 • February 2020

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Terminology whether elicited or spontaneous. The WHO International
Classification of Functioning, Disability and Health (ICF)
The World Health Organization (WHO; 1947) defines distinguishes between behavior elicited in a standardized
health as “a state of complete physical, social, and mental way or in a controlled environment and behavior that occurs
well-being, and not merely the absence of disease or infirmity” spontaneously. For example, a child client’s mean length
(as cited in Mayo et al., 2017). A PRO is health-related of utterance elicited in clinic is an example of “capacity”
information that comes directly from the patient without (what the client “can” do), whereas that same behavior
modification by clinicians or other health care professionals captured in the client’s normal environment (e.g., mean
(U.S. Department of Health and Human Services FDA length of utterance captured by Language Environment
Center for Drug Evaluation and Research et al., 2009). This Analysis) is an example of performance (what the client
information may be about health-related quality of life, “does” do; Threats, 2012).
functional status, symptoms and symptom burden, health
behaviors, or experiences of care (Cella et al., 2015; U.S.
Department of Health and Human Services FDA Center
for Drug Evaluation and Research et al., 2009). A PROM is Modern PRO Assessment
simply a standardized questionnaire designed to collect data Historically, the outcomes of health care have been
from patients on one or more PROs. Some authors have discussed as being distinct from the structure or function of
elaborated that a true PRO is one for which the patient’s health care (Donabedian, 1966, 1988). Traditionally, outcomes
perspective is the only valid perspective; for example, have been measured with relatively coarse metrics, for ex-
outcomes such as perception of difficulty, satisfaction, in- ample, rates of rehospitalization or death. However, the
tensity, impact, or bother (Mayo et al., 2017). In contrast, growing influence of the WHO ICF has contributed to the
a self-reported outcome is one for which another source of broadening of definitions of health outcomes (Portney &
information is complementary, such as measures of fre- Watkins, 2015) to include more constructs that are un-
quency, duration, physical appearance, or performance. For observable and subjective, such as quality of life or partici-
example, a client may provide a self-reported outcome of “se- pation, and which are difficult or impossible to fully assess
verity” of forgetfulness (which could also be measured by with performance-based assessment instruments.
other means), or a PRO of “distress” about forgetfulness Around the same time as the publication of the current
(for which the client’s perspective is the only source of evi- version of the ICF, EBP as a term and construct became
dence). A patient-reported experience measure assesses the relatively standardized and mainstream. Topics related to
patient’s satisfaction with and “experience with receiving EBP appeared in 1999 at the Annual Convention of the
care, including, for example, accessibility of services, quality American Speech-Language-Hearing Association, and
of clinician communication, cleanliness of care setting, ASHA’s first position statement regarding EBP was pub-
timeliness, and coordination of care” (Noonan et al., 2017). lished in 2005 (Dollaghan, 2007). Although PRO assess-
Clinically reported outcomes (ClinROs) are “evalua- ment is not an explicit component of EBP, it is a rigorous
tions from a trained professional after observation of a way of capturing core evidence that is hard to formally
patient’s health condition and involve clinical judgement or capture otherwise. “The patient is the only source of data
interpretation of the observable signs, behaviors, or other for certain therapeutic effects” (de Riesthal & Ross, 2015,
physical manifestations” (Mayo et al., 2017; U.S. Food p. 116), and meaningfully integrating the client’s subjective
and Drug Administration, 2015). The FIM and the FCMs experience and preferences (e.g., their experience with health
are examples of ClinROs. When the patient is rated by care) requires solicitation of this information. PROs are
someone who is not expertly trained, their report is termed one way of quantifying key components of EBP in a stan-
an “observer-reported outcome.” For example, a wife dardized way. The importance of PROs in other fields has
reporting her perspective of her husband’s anomia severity become unmistakable. For example, the U.S. Food and
would be considered an observer-reported outcome. Drug Administration and the European Medicines Agency
However, if an observer reports from the perspective of the require the inclusion of PROs for the evaluation of certain
patient (e.g., “How would your husband rate his anomia labeling claims (Bartlett & Ahmed, 2017; Patrick et al.,
severity?”), that person may be called a “proxy rater,” 2007).
resulting in a proxy-reported outcome. Observers and In 2002, the Director of the National Institutes of
proxies are most qualified to report health concepts related Health (NIH) moved to identify areas of significant oppor-
to limitations, restrictions, physical appearance, frequency, tunity related to biomedical research that would benefit
duration, and other behaviors (Mayo et al., 2017). It is also from broad (rather than institute-specific) support (NIH,
the case that agreement between observer or proxy reports 2014). In 2004, the NIH Roadmap for Medical Research
and patient reports is greater for domains defined by more was initiated and achieved additional funding from Con-
directly observable behaviors, for example, writing or talk- gress in 2006 through the NIH Common Fund. One of the
ing versus comprehension or confidence (Doyle et al., first Roadmap projects was the Patient-Reported Outcomes
2013; Duncan et al., 2002; Williams et al., 2006). Measurement Information System (PROMIS): “To see
Performance outcomes or performance-rated outcomes and understand the full picture of how a disease, condition,
are direct observations of behaviors (e.g., task or activity), or treatment affects patients, researchers needed an accurate,

Cohen & Hula: Patient-Reported Outcomes 359

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions
evidence-based tool for measuring symptoms…from the particular scale values based on how many respondents
patient’s perspective” (NIH, 2014, p. 36). One of the main reported higher versus lower levels of interference with
laments at that time was that questionnaires were too long communicative participation for each item.
and were not standardized or comparable across studies or After completing PROM items, an individual’s esti-
fields. There were too many different questionnaires about mated level of the latent trait is then transformed to a metric
symptoms of depression, for example, and this disunity that is more interpretable based on a reference or centering
complicated the aggregation and comparison of findings. sample. The calibration and centering samples are often,
The objective of the PROMIS project was to develop sets but not always, the same. Most PROMIS scores are centered
of psychometrically rigorous PROMs that were founded in on a very large sample of persons that match the demo-
item-response theory (IRT) and applicable across conditions graphics of the 2000 U.S. Census. Therefore, a PROMIS
and populations. Although PROMIS measures were ini- T score (M = 50, SD = 10) of 50 can be interpreted as the
tially developed for use in research settings (NIH, 2014), mean score of the general population of the United States.
they and other modern PROMs have been increasingly This is in contrast to PROMs that reference a clinical sam-
used in clinical practice. Several other measurement initia- ple (as described later in greater detail).
tives, such as the PROMIS or initiatives that were directly Items within an IRT-based item bank perform differ-
derived from the PROMIS, have been launched for specific ently. They confer different amounts of information about
conditions, such as Traumatic Brain Injury–Quality of Life different levels of the trait, and items can be administered
(TBI-QOL) for traumatic brain injury (Tulsky et al., 2016), in different combinations. One of the most sophisticated
Spinal Cord Injury–Quality of Life for spinal cord injury features of IRT-based PROMs is their ability to be adminis-
(Tulsky et al., 2015), Quality of Life in Neurological Dis- tered via computerized adaptive testing (CAT). CAT algo-
orders (Neuro-QOL) for neurological disorders broadly rithms strategically select an item to administer based on
(Cella et al., 2012; Gershon et al., 2012), and Adult Sickle the participant’s previous responses (Fries et al., 2005).
Cell Quality of Life Measurement Information System CATs can be customized to keep administering items until
(Keller et al., 2014). The reader is referred to www.health- a predetermined precision is achieved, or until a set number
measures.net for more information. At the 2017 Annual of items have been administered (or whichever comes first).
Conference of the PROMIS Health Organization (October In this way, CATs can be modified for different kinds of
2017), it was reported that 201 projects related to PROMIS assessment needs and contexts (Thomas, 2011). For settings
measure development and application had received and situations when a CAT administration is not possible
$111 million in research funding and resulted in 981 manu- (e.g., technology is not available), IRT-based PROMs can
scripts in 148 different journals (Cella & Gershon, 2017). At also be administered by “short form,” a predetermined set
that time, there were 97 PROMIS item banks for adults of items that can be administered by paper and pencil.
and 27 for children. For more information on development of an IRT-based
One of the distinguishing features of many new PROMs PROM, the reader is referred to Yorkston and Baylor (2019).
(including PROMIS measures) is that they are psycho-
metrically founded in IRT rather than in classical test
theory. The reader is referred to Baylor et al. (2011) for an Questions Related to PROs and EBP
excellent overview of IRT, particularly as it relates to the As the field of speech-language pathology moves to
field of speech-language pathology. Instruments based in incorporate PROMs into EBP, a number of questions de-
classical test theory typically require the computation of a serve consideration and future research. Below, we discuss
total score, which is then compared against a normative eight of these questions and preliminary thoughts and,
sample (Baylor et al., 2011). This requires every PROM when available, observations on PROMs from other fields.
item to be administered to every patient. In contrast, IRT-
based assessment is founded on the ability to use the same
Which Constructs Are Most Important to Assess
metric to describe a person’s level of the construct and items
that assess the trait. Thus, a particular person’s level of the by PRO?
construct can be estimated by their responses to many ASHA has communicated that the WHO ICF is the
different combinations of items from the item bank. In fact, preferred conceptual model to be used by clinicians and
the full item bank is almost never administered because researchers (ASHA, 2016), and Threats (2012) has provided
very good precision can be achieved with relatively few a compelling case for using the ICF as a framework and
items. During development of the PROM, items are “cali- classification system for PROMs. However, within the ICF
brated” on a large sample of participants in order to esti- framework, there are dozens, if not hundreds, of constructs
mate the “parameters” of the item—how responses to the related to communication and health conditions that affect
item relate to the latent trait scale. For example, in devel- communication that could be measured from the patient’s
oping the Communicative Participation Item Bank, Baylor perspective. Not all of these are equally important or rele-
et al. (2013) collected data from 701 individuals with a vant to speech-language therapy goals. One way to discern
range of conditions affecting speech. In their analysis, they which constructs are most important and relevant is to sim-
estimated the relative locations of the items on the latent ply ask the main stakeholders (e.g., clients, their families,
trait scale. Conceptually, this process assigned items to and clinicians). Indeed, the National Quality Forum has

360 American Journal of Speech-Language Pathology • Vol. 29 • 357–370 • February 2020

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

asserted that “outcomes should reflect the reason that an assess them more efficiently (e.g., de la Torre & Patz, 2005;
individual seeks healthcare services” (National Quality Haley et al., 2009).
Forum, 2009). As an illustrative example of how the field
might clinically ground assessment practices, Wallace et al.
Can Existing PROMs Serve Current and Future
conducted a qualitative consensus-building process with
people with aphasia, their families, and experienced clini- Measurement Needs?
cians (Wallace et al., 2016, 2017). This process revealed, A lack of PROMs, and a lack of consensus about
among other things, that people with aphasia most often which PROMs to use, has troubled the field of speech-
reported treatment outcomes that mapped to the activity/ language pathology for some time. A 2005 survey of aphasia
participation domains of the ICF. specialists revealed “a large and bewildering variety of tools
Because not every PRO construct can feasibly be and methods being used to measure outcomes, as well
measured in clinical practice, it may be wise to prioritize as a gap related to measures [of] life participation...[and]
the measurement of PRO constructs that are most directly quality of life” (Simmons-Mackie et al., 2005, p. 18).
assessed from the client’s perspective. For example, whereas Indeed, only 3% of respondents reported that they assess
clinicians and family members have a relatively direct per- quality of life. Recently, Neumann et al. (2019) conducted
spective on communication impairment (i.e., behavior), they a systematic review of PROMs used to assess quality of life
have a less direct perspective on constructs related to con- among individuals with neurogenic communication disor-
fidence, difficulty, burden, and social–emotional impact. ders. This review yielded 13 condition-generic measures and
In order to be maximally efficient in clinical settings, we 25 condition-specific measures, and a wide range of defini-
propose that clinicians continue to rely on ClinROs for con- tions (or a lack of any definition) of “quality of life” or
structs for which the clinician has a direct perspective “communication-related quality of life.” Measures
(e.g., not duplicate constructs that are already assessed by reviewed by these authors tended to be so condition-generic
ASHA’s FCMs) and augment these with PRO assessments that they failed to assess communication constructs at all
for constructs that the clinician is less able to observe (see or were so condition-specific that they were not applicable
Table 1). across neurogenic communication disorders. These authors
Finally, it would be worth developing a theoretical highlighted the value of developing PROMs that are ap-
and empirical model for how communication-relevant PRO plicable across communication disorders.
constructs are interrelated. It is likely that many PRO con- In addition to having too broad or narrow a scope,
structs are highly correlated, for example, confidence and many existing measures were not developed using the rigor-
participation. For these constructs, it may be productive to ous procedures that are now considered the industry stan-
consider the extent to which they are distinct and require dard (PROMIS Health Organization, 2013). For example,
separate measurement instruments. For correlated constructs a recent publication revealed concerns in the methodologi-
that are deemed sufficiently distinct on either rational or cal rigor underlying the development of PROMs related
empirical grounds to require separate score estimates, it to voice (Francis et al., 2017). This review found that most
may be possible to use multidimensional IRT models to voice-related PROMs did not include people with voice
conditions in the item development process, did not include
Table 1. Examples of construct by raters.
an adequate validation sample, and/or did not thoroughly
describe the validation sample; had underwhelming evidence
Patient, observer, for construct validity (especially responsivity to change);
or clinician report Patient report and no instrument reviewed had reported a literacy level.
A recent analysis of PROMs related to audiologic rehabili-
Impairment Burden tation found that every PROM designed for use in adult
Limitation Bother
audiologic rehabilitation requires a reading level above
Restriction Intensity
Duration Impact sixth grade (Douglas & Kelly-Campbell, 2018), which is
Physical appearance Difficulty the level recommended by health literacy experts (Wang
Ability Satisfaction et al., 2013).
Affect (display of emotion) Well-being This is certainly not to say that all existing PROMs
Frequency Social connectedness
Severity Stigma are inadequate. For example, rigorous development proce-
Effectiveness Confidence dures were used to develop the Aphasia Communication
Independence Comparison with internal standard Outcome Measure (ACOM; Hula et al., 2015), the Com-
Priorities munication Participation Item Bank (Baylor et al., 2013),
Self-efficacy
Emotion and the TBI-QOL Communication Item Bank (M. L.
Fulfillment Cohen, Kisala, et al., 2019). Other PROMs have under-
gone revisions to improve their rigor.
Note. Constructs in the right column are those that are most Because of the many measurement gaps that still exist
directly assessed from the patient’s perspective. Constructs in the
left column are more readily assessed from multiple perspectives in clinical speech-language pathology, it seems worth ex-
(patient, observer, or clinician). ploring whether any item banks from the PROMIS, Neuro-
QOL, or TBI-QOL would be valid and useful for this field.

Cohen & Hula: Patient-Reported Outcomes 361

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions
For example, the PROMIS has adult item banks that as- Score interpretation can be informed by a stakeholder
sess global health, cognitive function, self-efficacy, anxiety, consensus-building process called standard setting or
depression, life satisfaction, psychosocial impact of illness, bookmarking (Cella et al., 2014; Cook et al., 2015). This
ability to participate in social roles and activities, satisfac- can be especially helpful for PROMs with clinical reference
tion with social roles and activities, satisfaction with partic- samples such as the ACOM. Bookmarking involves creat-
ipation in discretionary social activities, and social isolation. ing vignettes about hypothetical people based on re-
The PROMIS also contains pediatric self-report item sponses to PROM items. For example, PROM responses
banks for constructs similar to those listed above, as well that would produce a score of T = 50 on the Neuro-QOL
as for family relationships and peer relationships. These Cognitive Function item bank v2.0 would lead to the fol-
constructs are likely of interest to SLPs although it re- lowing vignette:
mains to be studied whether the content of each item bank
Currently, Ms. Torres reports that she has no
is sufficiently specific to this patient population and whether difficulty planning an activity several days in advance,
the item bank is responsive to change in response to speech- like a meal, trip, or visit to friends. In the past seven
language therapy interventions. For measures such as the days, her thinking was never slow, and she never had
PROMIS item banks that were not developed to be used trouble planning out the steps of a task. Rarely in
with communication-impaired populations, additional is- the past seven days did she have to read something
sues, discussed below, must be addressed in order to ensure several times to understand it. She rarely had trouble
valid administration and interpretation of scores. keeping track of what she was doing if interrupted.
This vignette contains responses to five PRO items
How Should PROM Scores related to cognitive function. Similar vignettes are written
Be Interpreted Clinically? about people who would produce other PRO scores (e.g.,
A PRO may be measured for different reasons, and T = 55, T = 60). For more information about how most
PROMs are interpreted depending on these reasons. One likely responses are computed for PROMs, the reader is
reason might be to detect whether an individual has a nota- referred elsewhere (Cook et al., 2015). Stakeholder groups
bly elevated or reduced level of the trait. To answer this (e.g., clients, family members, experienced clinicians) then
question, the clinician could compare a particular score review sets of vignettes and achieve consensus on where
against a normative sample, such as with a z score (M = 0, the boundaries are between (i.e., set bookmarks between)
SD = 1) or T score (M = 50, SD = 10). For a PROM that PROM scores that represent “mild,” “moderate,” or “severe”
references a general population sample (such as most challenges.
PROMIS measures), a particular score’s distance from the A fourth reason for PRO assessment is to determine
mean is a relatively intuitive indicator of how normal or change over time—how much better or worse a particular
abnormal the score is. The reader is referred to M. L. PROM score is compared to a previous score. There are
Cohen, Weatherford, et al. (2019) for an illustration using two main ways to answer this question. The first is to ask
the PROMIS Cognitive Functioning item bank. whether the score change exceeds the error of the instru-
A second reason that a PRO might be measured is to ment; in other words, is the score change “statistically sig-
predict an external criterion, for example, a clinical diagno- nificant” or likely due to error? This is done by computing
sis or something that a person actually does or will do. For the “minimal detectable change” value, or the “reliable
example, Clover et al. (2018) evaluated the accuracy of the change” score (Revicki et al., 2006). A second way to answer
PROMIS Depression item bank in predicting clinical diag- this question is to ask whether the score change is “clinically
noses of major depressive disorder. significant;” that is, does the score change exceed a thresh-
A third reason that a PRO might be measured is to old at which the change is noticeable to most clients, signif-
determine an individual’s level of some trait compared to icant others, or clinicians? This threshold may be called
other people with a similar condition. For example, the the minimally important change, clinically important change,
ACOM is not intended to detect “whether” an individual responder definition, minimal clinically important difference
has aphasia, but rather to be sensitive to relative severity of (MCID), or something similar (Cook et al., 2015). There
aphasia. Because each individual in the normative sample are different ways to estimate this value, and a full de-
had aphasia, a score of T = 50 does not indicate normal scription of each can be found elsewhere (Coon & Cook,
language functioning, but rather the mean functioning of a 2018). However, it is important to note that this value
group of people with aphasia. The composition of the should be determined for each measure and population in
reference sample is very important for clinical interpretation; which the scores are to be interpreted (Revicki et al.,
misunderstanding the reference sample could easily lead to 2006; Wyrwich et al., 2013). For example, an MCID value
dramatic misinterpretation. Even within a measurement for individuals with cognitive symptoms secondary to
system, reference samples can vary. For example, some chemotherapy who complete the PROMIS Cognition item
Neuro-QOL item banks reference a general population bank does not necessarily mean the same as an MCID
sample whereas others reference a sample of adults with value for individuals with progressive cognitive symptoms
neurological conditions, so careful attention to reference from Alzheimer’s disease receiving speech-language therapy
samples is required for clinical interpretation. services. In order to be an effective outcome measure (i.e.,

362 American Journal of Speech-Language Pathology • Vol. 29 • 357–370 • February 2020

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

a measure that detects change), it is important for PROMs cognitive or language impairment, cannot complete any
to have strong test–retest reliability and, ideally, evidence PROM, regardless of modification or communication support;
for responsiveness to change. Otherwise, clinical change (c) those who can complete “some” PROMs, especially
might not be captured because of measurement error. those measures that are cognitively and linguistically simple,
One process that may be developed is the prospective or with some level of modification or communication support.
setting of quantitative goals for an individual client. For A fair, valid, and standardized approach is needed to
example, an individual could be asked, “What is the mini- determine which individuals cannot complete “any” PROM
mum improvement in a particular PROM item response because of profound cognitive or language impairment.
(and associated T score) that you would consider clinically Until a more precise operational definition exists, a helpful
meaningful?” This prospective setting of quantitative goals rule of thumb may be to exclude (or to at least interpret
would help mitigate the known problems associated with with extreme caution) the PROs of people who are cogni-
retrospective appraisal of satisfaction or improvement tively unable to provide informed consent for research or
(Rapkin & Schwartz, 2004; Schwartz et al., 2017; Schwartz clinical services (i.e., without a legally authorized represen-
& Rapkin, 2004). tative). In the longer term, it is incumbent upon developers
As with all clinical assessment scores, PROM scores and users of PROMs in the field of communication disorders
need to be interpreted by the clinician in context alongside to establish operationalized criteria for determining which
other information and data. PROM data are most inter- individuals can or cannot respond to them and to develop
pretable as part of a comprehensive assessment that may in- fair and valid tests of whether individuals meet these criteria.
clude clinician-reported and performance-based data. One path forward in this area might rely up on IRT-
based person-fit statistics (Meijer, 1996). These statistics
How Should PROMs Be Applied in Individuals quantify the degree to which the responses produced by a
given person accord with the measurement model’s expec-
With Varying Degrees of Cognitive
tations. These statistics take into account how the person
or Language Impairment? has responded to all of the items given and the relative loca-
Completing a PROM involves cognitive and language tion or trait level represented by those items, based on data
processes, so administering and interpreting PROMs requires collected from samples of the relevant population. They can
additional considerations when applied in individuals with alert a clinician or other test user to unexpected responses
cognitive or language impairment. Most of the cognitive and thus to instances when something other than the trait
models about the self-report process include four main pro- of interest is being measured. Person-fit statistics have
cessing stages: (a) comprehension of the instructions and traditionally been used in educational settings to identify
question, (b) retrieval of information from memory, (c) use patterns of student test-taking behavior that might decrease
of heuristic and decision processes to estimate an answer, the validity of test scores. For example, an overly careful
and (d) formulation and selection of a response (Jobe, 2003). (“plodding”) math student may correctly answer more easy
Rapkin and Schwartz (2004) have specifically postulated items and fewer difficult ones than would be expected
that individuals’ recall and appraisal of information is con- based on their total score because they spend maximal time
tingent on their frame of reference, strategies for recalling and effort on each item, regardless of its difficulty. Another
and sampling specific experiences, reference groups and student with a similar overall score (and thus similar math
standards of comparison, and the salience weights they as- ability) might show a careless (“sleeping”) response pattern
sociate with each recalled experience. Importantly, these in which they correctly answer fewer easy items and more
components of recall and appraisal may change over time, difficult items than expected based on their total score (Meijer,
leading to a “response shift,” defined as a change in PROM 1996). Other examples of behavioral patterns that might
score resulting from a change in the appraisal process, and be identified in the educational context include guessing,
not necessarily from a change in health status. For example, cheating, and deficiencies in specific ability subdomains.
an individual might change their perception of what it In the context of PROs for individuals with commu-
means to experience “difficulty,” or to whom or what they nication disorders, person-fit statistics might be used to
are comparing their health status. The field of self-report identify cases where comprehension or other cognitive im-
science has not come to a strong consensus on how to con- pairment compromises the validity of a score estimate. For
sider and address these appraisal processes in people with example, a person with aphasia who reports that they have
typical cognition and language, much less those with cogni- great difficulty saying their own name or understanding
tive and language disorders. However, this is a very worthy single words but no difficulty carrying on a group conver-
pursuit because it underlies the very nature of perceived sation or leaving a voicemail message might be identified
health and disability. This issue is particularly critical if clin- by a person-fit statistic as having produced an unexpected,
ical reimbursement algorithms are going to consider PROM and thus potentially invalid response string. To the extent
score changes as evidence of treatment benefit. that such statistics could be validated by, for example,
Thinking in “broad strokes,” would-be respondents showing that they are predicted by relevant abilities such
may fall into one of three categories: (a) those who can com- as comprehension or memory, they might be useful in estab-
plete any PROM as is, without any modification or com- lishing criteria for PRO administration in populations with
munication support; (b) those who, because of profound cognitive or language impairment.

Cohen & Hula: Patient-Reported Outcomes 363

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions
 A more complex problem involves determining The interpretation of PROs from individuals with
whether an individual can complete a “specific” PROM, cognitive and language disorders requires significantly more
particularly if that PROM was not developed for individuals research and thoughtful application. There are a number of
with communication disorders. PROMs are heterogeneous; neuropsychological deficits that may preclude or complicate
they ask about both abstract and concrete health constructs, the administration or interpretation of PROMs, and PROMs
are of varying lengths, are composed at different reading are probably not clinically useful for every client and con-
levels, and require different amounts of cognitive effort text (Barrett, 2010). However, it is worth remembering that,
(for example, selecting yes/no vs. using a 7-point scale). for most health constructs, no single source of evidence is
Having an administrator present for “some” communica- the “truth.” Health is inherently subjective, and a core value
tion support is common and acceptable, but the more sup- of EBP is solicitation and consideration of the patient’s
port the administrator offers, the more likely it is that the perspective, regardless of how well it coheres with other
administrator corrupts the measure, which can complicate sources of information.
the interpretation of the score against a normative sample
and across time. Tucker et al. (2012) administered a battery
of PROMs to people with aphasia, using a hierarchy of How Exactly Could PROs Inform EBP?
communication support. They reported adequate internal Ultimately, EBP involves the clinician deciding what
consistency of items administered with communication evidence to gather, how to combine and interpret multiple
support; however, the influence of strong support (e.g., sources of evidence, and how to act on it. Indeed, although
rephrasing PROM items) on the other psychometric prop- clinical experience is not conceptualized as a stand-alone
erties of the measures was not fully evaluated (Tucker et al., component of EBP, it is the “glue” that binds together the
2012). other elements (Dollaghan, 2007). Whether and how the
More superficial communication support includes EBP process interfaces with external entities and pressures
presenting PROM items in multiple modalities (visually and (e.g., third-party payers and reimbursement algorithms) is
aurally), changing text size as needed, repeating the item a separate issue that deserves discussion, but is beyond the
and response options, providing some word definitions, and scope of this article.
facilitating responses by verbal or manual selection of a Across clinical fields, PROMs are being used to screen
response option. There is mixed evidence regarding whether for health disorders and trigger referrals, to diagnose and
utilizing a live administrator biases PROM scores; however, make prognoses, to assess client needs, to help set goals and
when bias has been detected, it seems to be relatively monitor treatment progress, to match clients to treatments,
small for relatively nonsensitive topics (Kisala et al., 2019; to inform and facilitate clinician–client communication, to
Rutherford et al., 2016). In clinical settings, administrator monitor symptom change, to guide shared decision making
bias likely depends in part on who the administrator is, and self-management, to assess the effectiveness of inter-
although this remains to be studied. We predict that it would ventions, and to determine satisfaction with services (Ayers
be most in the spirit of PROs for the administrator to be et al., 2013; Bingham et al., 2016; Fayers, 2008; Jensen &
an individual who is relatively disinterested, that is, someone Snyder, 2016; Noonan et al., 2017; Snyder et al., 2012;
other than the treating clinician like a clinic staff member Velikova et al., 2004; Wagner et al., 2015). In speech-language
who is trained in communication support or an adequately pathology, PROMs could serve as a very useful comple-
trained and supervised graduate student clinician who is not ment to widely used clinician-reported and performance-
directly involved in treatment of that particular client. based measures. For example, a client with aphasia whose
PROMs that have not been calibrated in individuals treatment goals are highly functional may not show improve-
with communication disorders need to be tested for the ment on clinician-reported observations of behavior (e.g.,
presence of item bias (differential item functioning; Fieo the FCMs) or performance-based tests of language (e.g., the
et al., 2015) and perhaps recalibrated and/or formally modi- Western Aphasia Battery; Risser & Spreen, 1985). However,
fied if necessary. For example, Hunting Pompon et al. the client may show improvement on the Communication
(2018) recently reported an interesting modification of the Participation Item Bank, which assesses functional tasks
Perceived Stress Scale (S. Cohen et al., 1983) that makes it such as talking on the phone or ordering in a restaurant.
more accessible by adults with aphasia by including graphic A few early adopters of modern PROMs in clinical
supports, simplified wording, and a noncluttered appear- settings have concretely demonstrated their clinical po-
ance on the page. In general, PROMs that are modified tential. For example, Basch et al. (2016) randomly assigned
online for a particular client, for example, by using the hier- 766 patients receiving outpatient chemotherapy to one of
archy developed by Tucker et al. (2012), should be interpreted two conditions: (a) standardized, routine PROM-based
with significant caution because they are being adminis- monitoring of symptoms or (b) usual care, that is, symp-
tered and interpreted “off label,” in a manner that may pre- tom monitoring at the clinician’s discretion. The group
clude comparison with the normative sample. However, of patients systematically assessed by PROMs reported
one recent study of the Communicative Participation Item better health-related quality of life, were less likely to need
Bank found minimal differential item functioning when the emergency care, adhered to treatment better, and were
Tucker et al. hierarchy was used in administering it to more likely to survive 1 year. However, not all clinical inte-
persons with aphasia (Baylor et al., 2016). grations of PROMs have found them to be useful (Kean

364 American Journal of Speech-Language Pathology • Vol. 29 • 357–370 • February 2020

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

et al., 2016), and it would be beneficial for the field to em- outcome plan, the reader is referred to Higginbotham and
pirically study whether use of PROMs actually leads to Moulton (2017). PROMs may be useful for these assess-
change in clinical practice or outcomes (Greenhalgh et al., ment needs, but not necessarily. PROMs are an additional
2005). tool in the clinician’s toolbox, and relevant to many but
Dollaghan (2007) wrote that “the applicability of… not all jobs.
evidence from even a very strong study to an individual For relevant clients and goals, it is difficult to pre-
patient will need to be tested rather than assumed. Clini- scribe how PROMs should be selected and applied because
cians must adapt the features and processes associated with a number of variables may be relevant in the matching of
high-quality…evidence to questions about treatment with tools to clinical needs. However, a general recommendation
individual patients” (p. 115). She gives the examples of is to apply the checklist developed by Francis et al. (2016).
routine progress monitoring via treatment probes as well Based on the recommendations of American Psychological
as “measures to determine whether skills are generalizing Association, the U.S. Food and Drug Administration, and
to other domains or contexts.” PROMs could serve this other authoritative bodies, these authors developed a
function well. Quoting Janosky (2005, p. 549), Dollaghan checklist to help evaluate the strengths and weaknesses
(2007) goes on to write, “when sound research methodol- of a PROM’s scientific development and appropriateness
ogy is implemented, patient care is a special case of a single for a particular application based on factors like the PROMs
subject research design.” Applying this level of methodo- conceptual model, validity, reliability, interpretability, and
logical rigor to patient care requires precise measurement respondent burden. In addition to what is contained in this
tools that have been psychometrically vetted for that purpose. checklist, for reasons illustrated above, we would advise
It also requires psychometrically savvy clinicians to think SLPs to pay particular attention to the details that surround
critically about issues like reliable change. A very powerful scoring and interpretation, for example, the composition of
way of connecting “external” scientific evidence with “in- the normative sample and whether there is evidence that
ternal” practice evidence is to have researchers and clinicians the measure is reliable and sensitive to change with the tar-
use the same assessment tools, in line with the intention get population. We would also advise consideration of the
of modern PRO measurement initiatives (Iturralde et al., type and level of communication support that a client may
2017; Jones & Stukenborg, 2017; Rodriguez-Correa et al., require for a particular PROM.
2017; Wagner et al., 2015).
In laboratory settings, participants are often carefully How Should PROMs Be Used for EBP
vetted and relatively homogenous, a protocol is rigorously
With Child Clients?
adhered to, and extraneous variables are suppressed or
their effects are statistically removed. In real-world settings, Pediatric PROMs are typically developed for very
however, interventions often need to be flexible and involve specific age ranges because of children’s developing ca-
a sequence of decisions, and PROMs could help inform pacity for self-reflection, abstract thinking, and other skills
how an intervention should be altered or adapted. For ex- (Arbuckle & Abetz-Webb, 2013). However, it is worth
ample, Lei et al. (2012) published a model for delivering an noting that most age-specific measures are developed for
adaptive intervention that is responsive to client variables typically developing children and may not be appropriate
(e.g., severity, preferences) and intermediate outcomes (e.g., for children with atypical cognitive or language develop-
early response to treatment, adherence). ment. Whether, when, and how to rely on child self-report
As discussed by Daub et al. (2019), the future of versus parent or teacher report also deserves careful
measurement-informed EBP relies on better communication consideration.
between PROM developers and the clinicians who are end
users of those measures. To meet in the middle of the his- What Are the Barriers to Dissemination
torical gap between these groups, test developers may and Implementation of PRO-Informed Practice?
benefit from intentionally soliciting and valuing the perspec-
tives of clinicians, and clinicians may benefit from becom- Making PRO measurement a routine aspect of clinical
ing psychometrically savvy and advocating for their needs care requires significant and ongoing buy-in from multiple
and the needs of their clients. stakeholders, including patients, clinicians, administrators,
ancillary staff, information technology personnel, and pa-
tient advocates (Dobrozsi & Panepinto, 2015). Mandates
How Should a Clinician Select a PRO Measure? for PRO data from Centers for Medicare & Medicaid
First, a clinician should determine what assessments Services would motivate at least perfunctory adherence in
are needed at the beginning of treatment to diagnose or some areas of practice, but unless these stakeholders per-
describe a client and develop an outcome plan based on treat- ceive a tangible benefit, PRO-informed clinical practice is
ment goals. As described elsewhere, an outcome plan in- likely to languish (Bingham et al., 2016; Rolstad et al., 2011).
volves three levels of outcomes including functional outcomes, Indeed, it was recently said that “the most important barrier
treatment outcomes, and ultimate outcomes (Campbell & in introducing PRO assessments into…[clinical] care today
Bain, 1991; Golper & Frattali, 2013; Higginbotham & seem to be attitudinal barriers.… Health care providers…
Moulton, 2017). For more information on constructing an are not familiar with the questionnaires, and question their

Cohen & Hula: Patient-Reported Outcomes 365

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions
applicability and clinical usefulness” (Rose & Bezjak, 2009, whereas the response rate for paper-and-pencil PROMs
p. 128). For reasons outlined in this article and elsewhere, was 14% in their outpatient rheumatology clinic, electronic
there are many good reasons to think that the use of PROMs data capture raised the rate to 98%.
could benefit clinical practice. Even if PROs do not “change”
practice, they may provide more, and different evidence
that what is already being done has a measurable benefit Conclusion
from the patient’s perspective, adding to evidence for the PROs are becoming an increasingly central compo-
profession. Without the needed “buy-in” from stakeholders, nent of clinical research and practice in many health care
other potential barriers seem moot. It would be valuable to fields and are becoming increasingly considered in reimburse-
conduct iterative dissemination and implementation research ment algorithms. Despite significant effort associated with
to ensure that PROMs are maximally useful to these stake- changing clinical habits and workflows, the field of speech-
holders (Noonan et al., 2017). language pathology has much to gain from inclusion of
Historically, one of the main reasons cited by SLPs PROs. Speech-language pathology treatment may benefit
for not assessing PROs is the time and logistical constraints from more routinely measuring subjective and unobserv-
involved in administering and scoring PROMs (ASHA, able aspects of health that are not currently captured.
2013; Simmons-Mackie et al., 2005; Torrence et al., 2016). PROMs may help clinicians systematically and rigorously
This may be less of a problem than in the past, as more assess those internal health constructs, both for improving
PROMs are being psychometrically developed with IRT, the delivery of EBP and to use as evidence in support of
making them generally more time efficient, especially if they the profession. The frontier of PROs in this field seems to
are administered by CAT. In fact, some CAT algorithms be (a) re-evaluating whether legacy PROMs are meeting
can produce reliable scores with as few as two items per current and future needs and meet modern psychometric
construct (Hays et al., 2017), although a minimum of four standards; (b) testing whether recently developed, cross-
items is more common. A 2017 study found that clients in discipline measures (e.g., PROMIS measures) are valid and
an outpatient oncology clinic completed five 4-item PRO- clinically useful for the field of speech-language pathology;
MIS CATs in about 5 min (i.e., 60 s per CAT; Mooney (c) developing new PROMs where needed; and (d) con-
et al., 2017). However, more research is needed on the time ducting dissemination and implementation research to better
required to administer PROMs to individuals with commu- include clinician needs and perspectives, identify and mini-
nication disorders, with and without communication sup- mize barriers to PROM use, and maximize the applica-
port. Ultimately, however, time is only a problem if it is tion of PROMs to clinical practice. For more information
uncompensated. There has been interest in developing a on IRT, PROM development, and PROMs recently devel-
dedicated procedure code for PRO assessment, especially if oped for individuals with communication disorders, the
that assessment is mandated by third-party payers. interested reader is referred to Baylor et al. (2011); Yorkston
An important variable to consider is “how” and and Baylor (2019); M. L. Cohen, Kisala, et al. (2019), M. L.
“when” to capture PROs. In order to assimilate PRO-informed Cohen, Tulsky et al. (2019); and Hula et al. (2015). For
practice into existing workflows, there has been growing more information and to obtain PROMIS and Neuro-QoL
interest in integrating PROMs into electronic health record measures, the reader is referred to http://healthmeasures.net.
(EHR) systems. Indeed, the PROMIS and related measure-
ment systems have application program interface capabili-
ties with several major EHR vendors. Wagner et al. (2015),
Acknowledgments
for example, reported how PROMs could be adminis- This work was supported by the Gordon and Betty Moore
tered through the EHR patient portal and completed by Foundation Grant GBMF5299, awarded to Cohen. The authors
would like to thank Alyssa Lanzi and Aquiles Iglesias for their
the patient at home, with scores displayed for clinicians
helpful feedback on earlier versions of this article.
prior to the next clinical encounter. The logistics of PRO-
informed practice have not yet been reported for individuals
with communication disorders. One possibility is the use of References
an initial triage system to identify whether or not the client American Speech-Language-Hearing Association. (n.d.). National
needs communication support to complete PROMs. If no Outcomes Measurement System. Retrieved January 23, 2020,
support is needed, clients could complete PROMs at home from https://www.asha.org/noms/national-outcomes-measurement-
before a clinical encounter. Clients who need communica- system/
tion support could arrive early or stay after their clinical American Speech-Language-Hearing Association. (2013). ASHA
encounter for supported administration of PROMs, perhaps SLP Health Care Survey 2013. http://www.asha.org
with a clinic staff member who is trained in communica- American Speech-Language-Hearing Association. (2015). ASHA
tion support. SLP Health Care Survey 2015. http://www.asha.org
American Speech-Language-Hearing Association. (2016). Scope of
Some clients may prefer to complete paper-and- practice in speech-language pathology. http://www.asha.org/
pencil PROMs, for example, if underlining and circling helps policy
support their comprehension. However, computerized ad- American Speech-Language-Hearing Association. (2018). ASHA Stra-
ministration has been shown to be very helpful for response tegic Pathway to Excellence. http://www.asha.org/uploadedFiles/
rates. For example, Williams et al. (2004) reported that, Strategic-Objective-Highlights.pdf

366 American Journal of Speech-Language Pathology • Vol. 29 • 357–370 • February 2020

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Arbuckle, R., & Abetz-Webb, L. (2013). “Not just little adults”: Cella, D., Lai, J.-S., Nowinski, C. J., Victorson, D., Peterman, A.,
Qualitative methods to support the development of pediatric Miller, D., Bethoux, F., Heinemann, A., Rubin, S., Cavazos, J. E.,
patient-reported outcomes. The Patient: Patient Centered Reder, A. T., Sufit, R., Simuni, T., Holmes, G. L., Siderowf, A.,
Outcomes Research, 6(3), 143–159. https://doi.org/10.1007/ Wojna, V., Bode, R., McKinney, N., Podrabsky, T., Wortman, K.,
s40271-013-0022-3 Choi, S., Gershon, R., Rothrock, N., & Moy, C. (2012). Neuro-
Ayers, D. C., Zheng, H., & Franklin, P. D. (2013). Integrating QOL: Brief measures of health-related quality of life for clinical
patient-reported outcomes into orthopaedic clinical practice: research in neurology. Neurology, 78(23), 1860–1867. https://
Proof of concept from FORCE-TJR. Clinical Orthopaedics doi.org/10.1212/WNL.0b013e318258f744
and Related Research, 471(11), 3419–3425. https://doi.org/ Cella, D., Hahn, E. A., Jensen, S. E., Butt, Z., Nowinski, C. J.,
10.1007/s11999-013-3143-z Rothrock, N., & Lohr, K. N. (2015). Patient-reported outcomes
Barrett, A. M. (2010). Rose-colored answers: Neuropsychological in performance measurement. RTI Press.
deficits and patient-reported outcomes after stroke. Behav- Clover, K., Lambert, S. D., Oldmeadow, C., Britton, B., King, M. T.,
ioural Neurology, 22(1–2), 17–23. https://doi.org/10.3233/BEN- Mitchell, A. J., & Carter, G. (2018). PROMIS depression mea-
2009-0250 sures perform similarly to legacy measures relative to a structured
Bartlett, S. J., & Ahmed, S. (2017). Montreal Accord on Patient- diagnostic interview for depression in cancer patients. Quality
Reported Outcomes (PROs) use series–Paper 1: Introduction. of Life Research, 27(5), 1357–1367. https://doi.org/10.1007/
Journal of Clinical Epidemiology, 89, 114–118. https://doi.org/ s11136-018-1803-x
10.1016/j.jclinepi.2017.04.012 Cohen, M. L., Kisala, P. A., Boulton, A. J., Carlozzi, N. E., Cook,
Basch, E., Deal, A. M., Kris, M. G., Scher, H. I., Hudis, C. A., C. V., & Tulsky, D. S. (2019). Development and psychometric
Sabbatini, P., Rogak, L., Bennett, A. V., Dueck, A. C., characteristics of the TBI-QOL communication item bank.
Atkinson, T. M., Chou, J. F., Dulko, D., Sit, L., Barz, A., Journal of Head Trauma Rehabilitation, 34(5), 326–339.
Novotny, P., Fruscione, M., Sloan, J. A., & Schrag, D. (2016). Cohen, M. L., Tulsky, D. S., Boulton, A. J., Kisala, P. A., Bertisch,
Symptom monitoring with patient-reported outcomes during H., Yeates, K. O., Zonfrillo, M. R., Durbin, D. R., Jaffe, K. M.,
routine cancer treatment: A randomized controlled trial. Journal Temkin, N., Wang, J., & Rivara, F. P. (2019). Reliability and
of Clinical Oncology: Official Journal of the American Society construct validity of the TBI-QOL communication short form
of Clinical Oncology, 34(6), 557–565. https://doi.org/10.1200/ as a parent-proxy report instrument for children with traumatic
JCO.2015.63.0830 brain injury. Journal of Speech, Language, and Hearing Research,
Baylor, C., Hula, W., Donovan, N. J., Doyle, P. J., Kendall, D., & 62(1), 84–92. https://doi.org/10.1044/2018_JSLHR-L-18-0074
Yorkston, K. (2011). An introduction to item response theory Cohen, M. L., Weatherford, S., & Nandakumar, R. (2019). How
and Rasch models for speech-language pathologists. American normal are “normal” errors of language and cognition? Journal
Journal of Speech-Language Pathology, 20(3), 243–259. https:// of Speech, Language, and Hearing Research, 62(5), 1468–1472.
doi.org/10.1044/1058-0360(2011/10-0079) https://doi.org/10.1044/2018_JSLHR-L-18-0215
Baylor, C., Oelke, M., Bamer, A., Hunsaker, E., Off, C., Wallace, Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global mea-
S. E., Pennington, S., Kendall, D., & Yorkston, K. (2016). Vali- sure of perceived stress. Journal of Health and Social Behavior,
dating the Communicative Participation Item Bank (CPIB) for 24(4), 385–396. https://doi.org/10.2307/2136404
use with people with aphasia: An analysis of differential item Cook, K. F., Victorson, D. E., Cella, D., Schalet, B. D., & Miller,
function (DIF). Aphasiology, 31(8), 861–878. https://doi.org/ D. (2015). Creating meaningful cut-scores for Neuro-QOL
10.1080/02687038.2016.1225274 measures of fatigue, physical functioning, and sleep disturbance
Baylor, C., Yorkston, K., Eadie, T., Kim, J., Chung, H., & using standard setting with patients and providers. Quality of
Amtmann, D. (2013). The Communicative Participation Item Life Research, 24(3), 575–589. https://doi.org/10.1007/
Bank (CPIB): Item bank calibration and development of a s11136-014-0790-9
disorder-generic short form. Journal of Speech, Language, and Coon, C. D., & Cook, K. F. (2018). Moving from significance to
Hearing Research, 56(4), 1190–1208. https://doi.org/10.1044/ real-world meaning: Methods for interpreting change in clinical
1092-4388(2012/12-0140) outcome assessment scores. Quality of Life Research, 27(1),
Bingham, C. O., III., Bartlett, S. J., Merkel, P. A., Mielenz, T. J., 33–40. https://doi.org/10.1007/s11136-017-1616-3
Pilkonis, P. A., Edmundson, L., Moore, E., & Sabharwal, R. K. Daub, O., Skarakis-Doyle, E., Bagatto, M. P., Johnson, A. M.,
(2016). Using patient-reported outcomes and PROMIS in re- & Cardy, J. O. (2019). A comment on test validation: The
search and clinical applications: Experiences from the PCORI importance of the clinical perspective. American Journal of
pilot projects. Quality of Life Research, 25(8), 2109–2116. Speech-Language Pathology, 28(1), 204–210. https://doi.org/
https://doi.org/10.1007/s11136-016-1246-1 10.1044/2018_AJSLP-18-0048
Campbell, T. F., & Bain, B. A. (1991). How long to treat: A multiple de la Torre, J., & Patz, R. J. (2005). Making the most of what we
outcome approach. Language, Speech, and Hearing Services in have: A practical application of multidimensional item re-
Schools, 22(4), 271–276. https://doi.org/10.1044/0161-1461. sponse theory in test scoring. Journal of Educational and Be-
2204.271 havioral Statistics, 30(3), 295–311. https://doi.org/10.3102/
Cella, D., Choi, S., Garcia, S., Cook, K. F., Rosenbloom, S., Lai, 10769986030003295
J.-S., Tatum, D. S., & Gershon, R. (2014). Setting standards de Riesthal, M., & Ross, K. B. (2015). Patient reported outcome
for severity of common symptoms in oncology using the measures in neurologic communication disorders: An update.
PROMIS item banks and expert judgment. Quality of Life SIG 2 Perspectives on Neurophysiology and Neurogenic Speech
Research, 23(10), 2651–2661. https://doi.org/10.1007/s11136- and Language Disorders, 25(3), 114–120.
014-0732-6 Dobrozsi, S., & Panepinto, J. (2015). Patient-reported outcomes
Cella, D., & Gershon, R. (2017). The future of PROMIS is now. in clinical practice. Hematology, 2015(1), 501–506. https://doi.
Presented at the PROMIS Health Organization Annual Confer- org/10.1182/asheducation-2015.1.501
ence,, San Diego, CA, United States. http://www.promishealth. Dollaghan, C. A. (2007). The handbook for evidence-based practice
com/2017-conference-promis-in-action in communication disorders. Brookes.

Cohen & Hula: Patient-Reported Outcomes 367

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions
Donabedian, A. (1966). Evaluating the quality of medical care. multidimensional bi-factor model and computerized adaptive
The Milbank Memorial Fund Quarterly, 44(3), 166–206. https:// testing. Quality of Life Research, 18(3), 359–370. https://doi.org/
doi.org/10.2307/3348969 10.1007/s11136-009-9447-5
Donabedian, A. (1988). The quality of care. How can it be assessed? Hays, R. D., Schalet, B. D., Spritzer, K. L., & Cella, D. (2017).
JAMA, 260(12), 1743–1748. https://doi.org/10.1001/jama.1988. Two-item PROMIS global physical and mental health scales.
03410120089033 Journal of Patient-Reported Outcomes, 1(1). https://doi.org/
Douglas, A., & Kelly-Campbell, R. J. (2018). Readability of 10.1186/s41687-017-0003-8
patient-reported outcome measures in adult audiologic rehabil- Higginbotham, J., & Moulton, B. (Eds.). (2017). Appraising
itation. American Journal of Audiology, 27(2), 208–218. https:// change: Making evidence-informed decisions in speech-language
doi.org/10.1044/2018_AJA-17-0095 pathology. Ultra-Blue Capital, LLC. https://www.therapy-sci-
Doyle, P. J., Hula, W. D., Austermann Hula, S. N., Stone, C. A., ence.com/
Wambaugh, J. L., Ross, K. B., & Schumacher, J. G. (2013). Hula, W. D., Doyle, P. J., Stone, C. A., Austermann Hula, S. N.,
Self- and surrogate-reported communication functioning in Kellough, S., Wambaugh, J. L., Ross, K. B., Schumacher, J. G.,
aphasia. Quality of Life Research, 22(5), 957–967. https://doi. & St Jacque, A. (2015). The Aphasia Communication Outcome
org/10.1007/s11136-012-0224-5 Measure (ACOM): Dimensionality, item bank calibration, and
Duncan, P. W., Lai, S. M., Tyler, D., Perera, S., Reker, D. M., & initial validation. Journal of Speech, Language, and Hearing Re-
Studenski, S. (2002). Evaluation of proxy responses to the search, 58(3), 906–919. https://doi.org/10.1044/2015_JSLHR-L-
stroke impact scale. Stroke, 33(11), 2593–2599. https://doi. 14-0235
org/10.1161/01.STR.0000034395.06874.3E Hunting Pompon, R., Amtmann, D., Bombardier, C., & Kendall, D.
Fayers, P. M. (2008). Evaluating the effectiveness of using PROs (2018). Modifying and validating a measure of chronic stress
in clinical practice: A role for cluster-randomised trials. Quality for people with aphasia. Journal of Speech, Language, and
of Life Research, 17(10), 1315–1321. https://doi.org/10.1007/ Hearing Research, 61(12), 2934–2949. https://doi.org/10.1044/
s11136-008-9391-9 2018_JSLHR-L-18-0173
Fieo, R., Mukherjee, S., Dmitrieva, N. O., Fyffe, D. C., Gross, A. L., Iturralde, E., Adams, R. N., Barley, R. C., Bensen, R., Christofferson,
Sanders, E. R., Romero, H. R., Potter, G. G., Manly, J. J., M., Hanes, S. J., Maahs, D. M., Milla, C., Naranjo, D., Shah,
Mungas, D. M., & Gibbons, L. E. (2015). Differential item A. C., Tanenbaum, M. L., Veeravalli, S., Park, K. T., & Hood,
functioning due to cognitive status does not impact depressive K. K. (2017). Implementation of depression screening and global
symptom measures in four heterogeneous samples of older health assessment in pediatric subspecialty clinics. Journal of
adults. International Journal of Geriatric Psychiatry, 30(9), Adolescent Health, 61(5), 591–598. https://doi.org/10.1016/j.
911–918. https://doi.org/10.1002/gps.4234 jadohealth.2017.05.030
Francis, D. O., Daniero, J. J., Hovis, K. L., Sathe, N., Jacobson, Janosky, J. E. (2005). Use of the single subject design for practice-
B., Penson, D. F., Feurer, I. D., & McPheeters, M. L. (2017). based primary care research. Postgraduate Medicine Journal,
Voice-related patient-reported outcome measures: A systematic 81(959), 549–551. https://doi.org/10.1136/pgmj.2004.031005
review of instrument development and validation. Journal of Jensen, R. E., & Snyder, C. F. (2016). PRO-cision medicine: Per-
Speech, Language, and Hearing Research, 60(1), 62–88. https:// sonalizing patient care using patient-reported outcomes. Journal
doi.org/10.1044/2016_JSLHR-S-16-0022 of Clinical Oncology, 34(6), 527–529.
Francis, D. O., McPheeters, M. L., Noud, M., Penson, D. F., & Jobe, J. B. (2003). Cognitive psychology and self-reports: Models
Feurer, I. D. (2016). Checklist to operationalize measurement and methods. Quality of Life Research, 12(3), 219–227.
characteristics of patient-reported outcome measures. Systematic Jones, R. S., & Stukenborg, G. J. (2017). Patient-Reported Out-
Reviews, 5(1), 129. https://doi.org/10.1186/s13643-016-0307-4 comes Measurement Information System (PROMIS) use in
Fries, J. F., Bruce, B., & Cella, D. (2005). The promise of PRO- surgical care: A scoping study. Journal of the American College
MIS: Using item response theory to improve assessment of of Surgeons, 224(3), 245–254.e1. https://doi.org/10.1016/
patient-reported outcomes. Clinical and Experimental Rheuma- j.jamcollsurg.2016.11.015
tology, 23(Suppl. 39),, S53–S57. Kean, J., Monahan, P. O., Kroenke, K., Wu, J., Yu, Z., Stump, T. E.,
Gershon, R. C., Lai, J. S., Bode, R., Choi, S., Moy, C., Bleck, T., & Krebs, E. E. (2016). Comparative responsiveness of the
Miller, D., Peterman, A., & Cella, D. (2012). Neuro-QOL: PROMIS Pain Interference short forms, Brief Pain Inventory,
Quality of life item banks for adults with neurological disorders: PEG, and SF-36 Bodily Pain subscale. Medical Care, 54(4),
Item development and calibrations based upon clinical and 414–421. https://doi.org/10.1097/MLR.0000000000000497
general population testing. Quality of Life Research, 21(3), Keller, S. D., Yang, M., Treadwell, M. J., Werner, E. M., &
475–486. https://doi.org/10.1007/s11136-011-9958-8 Hassell, K. L. (2014). Patient reports of health outcome for
Golper, L. A. C., & Frattali, C. (2013). Outcomes in speech-language adults living with sickle cell disease: Development and testing
pathology. Thieme. of the ASCQ-Me item banks. Health and Quality of Life Out-
Greenhalgh, J., Long, A. F., & Flynn, R. (2005). The use of patient comes, 12, 125. https://doi.org/10.1186/s12955-014-0125-0
reported outcome measures in routine clinical practice: Lack Kisala, P. A., Boulton, A. J., Cohen, M. L., Slavin, M. D., Jette,
of impact or lack of theory? Social Science & Medicine, 60(4), A. M., Charlifue, S., Hanks, R., Mulcahey, M. J., Cella, D., &
833–843. https://doi.org/10.1016/j.socscimed.2004.06.022 Tulsky, D. S. (2019). Interviewer- versus self-administration of
Granger, C. V., Hamilton, B. B., Keith, R., Zielezny, M., & PROMIS® measures for adults with traumatic injury. Health
Sherwin, F. S. (1986). Advances in functional assessment for Psychology, 38(5), 435–444. https://doi.org/10.1037/hea0000685
medical rehabilitation. Topics in Geriatric Rehabilitation, 1(3), Lei, H., Nahum-Shani, I., Lynch, K., Oslin, D., & Murphy, S. A.
59–74. https://doi.org/10.1097/00013614-198604000-00007 (2012). A “SMART” design for building individualized treatment
Haley, S. M., Ni, P., Dumas, H. M., Fragala-Pinkham, M. A., sequences. Annual Review of Clinical Psychology, 8, 21–48. https://
Hambleton, R. K., Montpetit, K., Bilodeau, N., Gorton, G. E., doi.org/10.1146/annurev-clinpsy-032511-143152
Watson, K., & Tucker, C. A. (2009). Measuring global physical Mayo, N. E., Figueiredo, S., Ahmed, S., & Bartlett, S. J. (2017).
health in children with cerebral palsy: Illustration of a Montréal Accord on Patient-Reported Outcomes (PROs) use

368 American Journal of Speech-Language Pathology • Vol. 29 • 357–370 • February 2020

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

 series–Paper 2: Terminology proposed to measure what matters Rolstad, S., Adler, J., & Rydén, A. (2011). Response burden and
in health. Journal of Clinical Epidemiology, 89, 119–124. https:// questionnaire length: Is shorter better? A review and meta-analysis.
doi.org/10.1016/j.jclinepi.2017.04.013 Value in Health, 14(8), 1101–1108. https://doi.org/10.1016/j.
Meijer, R. R. (1996). Person-fit research: An introduction. Applied jval.2011.06.003
Measurement in Education, 9(1), 3–8. https://doi.org/10.1207/ Rose, M., & Bezjak, A. (2009). Logistics of collecting patient-
s15324818ame0901_2 reported outcomes (PROs) in clinical practice: An overview and
Mooney, K., Biber, J., Hess, R., Weeks, H., & Sweetenham, J. W. practical examples. Quality of Life Research, 18(1), 125–136.
(2017). Implementing routine assessment of patient-reported out- https://doi.org/10.1007/s11136-008-9436-0
comes in cancer care. Journal of Clinical Oncology, 35(Suppl. 8),, Rutherford, C., Costa, D., Mercieca-Bebber, R., Rice, H., Gabb, L.,
229. https://doi.org/10.1200/JCO.2017.35.8_suppl.229 & King, M. (2016). Mode of administration does not cause bias
National Institutes of Health. (2014). A decade of discovery: in patient-reported outcome results: A meta-analysis. Quality
The NIH roadmap and common fund. https://commonfund.nih. of Life Research, 25(3), 559–574. https://doi.org/10.1007/
gov/sites/default/files/ADecadeofDiscoveryNIHRoadmapCF.pdf s11136-015-1110-8
National Quality Forum. (2009). National voluntary consensus Schwartz, C. E., Powell, V. E., & Rapkin, B. D. (2017). When
standards for patient outcomes: A consensus report. global rating of change contradicts observed change: Examining
Neumann, S., Quinting, J., Rosenkranz, A., de Beer, C., Jonas, K., appraisal processes underlying paradoxical responses over time.
& Stenneken, P. (2019). Quality of life in adults with neuro- Quality of Life Research, 26(4), 847–857. https://doi.org/10.1007/
genic speech-language-communication difficulties: A systematic s11136-016-1414-3
review of existing measures. Journal of Communication Disorders, Schwartz, C. E., & Rapkin, B. D. (2004). Reconsidering the psy-
79, 24–45. https://doi.org/10.1016/j.jcomdis.2019.01.003 chometrics of quality of life assessment in light of response shift
Noonan, V. K., Lyddiatt, A., Ware, P., Jaglal, S. B., Riopelle, R. J., and appraisal. Health and Quality of Life Outcomes, 2, 16. https://
Bingham, C. O., III., Figueiredo, S., Sawatzky, R., Santana, M., doi.org/10.1186/1477-7525-2-16
Bartlett, S. J., & Ahmed, S. (2017). Montréal Accord on Patient- Simmons-Mackie, N., Threats, T. T., & Kagan, A. (2005). Outcome
Reported Outcomes (PROs) use series—Paper 3: Patient-reported assessment in aphasia: A survey. Journal of Communication
outcomes can facilitate shared decision-making and guide self- Disorders, 38(1), 1–27. https://doi.org/10.1016/j.jcomdis.2004.
management. Journal of Clinical Epidemiology, 89, 125–135. 03.007
https://doi.org/10.1016/j.jclinepi.2017.04.017 Snyder, C. F., Aaronson, N. K., Choucair, A. K., Elliott, T. E.,
Patrick, D. L., Burke, L. B., Powers, J. H., Scott, J. A., Rock, E. P., Greenhalgh, J., Halyard, M. Y., Hess, R., Miller, D. M., Reeve,
Dawisha, S., O’Neill, R., & Kennedy, D. L. (2007). Patient- B. B., & Santana, M. (2012). Implementing patient-reported
reported outcomes to support medical product labeling claims: outcomes assessment in clinical practice: A review of the options
FDA perspective. Value in Health, 10(Suppl. 2), S125–S137. and considerations. Quality of Life Research, 21(8), 1305–1314.
https://doi.org/10.1111/j.1524-4733.2007.00275.x https://doi.org/10.1007/s11136-011-0054-x
Portney, L. G., & Watkins, M. P. (2015). Foundations of clinical Thomas, M. L. (2011). The value of item response theory in clini-
research: Applications to practice (3rd ed.). F.A. Davis Company. cal assessment: A review. Assessment, 18(3), 291–307. https://
PROMIS Health Organization. (2013). PROMIS Instrument de- doi.org/10.1177/1073191110374797
velopment and psychometric evaluation scientific standards, Threats, T. T. (2012). Use of the ICF for guiding patient-reported
Version 2.0. http://www.healthmeasures.net/images/PROMIS/ outcome measures. SIG 2 Perspectives on Neurophysiology
PROMISStandards_Vers2.0_Final.pdf and Neurogenic Speech and Language Disorders, 22(4), 128–135.
Rao, P. R. (2015). Outcomes and quality: Key characteristics of a https://doi.org/10.1044/nnsld22.4.128
successful SLP value journey. SIG 2 Perspectives on Neuro- Torrence, J. M., Baylor, C. R., Yorkston, K. M., & Spencer, K. A.
physiology and Neurogenic Speech and Language Disorders, 25(3), (2016). Addressing communicative participation in treatment
94–106. https://doi.org/10.1044/nnsld25.3.94 planning for adults: A survey of U.S. speech-language patholo-
Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical gists. American Journal of Speech-Language Pathology, 25(3),
model of quality-of-life appraisal: Implications of findings from 355–370. https://doi.org/10.1044/2015_AJSLP-15-0049
studies of response shift. Health and Quality of Life Outcomes, Tucker, F. M., Edwards, D. F., Mathews, L. K., Baum, C. M., &
2, 14. https://doi.org/10.1186/1477-7525-2-14 Connor, L. T. (2012). Modifying health outcome measures for
Revicki, D. A., Cella, D., Hays, R. D., Sloan, J. A., Lenderking, people with aphasia. The American Journal of Occupational
W. R., & Aaronson, N. K. (2006). Responsiveness and minimal Therapy, 66(1), 42–50. https://doi.org/10.5014/ajot.2012.001255
important differences for patient reported outcomes. Health Tulsky, D. S., Kisala, P. A., Victorson, D., Carlozzi, N., Bushnik,
and Quality of Life Outcomes, 4, 70. https://doi.org/10.1186/ T., Sherer, M., Choi, S., Heinemann, A., Chiaravalloti, N.,
1477-7525-4-70 Sander, A., Englander, J., Hanks, R., Kolakowsky-Hayner, S.,
Risser, A. H., & Spreen, O. (1985). The Western Aphasia Battery. Roth, E., Gershon, R., Rosenthal, M., & Cella, D. (2016). TBI-
Journal of Clinical and Experimental Neuropsychology, 7(4), QOL: Development and calibration of item banks to measure
463–470. https://doi.org/10.1080/01688638508401277 patient reported outcomes following traumatic brain injury.
Rodriguez-Correa, C., Biber, J., Hess, R., Spertus, J. A., Nativi- The Journal of Head Trauma Rehabilitation, 31(1), 40–51. https://
Nicolau, J., Peritz, D., Walker, A., Hess, J., Drakos, S. G., doi.org/10.1097/HTR.0000000000000131
Wever-Pinzon, O., McKellar, S., Zickmund, S., Gilbert, E. M., Tulsky, D. S., Kisala, P. A., Victorson, D., Tate, D. G., Heinemann,
Fang, J., Selzman, C. H., & Stehlik, J. (2017). Real-time assess- A. W., Charlifue, S., Kirshblum, S. C., Fyffe, D., Gershon, R.,
ment of patient reported outcomes in heart failure clinic. Journal Spungen, A. M., Bombardier, C. H., Dyson-Hudson, T. A.,
of Cardiac Failure, 23(8), S29. https://doi.org/10.1016/j.cardfail. Amtmann, D., Kalpakjian, C. Z., Choi, S. W., Jette, A. M.,
2017.07.074 Forchheimer, M., & Cella, D. (2015). Overview of the Spinal
Rogers, M. (2018, June). Assessing the evidence: Rules and tools. Cord Injury–Quality of Life (SCI-QOL) measurement system.
Paper presented at the Eighteenth Annual Barbara P. Mastriano The Journal of Spinal Cord Medicine, 38(3), 257–269. https://
Communiversity Conference,, Philadelphia, PA, United States. doi.org/10.1179/2045772315Y.0000000023

Cohen & Hula: Patient-Reported Outcomes 369

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions
U.S. Department of Health and Human Services FDA Center for Wang, L.-W., Miller, M. J., Schmitt, M. R., & Wen, F. K. (2013).
Drug Evaluation and Research, U.S. Department of Health and Assessing readability formula differences with written
Human Services FDA Center for Biologics Evaluation and Re- health information materials: Application, results, and
search, & U.S. Department of Health and Human Services FDA recommendations. Research in Social and Administrative
Center for Devices and Radiological Health. (2009). Guidance Pharmacy, 9(5), 503–516. https://doi.org/10.1016/j.sapharm.
for industry: Patient-reported outcome measures: Use in medi- 2012.05.009
cal product development to support labeling claims. Health Williams, C. A., Templin, T., & Mosley-Williams, A. D. (2004).
and Quality of Life Outcomes, 4(1), 79. https://doi.org/10.1186/ Usability of a computer-assisted interview system for the
1477-7525-4-79 unaided self-entry of patient data in an urban rheumatol-
U.S. Food and Drug Administration. (2015). Clinical Outcome ogy clinic. Journal of the American Medical Informatics
Assessment (COA): Glossary of terms. Association, 11(4), 249–259. https://doi.org/10.1197/jamia.
Velikova, G., Booth, L., Smith, A. B., Brown, P. M., Lynch, P., M1527
Brown, J. M., & Selby, P. J. (2004). Measuring quality of life in Williams, L. S., Bakas, T., Brizendine, E., Plue, L., Tu, W., Hendrie,
routine oncology practice improves communication and patient H., & Kroenke, K. (2006). How valid are family proxy assess-
well-being: A randomized controlled trial. Journal of Clinical On- ments of stroke patients’ health-related quality of life? Stroke,
cology, 22(4), 714–724. https://doi.org/10.1200/JCO.2004.06.078 37(8), 2081–2085. https://doi.org/10.1161/01.STR.0000230583.
Wagner, L. I., Schink, J., Bass, M., Patel, S., Diaz, M. V., 10311.9f
Rothrock, N., Pearman, T., Gershon, R., Penedo, F. J., Rosen, World Health Organization. (1947). Constitution of the World
S., & Cella, D. (2015). Bringing PROMIS to practice: Brief Health Organization.
and precise symptom screening in ambulatory cancer care. Wyrwich, K. W., Norquist, J. M., Lenderking, W. R., Acaster, S.,
Cancer, 121(6), 927–934. https://doi.org/10.1002/cncr.29104 & Industry Advisory Committee of International Society for
Wallace, S. J., Worrall, L., Rose, T., & Le Dorze, G. (2016). Core Quality of Life Research. (2013). Methods for interpreting
outcomes in aphasia treatment research: An e-Delphi consen- change over time in patient-reported outcome measures. Quality
sus study of international aphasia researchers. American Jour- of Life Research, 22(3), 475–483. https://doi.org/10.1007/
nal of Speech-Language Pathology, 25(4S), S729–S742. https:// s11136-012-0175-x
doi.org/10.1044/2016_AJSLP-15-0150 Yorkston, K. (2015, December). Selecting patient-reported outcomes
Wallace, S. J., Worrall, L., Rose, T., Le Dorze, G., Cruice, M., measures: A committee update. American Speech-Language-
Isaksen, J., Pak Hin Kong, A., Simmons-Mackie, N., Scarinci, N., Hearing Association. https://www.asha.org/articles/selecting-
& Gauvreau, C. A. (2017). Which outcomes are most important patient-reported-outcomes-measures-/
to people with aphasia and their families? An international Yorkston, K., & Baylor, C. (2019). Patient-reported outcomes
nominal group technique study framed within the ICF. Disability measures: An introduction for clinicians. Perspectives of the
and Rehabilitation, 39(14), 1364–1379. https://doi.org/10.1080/ ASHA Special Interest Groups, 4(1), 8–15. https://doi.org/
09638288.2016.1194899 10.1044/2018_PERS-ST-2018-0001

370 American Journal of Speech-Language Pathology • Vol. 29 • 357–370 • February 2020

Downloaded from: https://pubs.asha.org 103.213.128.15 on 04/30/2021, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions