Research article

Measuring grief and loss after spinal cord

injury: Development, validation and
psychometric characteristics of the SCI-QOL
Grief and Loss item bank and short form
Claire Z. Kalpakjian 1, David S. Tulsky2,3, Pamela A. Kisala 2,
Charles H. Bombardier 4
1
Department of Physical Medicine and Rehabilitation, University of Michigan Medical School, University of
Michigan, Ann Arbor, MI, USA, 2New York University Langone Medical Center, New York, NY, USA, 3Kessler
Foundation Research Center, West Orange, NJ, USA, 4Department of Rehabilitation Medicine, University of
Washington, Seattle, WA, USA

Objective: To develop an item response theory (IRT) calibrated Grief and Loss item bank as part of the Spinal
Cord Injury – Quality of Life (SCI-QOL) measurement system.
Design: A literature review guided framework development of grief/loss. New items were created from focus
groups. Items were revised based on expert review and patient feedback and were then field tested.
Analyses included confirmatory factor analysis (CFA), graded response IRT modeling and evaluation of
differential item functioning (DIF).
Setting: We tested a 20-item pool at several rehabilitation centers across the United States, including the
University of Michigan, Kessler Foundation, Rehabilitation Institute of Chicago, the University of Washington,
Craig Hospital and the James J. Peters/Bronx Department of Veterans Affairs hospital.
Participants: A total of 717 individuals with SCI answered the grief and loss questions.
Results: The final calibrated item bank resulted in 17 retained items. A unidimensional model was observed
(CFI = 0.976; RMSEA = 0.078) and measurement precision was good (theta range between −1.48 to 2.48).
Ten items were flagged for DIF, however, after examination of effect sizes found this to be negligible with little
practical impact on score estimates.
Conclusions: This study indicates that the SCI-QOL Grief and Loss item bank represents a psychometrically
robust measurement tool. Short form items are also suggested and computer adaptive tests are available.
Keywords: Grief, Spinal cord injuries, Patient outcomes assessment, Quality of life, Psychometrics

Introduction to understanding adjustment to SCI.2 Most salient is

Loss is a universal experience. Bereavement, the loss of a that grief and bereavement classically focus on the loss
loved one, and grief, the distress resulting from bereave- of a loved one. Grief also has been used to characterize
ment,1 are concepts that patients and their loved ones reactions to other losses, but the extent to which models
naturally use to understand reactions to catastrophic of grief can be applied to SCI-related loss of physical
losses like spinal cord injury (SCI). However, questions capacity, social or occupational role function, and life
remain about the applicability of grief and bereavement goals remains an empirical question. Moreover, measur-
ing grief and loss, the focus of this paper, must be distin-
Correspondence to: David S. Tulsky, Professor and Director of the Center on guished from theories about the process of grief,
Assessment Research and Translation, STAR Campus, University of especially those characterized by Kubler-Ross’ stages
Delaware, 540 S. College Ave, Newark, DE 19713, USA. Email: dtulsky@
udel.edu. of grief.3

© The Academy of Spinal Cord Injury Professionals, Inc. 2015

MORE OpenChoice articles are open access and distributed under the terms
of the Creative Commons Attribution License 3.0
DOI 10.1179/2045772315Y.0000000015 The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3 347
Kalpakjian et al. Grief and spinal cord injury

Despite its heuristic appeal, grief is an abandoned adaptation to injury.18 Qualitative work suggests a
concept in SCI rehabilitation. Without empirical process of adaption and meaning-making, characterized
support, grief was thought to be universal after injury, by seeking a balance between holding onto pre-injury
required to achieve positive adjustment, and occurring goals and adaptation to life after injury.19
in a particular way, commonly described as ‘stage Given theoretical advances in the field of grief and
theory.’4,5 Grief also lacked a strong foundation in bereavement, we speculated that these perspectives
measurement. Rehabilitation psychologists eventually may also benefit our understanding of adjustment to
moved towards the measurement of core psychopatholo- losses associated with SCI. The concept of examining
gical constructs, such as depression6 and post-traumatic trajectories of adjustment has already been applied
stress disorder,7 as well as positive psychological con- to people with SCI; however, with few exceptions,20
structs, such as coping and appraisal.8,9 assessment of adjustment to SCI has been limited to
In the meantime, significant theoretical, empirical, measures of depression and anxiety.13,21 Popular scales
measurement, and treatment advancements have such as Core Bereavement Items,22 Texas Revised
occurred in the field of grief and bereavement. In Inventory of Grief,23 or the Inventory of Complicated
terms of theory, Stroebe and Shut10 have described the Grief24 focus on the loss of a loved one. Construct val-
influential dual-process model of coping with bereave- idity of such scales is examined almost exclusively in
ment. From this perspective, the bereaved are thought respondents who have experienced the death of a loved
to oscillate between a focus on loss and a focus on res- one. As such, the usefulness and validity of such items
toration in daily activities. In a groundbreaking prospec- is limited for persons with SCI, where losses are
tive study of spousal bereavement, Bonanno et al. 11 focused not on the loss of another, but on losses
described five trajectories of responses to loss, including related to the self. To address the gap in the availability
resilience as the modal response. Since then, similar tra- of relevant and valid measurement tools to measure
jectories have been demonstrated in response to other grief processes after SCI, we sought to create a new
forms of trauma, including SCI.12,13 Newer conceptual- item bank to capture features of grief/loss that were
izations of grief also suggest that the context of the loss, highly relevant to the losses experienced by persons
characterized by factors such as suddenness, unexpected- with SCI.
ness, or violence, may be critical predictors of outcome.14
Controversy remains about how to define concepts Methods
such as normal, complicated and prolonged grief. This study was approved by the Institutional Review
Nevertheless, there is growing evidence that pathologi- Board at all sites. The first study activity was to
cal grief occurs in a significant minority of people who develop and refine a Grief and Loss item pool. Next,
sustain loss and is distinct from ‘normal’ grief, anxiety, grief and loss items were administered to a large
and depression.15 Furthermore, pathological grief is sample of people with SCI using a computerized data
associated with functional impairment, physiological collection platform and interview format, so that each
changes, reduced quality of life, poor self-care, and elev- question was read to the respondent by a trained inter-
ated suicidal ideation and suicide attempts, after con- viewer and responses were directly entered into the data-
trolling for depression and anxiety.15 Prigerson et al. 15 base. Each of these steps is described in detail in Tulsky
posit that key symptoms of pathological grief include et al. 25 and is also outlined briefly in the section below.
yearning for what was, accompanied by other cognitive,
emotional and behavioral symptoms, such as difficulty Development of a grief and loss item pool
accepting the loss, emotional numbness, avoidance of The Grief and Loss item bank assesses emotional reac-
reminders and confusion about one’s role in life. tions or grief such as anger, guilt, anxiety, sadness, and
Distinguishing abnormal grief from depression also despair. Items comprising the Grief and Loss item bank
may have significant treatment implications. An influen- were drawn from various sources. We began by identify-
tial clinical trial based on the dual process model of grief ing candidate items from our initial pilot work, which
demonstrated that interpersonal therapy (IPT) alone included individual, semi-structured interviews and
was not as effective a treatment for complicated grief focus groups with individuals with SCI and clinicians
as was a combination of IPT and exposure therapy.16 with SCI (see Tulsky et al.25,26 for a full description).
Making meaning of losses is another emerging dimen- From these data, we developed a set of two preliminary
sion of grief and bereavement, particularly when the items related to grief and loss. To develop these new
loss is sudden or unexpected.17 The search for items, specific phrases or concepts were drawn from
meaning after injury has been associated with positive the interviews and focus group transcripts (12 items

348 The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3

 Kalpakjian et al. Grief and spinal cord injury

from pilot interviews and 20 items from focus groups). changed to ‘I was overwhelmed by everything that has
For example, from the patient focus group quote, happened to me,’ since there is no generic word for
‘[T]he first thing that came to my mind is, I guess, the ‘upset’ in some other languages such as Spanish. All
things that you miss, you know, that you could do items were written at or below the 5th grade reading
before. Yeah, even people say you can do them, you can level. A final item pool of 20 items was then field-tested.
do them in a different way, but I can’t walk my dog effort-
lessly like I used to. If he gets loose, I can’t just go over Calibration study participants and data collection
there and grab him and bring him back…’ we drafted procedures
the item, It made me sad to think about the things I As a part of a large-scale, multi-site item calibration
used to enjoy. Additionally, one item from the Neuro- study (sites included the Kessler Foundation,
QOL27 Stigma bank, I lost friends by telling them that University of Michigan, Rehabilitation Institute of
I have this [injury], was originally categorized as a Chicago, University of Washington, Craig Hospital
SCI-QOL Grief/Loss item raising the total pool to 33 and the James J. Peters/Bronx Veterans
items. Administration hospital), we administered the 20 grief
Expert item review (EIR), the first phase of the quali- and loss items along with other item pools reflecting
tative item review process used by the PROMIS28 and different emotional health subdomains to a sample of
Neuro-QOL27 project teams, was used to optimize this people with SCI.
preliminary pool of 33 items. Items were considered can- The calibration sample included 717 participants with
didates for deletion if they were redundant, overly SCI. Inclusion criteria were 18 years of age and older,
generic, too specific, or if they failed to represent the ability to read and understand English, and had a medi-
construct or domain definition. Items that were cally documented traumatic SCI. The sample was strati-
verbose, double-barreled (i.e. one item with content fied by level ( paraplegia versus tetraplegia),
related to two different issues), poorly or inconsistently completeness of injury (complete vs. incomplete), and
worded, or contained high-level vocabulary were time since injury (<1 year, 1–3 years, and >3 years) to
revised. For example, the item ‘It made me sad to ensure that the final sample was a heterogeneous
think about the things I used to enjoy’ (mentioned sample of individuals with SCI. Each participant’s diag-
above) was to read ‘I missed the activities I used to nosis was confirmed by medical record review; neuro-
do.’ Based on EIR feedback, 30 items were retained in logic level was documented by their most recent
the preliminary Grief and Loss item pool. Preliminary American Spinal Injury Association Impairment Scale
items then underwent an additional phase of item (AIS) rating.32 To ensure a consistent mode of adminis-
review and modification by members of the investigative tration across participants, all items were presented in a
team. Items were arranged on a hierarchy of ‘difficulty’, structured interview format, either in person or over the
from items indicating the lowest degree of grief and loss phone. The context (time frame) for all items was, ‘In
to the highest degree of grief and loss. Team members the past 7 days…,’ and the response options were
removed redundant items where there was oversatura- Never/Rarely/Sometimes/Often/Always. A more
tion in the middle range of the hierarchy, and suggested detailed description of the study methodology and pro-
new items to fill gaps in content coverage. Specifically, 3 cedures is provided elsewhere in this issue.25
items were deleted during this phase of review, 6 were
moved to other item banks (e.g. Trauma, Stigma) and Data analyses
5 new items were added. Analysis involved confirmation of construct unidimen-
We then conducted cognitive interviews29 with at least sionality, use of a graded-response IRT model to cali-
five individuals with SCI (see Introduction paper in this brate item parameters, and examination of DIF. We
issue for details on the methodology) to assess item used CFA to determine if our items conformed to a uni-
comprehension, decision making, and response retrieval dimensional model. Acceptable model fit indices were:
processes. After cognitive interviewing, 7 items were CFI > 0.90, RMSEA < 0.08, good; CFI > 0.95,
removed and 4 items were modified. After this phase, RMSEA < 0.06, excellent). Calibration was performed
the final 20 items were reviewed for translatability (for using iterative methods to reduce the item pool and
method, please see Eremenco et al.30) and reading obtain the best-fitting item parameters that would best
level (using the Lexile framework31). Slight modifi- allow estimation of a participant’s standing on a trait
cations were made to 3 items after the translatability of grief and loss. With each successive analytic iteration,
and cultural review. For example, the item ‘I was upset we identified poorly fitting items by examining item fit
about everything that has happened to me’ was to the 2-PL IRT model, DIF, local dependence

The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3 349

Kalpakjian et al. Grief and spinal cord injury

between items (residual correlations >|0.15|), and sig- Table 1 Demographic and injury characteristics of the
calibration sample
nificant loadings on the single factor (values >0.30).
We then removed these items from the item pool and Emotional domain sample,
repeated the analytic steps. Once an acceptable solution Variable N = 716; Mean (SD), N (%)
was reached with CFA statistics that supported a unidi- Age (years) 43.0 (15.3)
mensional model, and all items showing misfit to the Age at injury (years) 36.1 (16.8)
model or DIF were removed, the final IRT parameters Sex
Male 558 (78%)
were utilized to develop a computer adaptive test Female 158 (22%)
(CAT) algorithm for the Grief and Loss item bank. Ethnicity
Hispanic 81 (11%)
The CAT was programmed on the Assessment Center Non-Hispanic 631 (88%)
website (http://www.assessmentcenter.net) and can be Not reported/Refused 4 (1%)
administered directly from there. The item parameters Race
Caucasian 505 (70%)
were also used to select items for a static short form African-American 125 (17%)
which can also be downloaded as a PDF from the Asian 8 (1%)
Assessment Center website. Tulsky et al.25 within this American Indian/Alaska Native or 7 (1%)
Native Hawaiian/Pacific Islander
special issue described the detailed methodology and More than one race 9 (1%)
data analysis plan. Other 49 (7%)
Not provided/Refused 4 (1%)
Level of Education
Development of short forms High school or less 274 (38.3%)
To select items for short, fixed-length forms (as an Some college 248 (34.6%)
Bachelor’s degree or more 194 (27.1%)
alternative to CATs), project investigators reviewed
Time Since Injury 7.1 (10.0)
item difficulty (item location) and slope (discrimi- <1 year post injury 195 (27%)
nation). As a starting point, items were divided into 1–3 years post injury 186 (26%)
>3 years post injury 335 (47%)
quintiles based on location; at each quintile, the first Diagnosis
and/or second items with the highest slope were Paraplegia complete 182 (25%)
selected. Other considerations for item selection were Paraplegia incomplete 143 (20%)
Tetraplegia complete 157 (22%)
clinical relevance, item wording and similarity to other Tetraplegia incomplete 230 (32%)
items with the goal of having short form items as Unknown 4 (0%)
diverse as possible. Therefore, selection of items for
short forms used both item statistics and qualitative
five responses) in any category and no items had a cat-
characteristics.
egory inversion. No further items were removed at this
time.
Results
Participant characteristics Dimensionality
Demographic and injury characteristics are summarized Using CFA, a unidimensional model was observed
in Table 1. Please see Tulsky et al.33 introductory article (CFI = 0.976; RMSEA = 0.078). The R 2 values for all
within this special issue for additional details on the cali- 17 of the items were greater than 0.40 In terms of
bration sample, including education, income and mech- local dependence, no item pairs were identified (i.e.
anism of injury. residual correlations >|0.15|). Eigenvalue ratio (first to
second) was 11.8.
Preliminary analysis and item removal
Of the original 20 items that were tested, 3 were removed IRT parameter estimation and model fit
for the following reasons: local item dependence, low Slopes ranged from 1.65 to 3.15; thresholds ranged from
item-total correlation, and DIF for gender (Grief_31, −1.48 to 2.48 (see Table 2).
“I cried when I was reminded of the abilities I used to The measurement precision in the theta range
have”), respectively. For the final 17 retained items, between −0.8 and 1.8 is roughly equivalent to a classical
internal consistency was excellent (Cronbach’s α = reliability of 0.95 or better (Fig. 1).
0.947) and item-total correlations ranged from 0.59 to The S-X2 model fit statistics were examined using the
0.78. All of the items but one had more than 20% of IRTFIT macro program. All items had adequate or
the sample selecting the first category of category of 1 better model fit statistics (P < 0.05), with marginal
(‘Never’). One case was deleted due to excessive reliability equal to 0.947 and no item pairs were
missing data. No items had sparse data (fewer than flagged (|r|> = 0.4) for local dependence.

350 The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3

 Kalpakjian et al. Grief and spinal cord injury

Table 2 Grief and Loss items and item bank parameters

Item response theory calibration statistics

Threshold Threshold Threshold Threshold
Item ID Item stem Slope 1 2 3 4

Grief_14 I spent a lot of time thinking about what I have lost since 2.08941 −0.79108 0.07601 0.91063 1.73104
my injury
Grief_16 I felt sad thinking about things I used to enjoy 3.00519 −0.71947 −0.11166 0.78169 1.48040
Grief_15 Because of my injury, I felt like I lost many opportunities 2.49928 −0.75116 −0.22986 0.56348 1.21498
Grief_29 I felt that I lost my former life 3.01344 −0.44494 0.02941 0.70042 1.21062
Grief_10 I had difficulty accepting my injury 2.18233 0.05849 0.64106 1.37461 1.82508
Grief_7 I longed for the life I had before my injury 2.61178 −0.94542 −0.33790 0.41467 0.91664
Grief_2 I missed out on life because of my injury 2.53096 −0.45129 0.10510 0.88385 1.51899
Grief_21 I have lost spontaneity in my life. 1.64954 −0.45633 0.21509 1.13592 2.04732
Grief_13 Because of my injury, I was distressed about the abilities 2.91124 −0.72949 −0.08964 0.81757 1.59383
that I have lost
Grief_30 I was overwhelmed by everything that has happened to 1.74946 −0.00510 0.80341 1.82045 2.47958
me
Grief_28 I missed the activities I used to do 2.44946 −1.48060 −0.93461 0.04140 0.68014
Grief_20 Because of my injury, I had difficulty adjusting to the 1.84725 −0.40213 0.25354 1.46142 2.35339
changes in my body
Grief_11 I felt that my injury has taken away my future 3.15448 −0.19293 0.22748 1.06119 1.54145
Grief_9 I questioned why I was injured 1.69676 0.16936 0.69776 1.49720 2.00782
Grief_6 I felt lost because of my injury 2.92100 0.30080 0.78731 1.62381 2.35060
Grief_1 I felt I lost time because of my injury 2.10622 −0.62952 −0.14542 0.70884 1.45129
Grief_24 I felt that I am not who I used to be 2.37777 −0.67015 −0.06706 0.68927 1.25193

Context for all grief and loss items was ‘In the past 7 days…’; Response set was 1 = Never/2 = Rarely/3 = Sometimes/4 = Often/5 =
Always.
Items in bold represent short form selections. Items and parameters copyright © 2015 David Tulsky and Kessler Foundation. All Rights
Reserved. Scales should be accessed and used through the corresponding author or http://www.assessmentcenter.net. Do not
modify items without permission from the copyright holder.

Differential item functioning (incomplete n = 374 vs. complete n = 339), and time
DIF was examined using lordif34 for six categories: age post injury (<1 year n = 196 vs. >1 year n = 521).
(≤49 vs. ≥50), sex (male n = 559 vs. female n = 158), Items were flagged for possible DIF when the prob-
education (some college and lower n = 523 vs. ability associated with the χ 2 test was <0.01 and the
college degree and above n = 194), diagnosis (tetraplegia effect size measures (McFadden’s pseudo R 2) >0.02,
n = 388 vs. paraplegia n = 325), injury severity which is a small but non-negligible effect. Overall, 10
of the final items were flagged for DIF in at least one
category based on the chi-square test; however, when
the effect size measures were examined, the DIF was
negligible and all 17 items were retained in the final,
calibrated item bank.

Short form selection and mode of administration

Once the SCI-QOL Grief and Loss item bank was
finalized, all items and parameters were programmed
into the Assessment CenterSM 35 platform and the
bank is now freely available as a CAT. Since the
purpose of calibrating items using IRT is that only a
subset of items needs to be administered from a
given bank in order to estimate an individual’s
score, there is flexibility as to how the items are
selected and administered. On the Assessment
Center platform, the CAT administration parameters
can be modified to reduce standard error variance
Figure 1 SCI-QOL Grief and Loss Item Bank Information and (e.g. maximize reliability), or to reduce test burden.
Precision. There is also a predetermined static short form that

The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3 351

Kalpakjian et al. Grief and spinal cord injury

Table 3 Accuracy of variable- and fixed-length CAT and 9-item short form: correlations with full-bank score

# Items admin
Mode N Mean SD Min Max %Min %Max Corr. w/Full bank

Variable-length CAT (min 4) 716 6.0 2.7 4 12 42.7 13.0 0.98

Variable-length CAT (min 8) 716 8.6 1.4 8 12 82.3 13.0 0.99
9-Item fixed-length CAT 716 9 0 9 9 n/a n/a 0.99
9-Item short form 716 9 0 9 9 n/a n/a 0.99

can be downloaded. Finally, the individual items are this need, the Grief and Loss and other SCI-QOL
present and could be selected if the end user wanted item banks are also available as short forms. The
to administer a specific item. These administration project investigators utilized psychometric and clini-
options are reviewed below. cal input to develop a fixed, 9-item ‘short form’
The SCI-QOL utilizes the same default CAT discon- version of the Grief and Loss item bank. The goal
tinue criteria as PROMIS; namely, the CAT minimum of the short form selection process was to include
number of items to administer is four and the the most informative items across a wide range of
maximum is 12 with a maximum standard error of ‘difficulty’, or amount of the underlying trait. Since
0.3. In other words, in the default settings, the CAT all items are calibrated on the same metric, scores
will always administer at least 4 items, then will discon- on the short form are directly comparable to those
tinue when the standard error of the individual’s score on the CAT or full item bank. The correlation of
estimate drops below 0.3 or a maximum of 12 items is the short form and various CATs with the full bank
reached (and the standard error variance criterion are given in Table 3. Short forms may be adminis-
cannot be met). tered directly within Assessment Center, or may be
Alternatively, the user could change the ‘discontinue downloaded for administration by paper and pencil
criteria’ of the CAT so that it will administer additional or an alternate data capture platform or system.
items and obtain a more precise assessment of function- Individual investigators or clinicians could also
ing. For instance, if the user selected an option that the develop additional, custom short forms, which
CAT administers a minimum of 8 items before disconti- could then be scored on the same IRT-based metric
nuing, a lengthier test would be administered, but a with the help of a psychometrician.
more reliable score will be obtained. In some cases, To determine the degree of measurement precision
greater precision over test burden is desirable based on and error for these assessments, we compared the
factors such as resource allocation where specificity is reliability of the full bank, 9-item short form, and
critical. variable-length CAT with the default minimum of
However, in some cases it is neither possible (e.g. 4 items. When we compared the reliability of a
internet unavailable) nor practical (e.g. laptop/ CAT that was either fixed to 8 items, or a variable-
tablet computer equipment beyond budget of length CAT with a minimum of 8 items, CAT
project) to administer items via CAT. To address values for both reliability (Fig. 2) and precision

Figure 2 Measurement reliability by T-Score and assessment method.

352 The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3

 Kalpakjian et al. Grief and spinal cord injury

Table 4 Breadth of coverage for variable length CAT, fixed length CAT, 9-item short form, and full item bank

T score Standard error

Mode N Mean ± SD Range % Ceiling % Floor Mean ± SD Range

Variable-length CAT (min 4) 716 50.0 ± 9.6 29.4–79.8 0.3% 4.9% 0.30 ± 0.05 0.26–0.49
Variable-length CAT (min 8) 716 50.0 ± 9.6 29.4–79.8 0.3% 4.9% 0.56 ± 0.06 0.20–0.49
9-Item fixed-length CAT 716 49.9 ± 9.6 29.9–79.1 0.3% 5.4% 0.25 ± 0.07 0.19–0.49
9-Item short form 716 50.0 ± 9.5 31.0–76.0 1.4% 6.2% 0.27 ± 0.08 0.20–0.51
Full bank 716 49.9 ± 9.7 29.1–80.0 0.3% 4.8% 2.18 ± 0.75 0.16–0.48

(Table 4) demonstrated improvement over the short of 10; this is based on the SCI-QOL calibration data;
form values. Table 4 presents the mean, standard that is, a mean of 50 reflects the mean of an SCI popu-
deviation, range, and standard error ranges for the lation rather than the general population. All CAT
various administration modes. Additionally, administrations of the SCI-QOL Grief and Loss item
reliability curves for the full bank, short form, vari- bank are automatically scored by Assessment Center.
able length CAT (minimum of 4 items) and fixed- When administering the short form, whether via
length CAT (8 items) are given in Fig. 2. Assessment Center, paper and pencil, or another data
capture platform, an individual must complete all 8
Scoring component items in order to receive a score. The raw
SCI-QOL Grief and Loss scores are standardized on a score for the short form is computed by simply
T-metric, with a mean of 50 and a standard deviation summing the response scores for the individual com-
ponent items. The T-score and associated standard
Table 5 T-score lookup table for SCI-QOL Grief/Loss SF9a error for each raw score value is given in Table 5.

Raw score Scaled score Standard error Reliability

9 30.9 5.1 As a part of the reliability study described in the Tulsky
10 35.3 3.9 et al. 25 methods paper in this issue, we compared Grief/
11 37.4 3.6 Loss scores at Baseline with those from the 1-2 week
12 39.4 3.3
13 41.0 3.1 retest assessment. In a sample of 245 individuals with
14 42.3 2.9 SCI, Pearson’s r = 0.84 and ICC (2,1) = 0.83 (95%
15 43.6 2.8
CI = 0.78 to 0.87).
16 44.7 2.7
17 45.8 2.6
18 46.7 2.5 Discussion
19 47.7 2.5
20 48.6 2.5
We developed the SCI-QOL Grief and Loss item bank
21 49.4 2.4 to assess an individual’s subjective experience of losses
22 50.3 2.4 that is relevant to people with SCI. This new item
23 51.1 2.4
24 51.9 2.4 bank is a departure from the status quo of currently
25 52.7 2.4 available grief measures that focus primarily on death
26 53.5 2.4 of a loved one. In contrast, items in the new item bank
27 54.3 2.4
28 55.1 2.4 reflect losses of a life before injury, vision of a future
29 55.9 2.4 life, time and opportunities, and the self they used to
30 56.7 2.4
be. All of the items in the Grief and Loss item bank
31 57.5 2.4
32 58.3 2.4 were developed directly from verbatim interview or
33 59.1 2.4 focus group quotes; no items were drawn from
34 60.0 2.4
35 60.8 2.5
PROMIS or Neuro-QOL. As a result, the new SCI-
36 61.8 2.5 QOL Grief and Loss item bank fills an important gap
37 62.7 2.5 in existing grief and bereavement instruments as well
38 63.7 2.6
39 64.7 2.7 as these important patient oriented outcome measure-
40 65.9 2.8 ment initiatives.
41 67.2 2.9 The use of IRT to calibrate the SCI-QOL Grief and
42 68.6 3.0
43 70.2 3.3 Loss items has yielded a variety of administration
44 72.3 3.5 options including a short form and CAT. The CAT out-
45 76.1 4.4
performed the short form in reliability and precisions; as

The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3 353

Kalpakjian et al. Grief and spinal cord injury

such, if a user’s goal is to optimize reliability, especially Conclusion

at the ceiling and floor of the distribution, we would rec- The final SCI-QOL Grief and Loss item bank contains
ommend administering the Grief and Loss item bank as 17, IRT-calibrated items. This item bank is the first, to
a CAT with a minimum of 8 items. In cases where it may our knowledge, to specifically capture grief and loss
not be feasible or practical to administer items via CAT/ after SCI. Because of the flexibility of IRT-based
Assessment Center, or if having participants answer the measures, the use of CATs is also possible with this
same subset of 8 items is necessary to answer a given item bank, enabling researchers and clinicians to admin-
research question, we would recommend short form ister only the most precise and informative items based
administration. An additional administration option is on an individual’s responses. This has important impli-
to administer both the CAT and any short form items cations for the use of such innovative applications in
not included in the CAT by using the ‘no duplicates’ symptom monitoring and self-management in post-
option in Assessment Center. In this way, the user acute care settings and community living.
could optimize reliability and have the option of directly To the best of our knowledge, this is the first time that
comparing individuals’ responses on specific items to a patient-centered, modern measurement theory derived
each other or to themselves over time. approach has been used to develop and test a grief and
loss self-reported measurement tool specifically for indi-
viduals with SCI. Our work using focus groups and
interviews strengthened our understanding of grief and
loss as it is experienced in the context of SCI. While
Clinical applications this work has made great strides forward, there
The flexibility of methods to administer the SCIQOL remains work to be done to more comprehensively
Grief and Loss item bank provides scientists and clini- define grief and loss in this population. This new item
cians with an efficient and accessible way to integrate bank makes an important first step towards reliably
the measurement of grief and loss specific to SCI into measuring the construct of grief and loss in persons
research and ultimately clinical practice. Next, research with SCI. Such measurement will significantly contrib-
is needed to examine the validity of this Grief and Loss ute to the development of conceptual models. Future
measure. For example, studies are needed to determine work will also include establishing discriminant validity
how grief symptoms change over time and whether from depression and other grief measures to further
grief and loss can be discriminated from depression ensure that the SCI-QOL Grief and Loss item bank cap-
and anxiety in people with SCI. It will also be important tures the unique losses experienced after injury. The
to determine whether the magnitude or persistence of knowledge gained from the use of this new measure
grief and loss predict other important outcomes such will be essential for the development of grief and loss
as quality of life, functional impairment, and self-care conceptual models to inform treatment approaches to
after controlling for depression, anxiety and other maximize HRQOL after SCI.
salient variables. If, as is the case with other type of
loss,14 there is a significant minority of persons who Disclaimer statements
have prolonged, disabling grief symptoms, tailored Contributors All authors have contributed significantly
grief treatment approaches should be studied.16 to the design, analysis and writing of this manuscript.
The integration of this measure in clinical practice can The contents represent original work and have not
be particularly useful for guiding therapy to target been published elsewhere.
specific areas of difficulty. For example, losses of a
Funding This study was supported by grant
future, loss of a sense of self, or acceptance of injury
#5R01HD054659 from the National Institutes of
all have important implications for tailoring therapy to
Health – Eunice Kennedy Shriver National Institute of
individual needs. Use of the measure in clinical settings
Child’s Health and Human Development/National
can also aid the clinician for tracking change over time
Center on Medical Rehabilitation Research and the
to assess the effectiveness of therapy and guide changes
National Institute on Neurological Disorders and Stroke.
in their approach. By design, interpretation of SCI-QOL
scores is simplified by a standardized metric that allows Conflicts of interest No commercial party having a direct
the clinician to readily identify how the individual com- financial interest in the results of the research supporting
pares to the broader SCI population. Finally, the use of this article has or will confer a benefit upon the authors
this new measure can aid clinicians in the distinguishing or upon any organization with which the authors
grief and loss from depression to guide treatment. are associated. All SCI-QOL items and parameters are

354 The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3

 Kalpakjian et al. Grief and spinal cord injury

© 2015 David Tulsky and Kessler Foundation. All 17 Davis CG, Nolen-Hoeksema S. Loss and meaning: how do people
make sense of loss? Am Behav Sci 2001;44(5):726–41.
rights reserved. All items are freely available to the 18 de Roon-Cassini TA, de St. Aubin E, Valvano A, Hastings J, Horn
public via the Assessment Center platform (http://www. P. Psychological well-being after spinal cord injury: perception of
loss and meaning making. Rehabil Psychol. 2009;54(3):306–14.
assessmentcenter.net). There are currently no plans for 19 Angel S, Kirkevold M, Pedersen BD. Getting on with life following
Dr. Tulsky or Kessler Foundation to benefit financially a spinal cord injury: regaining meaning through six phases. Int J
from the copyright. Qual Stud Health Well-being 2009;4(1):39–50.
20 Niemeier J, Kennedy R, McKinley W, Cifu D. The loss inventory:
preliminary reliability and validity data for a new measure of
Ethics approval The Institutional Review Board at each emotional and cognitive responses to disability. Disabil Rehabil.
site reviewed and approved this project. 2004;26(10):614–23.
21 Quale AJ, Schanke AK. Resilience in the face of coping with a
severe physical injury: a study of trajectories of adjustment in a
References rehabilitation setting. Rehabil Psychol. 2010;55(1):12–22.
1 Ayers T, Balk D, Bolle J, Bonanno GA, Connor SR, Cook AS, 22 Burnett P, Middleton W, Raphael B, Martinek N. Measuring core
et al. Report on bereavement and grief research. Death Stud bereavement phenomena. Psychol Med. 1997;27(1):49–57.
2004;28(6):491–575. 23 Faschingbauer TR, Devaul RA, Zisook S. Development of texas
2 Niemeier JP, Burnett DM. No such thing as ‘uncomplicated inventory of grief. Am J Psychiatry. 1977;134(6):696–7.
bereavement’ for patients in rehabilitation. Disabil Rehabil 2001; 24 Prigerson HG, Maciejewski PK, Reynolds CF, Bierhals AJ,
23(15):645–53. Newsom JT, Fasiczka A, et al. Inventory of complicated grief: a
3 Kubler-Ross E. On death and dying, London, UK: Routelage; scale to measure maladaptive symptoms of loss. Psychiatry Res.
1969. 1995;59(1–2):65–79.
4 Frank R, Elliott T, Corcoran J, Wonderlich S. Depression after 25 Tulsky DS, Kisala PA, Victorson D, Choi SW, Gershon R,
spinal cord injury: is it necessary? Clin Psychol Rev 1987;7(6):611–30. Heinemann AW, et al. Methodology for the development and cali-
5 Wortman CB, Silver RC. The myths of coping with loss. J Consult bration of the SCI-QOL item banks. J Spinal Cord Med. 2015;
Clin Psychol 1989;57(3):349–57. 38(3):270–87.
6 Elliott TR, Frank RG. Depression following spinal cord injury. 26 Tulsky DS, Kisala PA, Victorson D, Tate D, Heinemann AW,
Arch Phys Med Rehabil 1996;77(8):816–23. Amtmann D, et al. Developing a contemporary patient-reported
7 Radnitz CL, Hsu L, Tirch DD, Willard J, Lillian LB, Walczak S, outcomes measure for spinal cord injury. Arch Phys Med
et al. A comparison of posttraumatic stress disorder in veterans Rehabil. 2011;92(10):S44–51.
with and without spinal cord injury. J Abnorm Psychol 1998; 27 Cella D, Nowinski C, Peterman A, Victorson D, Miller D, Lai J-S,
107(4):676–80. et al. The neurology quality-of-life measurement initiative. Arch
8 Buckelew SP, Baumstark KE, Frank RG, Hewett JE. Adjustment Phys Med Rehabil. 2011;92(10, Supplement):S28–36.
following spinal cord injury. Rehabil Psychol 1990;35:101–9. 28 DeWalt DA, Rothrock N, Yount S, Stone AA. Evaluation of item
9 Kennedy P, Lowe R, Grey N, Short E. Traumatic spinal cord candidates: the promis qualitative item review. Med Care. 2007;
injury and psychological impact: a cross sectional analysis of 45(5 Suppl 1):S12–21.
coping strategies. Br J Clin Psychol 1995;34:627–39. 29 Willis GB. Cognitive interviewing: a tool for improving question-
10 Stroebe M, Schut H. The dual process model of coping with naire design. Thousand Oaks, CA: Sage Publications; 2005.
bereavement: rationale and description. Death Stud 1999;23(3): 30 Eremenco SL, Cella D, Arnold BJ. A comprehensive method for
197–224. the translation and cross-cultural validation of health status ques-
11 Bonanno GA, Wortman CB, Lehman DR, Tweed RG, Haring M, tionnaires. Eval Health Prof. 2005;28(2):212–32.
Sonnega J, et al. Resilience to loss and chronic grief: a prospective 31 MetaMetrics. The lexile framework for reading. Durham, NC:
study from preloss to 18-months postloss. J Pers Soc Psychol 2002; MetaMetrics Inc.; 1995.
83(5):1150–64. 32 American Spinal Injury Association. Reference manual for the
12 Bonanno GA, Westphal M, Mancini AD. Resilience to loss and international standards for neurological classification of spinal
potential trauma. Annu Rev Clin Psychol 2011;7:511–35. cord injury. Chicago, IL: American Spinal Injury Association;
13 Bonanno GA, Kennedy P, Galatzer-Levy IR, Lude P, Elfstrom 2003.
ML. Trajectories of resilience, depression, and anxiety following 33 Tulsky DS, Kisala PA, Victorson D, Tate DG, Heinemann AW,
spinal cord injury. Rehabil Psychol 2012;57(3):236–47. Cella DF. Overview of the Spinal Cord Injury – Quality of Life
14 Bonanno GA, Kaltman S. Toward an integrative perspective on (SCI-QOL) measurement system. J Spinal Cord Med. 2015;38(3):
bereavement. Psychol Bull 1999;125(6):760–76. 257–69.
15 Prigerson HG, Horowitz MJ, Jacobs SC, Parkes CM, Aslan M, 34 Choi SW, Gibbons LE, Crane PK. Lordif: an r package for detect-
Goodkin K, et al. Prolonged grief disorder: psychometric vali- ing differential item functioning using iterative hybrid ordinal
dation of criteria proposed for DSM-V and ICD-11. Plos logistic regression/item response theory and monte carlo simu-
Medicine. 2009;6(8):e1000121. lations. J Stat Softw. 2011;39(8):1–30.
16 Shear K, Frank E, Houck PR, Reynolds CF. Treatment of compli- 35 Gershon R, Rothrock NE, Hanrahan RT, Jansky LJ, Harniss M,
cated grief – a randomized controlled trial. JAMA 2005;293(21): Riley W. The development of a clinical outcomes survey research
2601–8. application: assessment center. Qual Life Res. 2010;19(5):677–85.

The Journal of Spinal Cord Medicine 2015 VOL. 38 NO. 3 355