CLINICAL REVIEW CLINICIAN’S CORNER

Risk Prediction Models for Hospital Readmission

A Systematic Review
Devan Kansagara, MD, MCR Context Predicting hospital readmission risk is of great interest to identify which pa-
Honora Englander, MD tients would benefit most from care transition interventions, as well as to risk-adjust
Amanda Salanitro, MD, MS, MSPH readmission rates for the purposes of hospital comparison.

David Kagen, MD Objective To summarize validated readmission risk prediction models, describe their
performance, and assess suitability for clinical or administrative use.
Cecelia Theobald, MD
Data Sources and Study Selection The databases of MEDLINE, CINAHL, and
Michele Freeman, MPH the Cochrane Library were searched from inception through March 2011, the EMBASE
Sunil Kripalani, MD, MSc database was searched through August 2011, and hand searches were performed of
the retrieved reference lists. Dual review was conducted to identify studies published

A
N INCREASING BODY OF LIT- in the English language of prediction models tested with medical patients in both deri-
erature attempts to describe vation and validation cohorts.
and validate hospital read- Data Extraction Data were extracted on the population, setting, sample size, fol-
mission risk prediction tools. low-up interval, readmission rate, model discrimination and calibration, type of data
Interest in such models has grown for used, and timing of data collection.
2 reasons. First, transitional care inter- Data Synthesis Of 7843 citations reviewed, 30 studies of 26 unique models met
ventions may reduce readmissions the inclusion criteria. The most common outcome used was 30-day readmission; only
among chronically ill adults.1-3 Read- 1 model specifically addressed preventable readmissions. Fourteen models that relied
mission risk assessment could be used on retrospective administrative data could be potentially used to risk-adjust readmis-
to help target the delivery of these re- sion rates for hospital comparison; of these, 9 were tested in large US populations and
had poor discriminative ability (c statistic range: 0.55-0.65). Seven models could po-
source-intensive interventions to the pa-
tentially be used to identify high-risk patients for intervention early during a hospital-
tients at greatest risk. Ideally, models ization (c statistic range: 0.56-0.72), and 5 could be used at hospital discharge (c sta-
designed for this purpose would pro- tistic range: 0.68-0.83). Six studies compared different models in the same population
vide clinically relevant stratification of and 2 of these found that functional and social variables improved model discrimina-
readmission risk and give information tion. Although most models incorporated variables for medical comorbidity and use
early enough during the hospitaliza- of prior medical services, few examined variables associated with overall health and
tion to trigger a transitional care inter- function, illness severity, or social determinants of health.
vention, many of which involve dis- Conclusions Most current readmission risk prediction models that were designed
charge planning and begin well before for either comparative or clinical purposes perform poorly. Although in certain set-
hospital discharge. Second, there is in- tings such models may prove useful, efforts to improve their performance are needed
terest in using readmission rates as a as use becomes more widespread.
quality metric. The Centers for Medi- JAMA. 2011;306(15):1688-1698 www.jama.com

care & Medicaid Services (CMS) re-

cently began using readmission rates as Author Affiliations: VA Evidence-Based Synthesis Pro- Medicine, Vanderbilt University, Nashville, Tennes-
gram (Dr Kansagara and Ms Freeman), Department see (Drs Salanitro, Theobald, and Kripalani).
a publicly reported metric and has plans of General Internal Medicine (Drs Kansagara and Corresponding Author: Devan Kansagara, MD, MCR,
to lower reimbursement to hospitals Kagen), Portland Veterans Affairs Medical Center, Port- Portland Veterans Affairs Medical Center, Mailcode
land, Oregon; Department of Internal Medicine, RD71, 3710 SW US Veterans Hospital Rd, Portland,
Oregon Health & Science University, Portland (Drs OR 97239 (kansagar@ohsu.edu).
Kansagara, Englander, and Kagen); Geriatric Research, Clinical Review Section Editor: Mary McGrae
CME available online at Education and Clinical Center, VA Tennessee Valley McDermott, MD, Contributing Editor. We encour-
www.jamaarchivescme.com Healthcare System, Nashville (Dr Salanitro); and Sec- age authors to submit papers for consideration as a
and questions on p 1716. tion of Hospital Medicine, Division of General Inter- Clinical Review. Please contact Mary McGrae
nal Medicine and Public Health, Department of McDermott, MD, at mdm608@northwestern.edu.

1688 JAMA, October 19, 2011—Vol 306, No. 15 ©2011 American Medical Association. All rights reserved.

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

with excess risk-standardized readmis- were presented in separate reports. We stay or discharge diagnostic codes for the
sion rates.4 Valid risk adjustment meth- neither prespecified the method of vali- index hospitalization would be classi-
ods are required for calculation of risk- dation, nor excluded studies in which the fied as using retrospective data. Be-
standardized readmission rates, which derivation and validation cohorts were cause of coding delays, models relying
could be used for hospital compari- drawn from the same population (ie, on administrative codes from index hos-
son, public reporting, and reimburse- split-half validation). We did not limit pital admissions were considered
ment determinations. Models de- studies by diagnosis within medical retrospective.
signed for these purposes should have populations. We excluded studies that The c statistic with 95% confidence
good predictive ability; be deployable focused on psychiatric, surgical, and pe- intervals (when available) were used to
in large populations; use reliable data diatric populations because factors con- describe model discrimination. The c sta-
that can be easily obtained; and use vari- tributing to readmission risk might be tistic, which is equivalent to the area
ables that are clinically related to and considerably different in these patient under the receiver operating character-
validated in the populations in which groups. Finally, we excluded studies istic curve, is defined as the proportion
use is intended.5 from developing nations because these of times the model correctly discrimi-
This systematic review was per- were unlikely to provide directly appli- nates a pair of high- and low-risk indi-
formed to synthesize the available lit- cable results. viduals.8 A c statistic of 0.50 indicates that
erature on validated readmission risk the model performs no better than
prediction models, describe their per- Data Extraction chance; a c statistic of 0.70 to 0.80 indi-
formance, and assess their suitability for and Quality Assessment cates modest or acceptable discrimina-
clinical or administrative use. From each study, we abstracted the fol- tive ability; and a c statistic of greater than
lowing: population characteristics, set- 0.80 indicates good discriminative abil-
METHODS ting, number of patients in the deriva- ity.9,10 If the c statistic was not reported,
Data Sources and Searches tion and validation cohorts, timeframe we abstracted other operational statis-
We searched Ovid MEDLINE, CINAHL, of readmission outcome, readmission tics such as sensitivity, specificity, and
and the Cochrane Library (Central Trial rate, range of readmission rates accord- predictive values for representative risk
Registry, Systematic Reviews, and Ab- ing to predicted risk, and model dis- score cutoffs when available. Model cali-
stracts of Reviews of Effectiveness) from crimination. To facilitate a high-level bration is the degree to which predicted
database inception through March 2011, comparison of predictor variables, we rates are similar to those observed in the
and EMBASE through August 2011, for grouped final model variables into 1 of population. To describe model calibra-
studies published in the English lan- 6 categories (medical comorbidity, tion, we report the range of observed
guage of readmission risk prediction mental health comorbidity, illness se- readmission rates from the predicted low-
models in medical populations. All ci- verity, prior use of medical services, est to highest risk groupings.
tations were imported into an elec- overall health and function, and so- To guide our methodological assess-
tronic database (EndNote X2, Thom- ciodemographic and social determi- ment of included studies, we adapted
son Reuters, New York, NY). The search nants of health).7 elements (including cohort defini-
strategies are provided in detail in eAp- To characterize the practical utility of tion, follow-up, adequacy of prognos-
pendix 1 at http://www.jama.com. each model, 2 of the authors indepen- tic and outcome variable measure-
dently abstracted the type of data used ment, and the validation method) from
Study Selection and the timing of data collection from a prognosis study quality tool and clini-
All of the authors reviewed the cita- each study. Disagreements between re- cal decision rule assessment tool
tions and abstracts identified from elec- viewers about these classifications were (eTable at http://www.jama.com).6,11
tronic literature searches using the eli- resolved through group discussion. Data
gibility criteria shown in eAppendix 2. type consisted of administrative, pri- Data Synthesis
Full-text articles of potentially relevant mary (eg, survey, chart review), or both. The included studies were too heter-
references were retrieved and each was Regarding timing, we classified a model ogenous to permit meta-analysis. There-
independently assessed for eligibility by as using real-time data if the variables fore, we qualitatively synthesized re-
2 of the authors. Eligible articles were would be available on or shortly after in- sults, focusing on model discrimination,
published in English and evaluated the dex hospital admission, and as using ret- the populations in which the model has
ability of statistical models to predict hos- rospective data if the variables would not been tested, practical aspects of model
pital readmission risk. Because a set of be available early during a hospitaliza- implementation, and the types of vari-
predictive factors derived in only 1 popu- tion. For example, a model using prior ables included in each model.
lation may lack validity and applicabil- health care use and data from patient sur-
ity,6 we included only studies of models veys conducted early during a hospital- RESULTS
that were tested in both a derivation and ization would be classified as using real- From 7843 titles and abstracts, 286 ar-
a validation cohort, even if these results time data, while a model using length of ticles were selected for full-text re-
©2011 American Medical Association. All rights reserved. JAMA, October 19, 2011—Vol 306, No. 15 1689

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

view (FIGURE). Of these, 30 studies of ported a c statistic above 0.70, indicat- potentially be used for hospital com-
26 unique models across a broad vari- ing modest discriminative ability. Per- parison purposes (Table 1). Most of
ety of settings and patient populations formance was similar between studies these included variables for medical co-
met our inclusion criteria (TABLE 1, using split-sample validation methods morbidity and use of prior medical ser-
TABLE 2, and TABLE 3). Most studies (n=21; c statistic range: 0.59-0.75), and vices, but a few considered mental
(n=23) were based on US health care those that used external validation meth- health, functional status, and social
data. The remainder were from Aus- ods (n=9; c statistic range: 0.53-0.83). determinant variables (TABLE 4). The
tralia (2 studies), England (n = 2), Ire- Among models that analyzed the rela- 3 models with c statistics of 0.70
land (n = 1), Switzerland (n = 1), or tionship between risk categories and ac- or higher were developed and tested
Canada (n = 1). Fourteen studies in- tual readmission rates, a substantial gra- in large European or Australian co-
cluded only patients aged 65 years or dient in readmission rate was present horts. One examined the risk of 2 or
older. Of these, 7 relied solely on Medi- between patients at the lowest and at the more unplanned readmissions for all
care administrative data. Four studies highest risk level. For example, among hospitalized patients in England, in-
used Veterans Affairs’ data. 6 models using 30-day readmission as an cluding pediatric and obstetric pa-
Total sample size ranged from 173 pa- outcome, the lowest and highest risk tients, for 1 calendar year.13 A Swiss
tients to more than 2.7 million patients. groups differed by 20.4 to 34.5 percent- study17 examined potentially prevent-
The outcome of 30-day readmission was age points in their actual readmission able readmissions. An Australian model
reported most commonly, although some rates. incorporating more than 100 medical
models chose other follow-up intervals comorbidities and administrative so-
ranging from 14 days to 4 years. Among Models Relying on Retrospective cial determinant variables performed at
21 studies reporting c statistics (Table 1, Administrative Data a modest level in asthma patients, but
Table 2, and Table 3), values ranged from Fourteen models were based on retro- poorly in patients with myocardial
0.55 to 0.83, but only 6 studies re- spective administrative data and could infarction.20
The 9 large population-based or mul-
Figure. Literature Flow of Risk Prediction Models for Hospital Readmission ticenter US studies generally had poor
discriminative ability (c statistic range:
12 042 Citations identified from 0.55-0.65). The CMS used a method-
electronic database searches
4222 From MEDLINE ologically rigorous process to create 3
4185 From EMBASE models for congestive heart failure,
2647 From CINAHL
988 From Cochrane Library acute myocardial infarction, and pneu-
monia admissions based on hierarchi-
cal condition categories, which are
4257 Duplicate citations excluded
groups of related comorbidities.14-16 All
7785 Citations screened for title 58 Citations identified from reference lists
3 models showed relatively poor abil-
and abstract review of review articles and authors’ libraries ity to predict 30-day all-cause readmis-
sions (c statistics: 0.61 for congestive
heart failure, 0.63 for acute myocar-
7843 Potentially relevant citations
identified for further review dial infarction, and 0.63 for pneumo-
nia). A recent study evaluating the CMS
7557 Citations excluded based on heart failure model and an older heart
review of title and abstract failure model fared similarly (c statis-
tics: 0.59 and 0.61, respectively).18,23
286 Potentially relevant articles
identified for further review The other 4 US models have limited
generalizability; for example, one model
256 Articles excluded captured readmissions to 1 medical cen-
170 Used for contextual purposes
or for reviewing references
ter only,24 and the other models were
34 Did not develop or test developed more than 2 decades
a prediction model
30 Prediction model not validated ago.12,22,25
18 Study population not in scope
4 Non-English language Models Using Real-Time
Administrative Data
30 Articles of 26 unique models
included in systematic review Three administrative data−based mod-
els were designed to identify high-risk
eAppendix 2 at http://www.jama.com lists the inclusion and exclusion criteria for title and abstract review. patients in real-time to potentially fa-
Specific exclusion codes were not recorded at the abstract level.
cilitate targeted interventions (Table 2).
1690 JAMA, October 19, 2011—Vol 306, No. 15 ©2011 American Medical Association. All rights reserved.

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

Table 1. Characteristics of Models Using Retrospective Administrative Data

No. of Patients
by Cohort Readmission

Actual Range of Model

Source Population and Setting DC VC a Outcome b Rate, % Rates (VC) Discrimination c
Anderson and Medicare patients in general US 21 043 10 522 60 d NR in 4-40 d (lowest to NR
Steinberg,12 population (excluded those with both highest decile)
1985 ESRD) from 1974-1977 cohorts
Bottle et al,13 Inpatients from general population ~ 1.4 ~ 1.4 12 mo 9.80 NR All patients: 0.72 (0.70
2006 in England from 2000-2001 million e million e overall when 12-mo
deaths excluded);
sensitive
conditions f: 0.75
CMS model
Krumholz Medicare patients aged ⱖ65 y with 100 465 100 285 30 d DC: 18.9 8.0-33.0 (lowest to 0.63
et al,14 AMI in general US population VC: 19.2 highest decile)
2008 from 2005-2006
Krumholz Medicare patients aged ⱖ65 y with 283 919 283 528 30 d DC: 23.6 15.0-37.0 (lowest to 0.60
et al, CHF in general US population VC: 23.7 highest decile
200815 from 2003-2004
Krumholz Medicare patients aged ⱖ65 y with 226 545 226 706 30 d DC: 17.4 9.0-31.0 (lowest to 0.63
et al,16 pneumonia in general US VC: 17.5 highest decile)
2008 population from 2005-2006
Halfon et al,17 All hospitalizations in general 65 740 66 069 30 d DC: 5.1 NR Nonclinical: 0.67;
2006 population in Switzerland in (potentially VC: 5.2 Charleson-based:
2000 avoidable) 0.69; SQLape:
0.72
Hammill et al,18 Patients aged ⱖ65 y from CHF 24 163 g NA 30 d 21.9 Claims-only model: Claims-only model:
2011 registry in general US overall 14.4-32.7 (lowest 0.59; clinical
population from 2004-2006 to highest decile); claims model: 0.60
clinical claims
model: 13.5-33.9
Holloway et US medical, neurological, surgical, 2970 Unclear 30 d 22.0 NR NR
al,19 1990 and geriatric inpatients at single overall
VA hospital from 1981-1982
Holman et al,20 Medical, surgical, and psychiatric 326 456 5289 30 d NR NR Asthma: 0.71;
2005 inpatients from Western (asthma) AMI: 0.64
Australia’s general population 5265 (AMI)
from 1989-1997
Howell et al,21 General medical inpatients with 13 207 4492 12 mo DC: 45.5 Risk scores (positive 0.65
2009 ambulatory care sensitive VC: 45.1 LR): 50 (2.04),
condition f in Queensland, 70 (3.11), 80
Australia’s general population (7.02); (overall
from 2005-2006 range: 0-100)
Naessens Inpatients aged ⱖ65 y from general 5854 10% of DC 60 d (and 20.8 15.6-36.0 (lowest to HCFA model: 0.59
et al,22 US population and living in a mortality) overall highest quartile) HCFA model plus
1992 single county in 1980, 1985, COMPLEX
and 1987 measure: 0.61 (SE,
0.01)
Philbin and Inpatients with CHF treated at 21 227 21 504 Within 21.3 9.8-45.4 (lowest to Simple scoring
DiSalvo,23 multiple centers in a single US calendar overall highest ninth) system: 0.60;
1999 state in 1995 year for weighted scoring
CHF system: 0.61
Silverstein Inpatients aged ⱖ65 y treated at 19 528 9764 30 d 11.7 NR 0.65 (same for both
et al,24 multiple centers in a single US overall Elixhauser and
2008 city from 2002-2004 HRDES methods)
Thomas,25 Medicare inpatients aged ⱖ65 y Range: NA 15, 30, 60, 3-40 NR Range among 8
1996 treated at multiple centers in a 1163- and 90 d overall h conditions and 4
single US state from 1989-1991 14 590 h periods: 0.55-0.61
Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; CMS, Centers for Medicare & Medicaid Services; COMPLEX, a measurement of comorbidity and disease
severity22; DC, derivation cohort; ESRD, end-stage renal disease; HCFA, Health Care Financing Administration; HRDES, High-risk Diagnoses for the Elderly Scale; LR, likelihood ratio;
NA, not applicable; NR, not reported; SE, standard error; SQLape, Striving for Quality Level and Analyzing of Patient Expenditures; VA, Veterans Affairs; VC, validation cohort.
a The most recent cohort is listed if a study had multiple VCs.
b Unplanned, all-cause readmissions unless otherwise indicated.
c Values are from the c statistic unless otherwise indicated.
d Approximate values of data that were presented in a bar graph.
e The total number of patients was divided equally between the DC and the VC, but the exact numbers of patients were not specified.
f Includes patients with ambulatory care reference conditions such as CHF, chronic obstructive pulmonary disease, diabetes, and asthma, for which timely and effective case manage-
ment has the potential to reduce the risk of readmission.
g The bootstrap method was used for internal validation. There was not a separate VC.
h Study had 12 different cohorts based on diagnosis and reported 15-, 30-, 60-, and 90-day readmission rates for 12 conditions.

©2011 American Medical Association. All rights reserved. JAMA, October 19, 2011—Vol 306, No. 15 1691

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

A model with modest discriminative system with a large socioeconomically cluding numerous social factors such
ability (c statistic: 0.72; 95% CI, 0.70- disadvantaged population.26 It incor- as number of address changes, census
0.75) examined 30-day heart failure re- porated variables from an automated tract socioeconomic status, history of
admissions in a single urban US health electronic medical record system, in- cocaine use, and marital status. The

Table 2. Characteristics of Models Using Real-Time Administrative Data and Retrospective Primary Data Collection
No. of Patients
by Cohort Readmission

Actual Range of Model

Source Population and Setting DC VC a Outcome b
Rate, % Rates (VC) Discrimination c
Models Using Administrative Data in Real Time
Amarasingham Patients with CHF treated at a 1029 343 30 d 24.1 12.2-45.7 (lowest to 0.72 (95% CI,
et al,26 single US center from overall highest quintile) 0.70-0.75)
2010 2007-2008
Billings and Patients eligible for mandatory ~ 35 000 d ~ 35 000 d 12 mo NR in NR (overall range: Risk score range:
Mijanovich,27 Medicaid managed care both 0-100) e 0-100; using risk
2007 enrollment in general US cohorts scores ⬎50:
population in a single city sensitivity: 58%;
from 2000-2004 specificity: 74%;
PPV: 69.5%;
positive LR: 2.23
Billings et al,28 Inpatients with an ambulatory 10% sample of Another 12 mo NR in NR 0.69
2006 f care sensitive reference hospital 10% both
condition f in general episodes sample cohorts
population of England from of
2002-2003 hospital
episodes
Models Using Retrospective Primary Data Collection
Coleman Medicare inpatients aged ⱖ65 y 700 704 30 d g DC: 21.9 NR Administrative data
et al,29 in general US population VC: 25.0 model: 0.77;
2004 from 1997-1998 administrative data
model plus
self-report data
model: 0.83
Krumholz Medicare patients aged ⱖ65 y 1129 1047 180 d DC: 50.0 All-cause: 26.0-59.0; No. of risk factors
et al,30 with CHF and treated at VC: 47.0 CHF: 9.0-31.0 associated with
2000 multiple centers in a single (lowest to highest readmission risk
US state from 1994-1995 tertile) (P⬍.001); 0 risk
factors: 26%; 3-4
risk factors: 59%
Morrissey Medical inpatients aged ⱖ65 y 487 732 12 mo DC: 40.7 NR 0.70
et al,31 treated at a single rural VC: 29.0
2003 hospital in Ireland from
1997-1998
Smith et al,32 Medical inpatients treated at a 1007 499 90 d DC: 16.9 7.3-38.0 (lowest to Sensitivity: 59.0%;
1985 single US county hospital VC: NA highest octile) specificity: 69.3%;
from 1979-1980 PPV: 29.9%;
positive LR: 1.92
Smith et al, Medical inpatients treated at a 502 (control); 0 By month/patient; DC: NA 0.07-0.18 (lowest to NR
198833 single US county hospital in 499 mean: 180 d VC: 10.0 highest tertile)
1985 (interven- of follow-up
tion)
Smith et al,34 US medical inpatients aged 0 662 90 d DC: NA NR 0.66
1996 ⱖ45 y treated at a single VA VC: 20.1
hospital from 1988-1990
van Walraven et Medical and surgical inpatients 4812 (split DC 1 million 30 d DC: 7.3 0-42.9 h 0.68 (95% CI,
al,35 2010 treated at multiple centers in and from VC: 7.3 0.65-0.71)
Canada internal VC) external
VC
Abbreviations: CHF, congestive heart failure; DC, derivation cohort; LR, likelihood ratio; NA, not applicable; NR, not reported; PPV, positive predictive value; VA, Veterans Affairs; VC,
validation cohort.
a The most recent cohort is listed if a study had multiple VCs.
b Unplanned, all-cause readmissions unless otherwise indicated.
c Values are from the c statistic unless otherwise indicated.
d The total number of patients was divided equally between the DC and the VC, but the exact numbers of patients were not specified.
e Inpatient costs ranged from $23 687 to $44 385 for risk scores of 50 to 90.
f The patients at risk for rehospitalization algorithm was used for this study. Ambulatory care sensitive reference conditions include CHF, chronic obstructive pulmonary disease, diabetes,
and asthma, for which timely and effective case management has the potential to reduce the risk of readmission.
g Includes patients who were transferred at least once from a lower- to a higher-intensity care environment (ie, complicated care transitions).
h Scores that ranged from 0 to 17 correspond to an expected probability range of 2.0% to 34.6% for readmission or death.

1692 JAMA, October 19, 2011—Vol 306, No. 15 ©2011 American Medical Association. All rights reserved.

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

only study that focused specifically on and prior use of medical services (c 0.65-0.71). 35 Increasing scores on
Medicaid enrollees used a risk score statistic: 0.77) along with functional another 4-item model of medical
range of 0 to 100 for 12-month read- status data (c statistic: 0.83) from the comorbidities, prior use of medical
missions and found that patient cost Medicare Beneficiaries Survey to pre- services, and levels of creatinine at
profiles varied widely with risk score.27 dict a composite outcome of hospital discharge were associated with
Finally, a British model used data on use readmissions and nursing home trans- increasing readmission rates in
of prior medical services and comor- fers.29 The survey was not routinely patients with heart failure.30
bidity, and also controlled for ob- administered during index hospital- Four models incorporated primary
served and expected hospital readmis- ization and it is unclear to what data collected in real time (Table 3).
sion rates, but predictive ability extent the use of retrospective survey Only 2 of these models have been tested
remained modest (c statistic: 0.69).28 data affects the predictive ability of in contemporary populations; the oth-
the model. Similarly, a medical record ers were conducted more than 2 de-
Models Incorporating study in Ireland retrospectively cades ago. One survey-based model de-
Primary Data Collection applied a 9-item questionnaire, veloped at 6 academic hospitals
Nine models incorporated survey or including items such as discharge included social determinant, comor-
chart review data and could poten- polypharmacy, and performed mod- bidity, prior use of medical services, and
tially be used for clinical intervention estly well (c statistic: 0.70).31 A simple self-rated health variables, but had poor
purposes, although 5 used data Canadian model used medical comor- predictive ability (c statistic: 0.61).38 The
unlikely to be available early during a bidities up through index hospital dis- Probability of Repeated Admission is a
hospitalization (Table 2). The best charge along with index hospital simple 8-item survey tool developed in
performing of these models used length of stay and prior use of medical older Medicare beneficiaries; how-
administrative data on comorbidity services (c statistic: 0.68; 95% CI, ever, it also had poor predictive ability

Table 3. Characteristics of Models Using Primary Data Collected in Real Time

No. of Patients
by Cohort Readmission

Actual Range of Model

Source Population and Setting DC VC a Outcome b Rate, % Rates (VC) Discrimination c
Burns and US medical inpatients aged 134 34 60 d 30.6 NR NR
Nichols,36 ⱖ65 y treated at a single VA overall
1991 hospital in 1987
Evans et al,37 US medical, neurological, and 532 177 Composite 21.0 Patients with high Risk score range: 0-8
1988 surgical inpatients treated of overall use of care: using risk scores
over a 6-wk period at a 60 d d 34.7%-91.7% ⱖ3: sensitivity:
single VA hospital (lowest to highest 0.60; specificity:
eighth) 0.76; positive LR:
2.5; using risk
scores ⱖ4:
sensitivity: 0.42;
specificity: 0.93;
positive LR: 6
Hasan et al,38 Medical inpatients treated at 7287 3659 30 d DC: 17.5 5.9-28.9 (lowest to 0.61
2010 multiple US centers from VC: 17.4 highest quartile)
2001-2003
PRA
Boult US noninstitutionalized 2942 2934 4y DC: 28.4 26.1 (score range: 0.61 (SE, 0.01)
et al,39 Medicare patients aged VC: NA 0-3) to 41.8
1993 ⱖ70 y in 1984 (score range: ⬎4)
Allaudeen Medical inpatients aged ⱖ65 y NA 159 30 d DC: NA NR PRA: 0.56 (95% CI,
et al,40 treated at a single US VC: 32.7 0.44-0.67) e
2011 academic center during
5-wk period in 2008
Novotny Medical inpatients treated at a 1077 NR 41 d DC: NA NR PRA score: 0.53;
and single US academic center VC: 14.0 positive LR: 1.67
Anderson,41 from 2005-2007
2008
Abbreviations: DC, derivation cohort; LR, likelihood ratio; NA, not applicable; NR, not reported; PRA, Probability of Repeated Admission; SE, standard error; VA, Veterans Affairs; VC,
validation cohort.
a The most recent validation cohort is listed if a study had multiple VCs.
b Unplanned, all-cause readmissions unless otherwise indicated.
c Values are from the c statistic unless otherwise indicated.
d Includes readmission, nursing home placement, or length of stay longer than expected per mean length of stay of diagnosis-related group.
e The prediction range by a physician is 0.58 to 0.59 (SE range, 0.46-0.70) and by a nonphysician is 0.50 to 0.55 (SE range, 0.38-0.67).

©2011 American Medical Association. All rights reserved. JAMA, October 19, 2011—Vol 306, No. 15 1693

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

across several studies (c statistic range: prediction (Table 4). Nearly all stud- in the final model. Table 4 also high-
0.56-0.61; 95% CI, 0.44-0.67).39-41 ies included medical comorbidity data lights important gaps in model devel-
and many included variables for prior opment in that few studies considered
Use of Variables use of medical services, usually prior variables associated with illness sever-
A comparison of the types of variables hospitalizations. Basic sociodemo- ity, overall health and function, and so-
considered for and included in the fi- graphic variables such as age and sex cial determinants of health.
nal models can provide some informa- were considered by most studies but, Six studies compared the perfor-
tion about the contribution of differ- in many instances, these variables did mance of different models within the
ent types of variables to readmission risk not contribute enough to be included same population and offer further in-
sights about the incremental value of
Table 4. Variables Considered by Studies in Evaluating the Risk of Hospital Readmission different types of variables (TABLE 5).
No. of Studies Amarasingham et al26 found a model
based on automated electronic medi-
Included in Evaluated, but Not
Final Model Not Included Considered a
cal records that incorporated sociode-
Specific medical diagnoses 2413-25,27-31,34-39 0 312,26,32
mographic factors such as drug use and
or comorbidity index housing discontinuities was more pre-
Mental health comorbidities dictive than comorbidity-based mod-
Mental illness 915-18,20,21,26,27,37 414,24,28,36 1112,19,22,23,30-32,34,35,38,39 els. Coleman et al29 found that the in-
Alcohol or substance use 1115-21,23,26-28 514,24,31,34,37 812,22,30,32,35,36,38,39 clusion of variables such as functional
Illness severity
Severity index 126 136 1913-19,21,22,24,28,30-32,34,35,37-39
status from survey data improved model
Laboratory findings 418,30,32,34 131 1513-17,19,21,22,24,28,35-39
performance slightly compared with the
Other b 42,3,24 418,30,34,37 1114-16,21,24,28,31,32,35,38,39 use of medical services and comorbid-
Prior use of medical services ity-based administrative data alone (c
Hospitalizations 1412,13,17,21,26-31,36-39 135 1014-16,18,19,22-24,32,34 statistics: 0.83 vs 0.77, respectively). A
Emergency department 427,32,34,35 126 1712,14-16,18,19,21-24,28,30,31,36-39 large Swiss study of potentially pre-
visits
ventable readmission risk compared a
Clinic visits or missed 326,27,39 0 1912,14-16,18,19,21-24,28,30-32,34-38
clinic visits
simple nonclinical model, a Charlson
Index hospital length 423,25,35,38 319,30,36 1512,14-16,18,21,22,24,26,28,31,32,34,37,39 comorbidity–based model, and a more
of stay complex hierarchical diagnosis and pro-
Overall health and function cedures-based model called SQLape
Functional status, 229,34 630,35-39 1412,14-16,18,19,21-24,26,28,31,32 (Striving for Quality Level and Analyz-
ADL dependence,
and mobility ing of Patient Expenditures), and found
Self-rated health, quality 329,38,39 231,34 1712,14-16,18,19,21-24,26,28,30,32,35-37 small differences among them (c sta-
of life tistics: 0.67, 0.69, and 0.72,
Cognitive impairment 714-16,18,31,34,37 521,24,36,38,39 912,19,22,23,26,28,30,32,35 respectively).17
Visual or hearing 129 139 2112,14-16,18,19,21-24,26,28,30-32,34-39 Other comparative studies found
impairment
Sociodemographic factors
little difference among models. Clini-
Age 1912-22,24,25,27-29,34,37,39 723,26,30,32,35,36,38 131 cal data such as laboratory and physi-
Sex 1512-18,20,22,24-28,39 819,21,23,30,32,35,36,38 131 ological variables from medical rec-
Race/ethnicity 712,13,20,23,24,27,28 821,26,30,32,34,36,38,39 814-16,18,19,22,31,35 ords or registries did not enhance
Social determinants of health performance of claims-only CMS mod-
SES, income, and 513,20,21,26,27 724,28,34,36-39 1012,14-16,18,19,22,23,31,35 els.14-16,31 A US study of older patients
employment status
Insurance status c 619,23,24,26,29,38 134 530,32,36,37,39
found that an intricate International
Education 0 431,36,38,39 1712,14-16,18,19,21-24,26,28,30,32,34,35,37
Classification of Diseases, Ninth Revi-
Marital status and No. of 426,31,37,38 619,21,34-36,39 1112,14-16,18,22-24,28,30,32
sion code-based disease complexity sys-
people in home tem added little discriminative ability
Caregiver availability, 234,39 138 1912,14-16,18,19,21-24,26,28,30-32,34-37 to a poorly performing Health Care Fi-
other social support nancing Administration model.22 Fi-
Access to care or limited 512,19,21,23,38 224,35 1414-16,18,22,26,28,30-32,34,36,37,39 nally, Allaudeen et al40 found internal
access (eg, rural area)
Discharge location 223,24 119 1812,14-16,18,21,22,26,28,30-32,34-39
medicine interns using a gestalt ap-
(home, nursing home) proach predicted readmissions with a
Abbreviations: ADL, activities of daily living; SES, socioeconomic status.
a Six studies did not report candidate variables and only reported the final model.13,17,20,25,27,29
similarly poor level of ability as an older,
b Examples include use of telemetry, shock, planned vs emergent index hospitalization, heart rate, and left ventricular ejec- established survey-based model (ie,
tion fraction.
c This category is not relevant to studies of Medicare patients14-16,18,22 and non-US studies.13,17,21,31,35 Probability of Repeated Admission) in
a small, single-center cohort.
1694 JAMA, October 19, 2011—Vol 306, No. 15 ©2011 American Medical Association. All rights reserved.

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

Potentially Preventable medical patients tested in a variety of comorbidities, basic demographic data,
Readmissions settings and populations. Several are and clinical variables are much better
Only 1 model attempted to explicitly being applied currently in clinical, re- able to predict mortality than readmis-
define and identify potentially prevent- search, and policy arenas. Half of the sion risk.18,26,35 Broader social, environ-
able readmissions.46 Investigators con- models were largely designed to facili- mental, and medical factors such as ac-
ducted a systematic medical record re- tate calculation of risk-standardized re- cess to care, social support, substance
view to define potentially preventable admission rates for hospital compari- abuse, and functional status contrib-
readmissions and develop an adminis- son purposes. The other half were ute to readmission risk in some mod-
trative data–based algorithm. A subse- clinical models that could be used to els, but the utility of such factors has
quent Swiss study compared the per- identify high-risk patients for whom a not been widely studied.
formance of 3 models in predicting transitional care intervention might be It is likely that hospital and health
readmissions according to their algo- appropriate. Most models in both cat- system–level factors, which are not
rithm.17 egories have poor predictive ability. present in current readmission risk
Readmission risk prediction re- models, contribute to risk.47 For in-
COMMENT mains a poorly understood and com- stance, the timeliness of postdischarge
In this systematic review, we found 26 plex endeavor. Indeed, models of pa- follow-up, coordination of care with the
readmission risk prediction models of tient-level factors such as medical primary care physician, and quality of

Table 5. Studies That Compared Models Within a Population

Model Description C Statistic a
Halfon et al,17 2006
Nonclinical model Age, sex, prior medical services use 0.67
Modified Charlson score−based model Charlson score42 plus prior medical services use 0.69
Modified SQLape model43 Complex administrative model combining comorbidity, age, and medical 0.72
services use data into 49 risk categories
Hammill et al,18 2011
Claims-only model CMS administrative heart failure model15 0.59
Clinical claims model CMS administrative heart failure model plus levels of serum creatinine, serum 0.60
sodium, and hemoglobin, and systolic blood pressure
Allaudeen et al,40 2011
Probability of repeated admission32 b Age, sex, self-rated health, availability of informal caregiver, coronary disease, 0.56 (0.44-0.67)
diabetes, hospital admission within past year, prior medical services use
Prediction by physician Interns, residents, and attending physicians predicted risk of readmission 0.58-0.59 (0.46-0.70)
based on overall evaluation of patient
Prediction by nonphysician Nurses and case managers predicted risk of readmission based on overall 0.50-0.55 (0.38-0.67)
evaluation of patient
Amarasingham et al,26 2010
ADHERE mortality model Levels of blood urea nitrogen and creatinine, and systolic blood pressure 0.56 (0.54-0.59)
Tabak mortality model44 Age, 17 laboratory and vital sign variables within 24 h of hospital presentation 0.61 (0.59-0.64)
CMS heart failure model15 Complex administrative comorbidity model consisting of age, sex, and 35 0.66 (0.63-0.68)
hierarchical condition categories
Electronic readmission model Includes Tabak mortality score, history of depression or anxiety, single status, 0.72 (0.70-0.75)
sex, residential stability, Medicare status, residential census tract in
lowest socioeconomic quintile, history of confirmed cocaine use, history
of missed clinic visit, use of a health system pharmacy, number of prior
admissions, presented to emergency department between 6 AM and 6
PM for index admission
Coleman et al,29 2004
Administrative model Age, sex, prior medical services use, Medicaid status, Charlson score,42 heart 0.77
disease, cancer, or diabetes
Administrative model plus self-report model Self-rated health, activities of daily living assistance need, visual impairment, 0.83
functional status
Naessens et al,22 1992
Modified HCFA mortality model45 Age, sex, disease diagnosis from 1 of 16 diagnosis-related groups, 0.59 (0.01)
and 8 comorbidities
HCFA model plus COMPLEX measure Complicated administrative model incorporating diagnosis-related 0.61 (0.01)
group−based disease staging and number of body systems affected
plus HCFA model
Abbreviations: ADHERE, Acute Decompensated Heart Failure registry; CMS, Centers for Medicare & Medicaid Services; COMPLEX, a measurement of comorbidity and disease sever-
ity22; HCFA, Health Care Financing Administration; SQLape, Striving for Quality Level and Analyzing of Patient Expenditures.
a If reported, values in parentheses are expressed as 95% CI or standard error.
b Variables were obtained from chart abstraction, whereas original probability of repeated admission instrument is based on patient surveys.

©2011 American Medical Association. All rights reserved. JAMA, October 19, 2011—Vol 306, No. 15 1695

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

medication reconciliation may be asso- able.17 A recent systematic review of 34 transition interventions. Even limited
ciated with readmission risk.48,49 The studies found wide variation in the per- ability to identify a proportion of pa-
supply of hospital beds may indepen- centage of readmissions considered pre- tients at risk for future high-cost medi-
dently contribute to higher readmis- ventable and estimates ranged from 5% cal services use can increase the cost-
sion rates.50 Finally, the quality of inpa- to 79% (median, 27%).58 More work is effectiveness of such programs.28,59
tient care could also contribute to risk,51 needed to develop readmission risk pre- Of note, few models incorporated
although the evidence is mixed.52 Al- diction models with an outcome of pre- clinically actionable data that could be
though the inclusion of such hospital- ventable readmissions. This could not used to triage patients to different types
level factors would conceivably im- only improve risk-standardization ef- of interventions. For example, margin-
prove the predictive ability of models, forts, but also allow hospitals to better ally housed patients or those strug-
it would be inappropriate to include focus limited clinical resources in re- gling with substance abuse might re-
them in models that are used for risk- admission avoidance programs. quire unique discharge services.
standardization purposes. Doing so As with models that are used for risk- Relatively simple, practical models that
would adjust hospital readmission rates standardization, readmission risk mod- use real-time clinically actionable data,
for the very deficits in quality and effi- els that are intended for clinical use also such as the Project BOOST model, have
ciency that hospital comparison efforts have certain requirements and limita- been created, but their performance has
seek to reveal, and which could be tar- tions. Clinical models would ideally not yet been rigorously validated.60
gets for quality improvement interven- provide data prior to discharge, dis- Our review concurs with and adds
tions. criminate high- from low-risk pa- to the findings of several other reviews
Public reporting and financial penal- tients, and would be adapted to the set- that found deficiencies in risk predic-
ties for hospitals with high 30-day read- tings and populations in which they are tion models. One recent review lim-
mission rates are spurring organiza- to be used. Few models met all these ited to US studies examined general risk
tions to innovate and implement quality criteria, and only 1 of these (a single- factors for preventable readmissions, but
improvement programs.53,54 Neverthe- center study) had acceptable discrimi- did not search explicitly for validated
less, the poor discriminative ability of native ability. 26 As with the risk- models, and many of the included stud-
most of the administrative models we ex- adjustment models, most of the models ies had poor study designs.61 The study’s
amined raises concerns about the abil- developed for clinical purposes had authors suggested that measures of poor
ity to standardize risk across hospitals to poor predictive ability, although no- health such as comorbidity burden,
fairly compare hospital performance. Un- table exceptions suggest the addition of prior medical services use, and increas-
til risk prediction and risk adjustment be- social or functional variables may im- ing age were associated with readmis-
come more accurate, it seems inappro- prove overall performance.26,29 sions. Three other reviews focused on
priate to compare hospitals in this way The best choice of model may de- specific diagnoses and found few read-
and reimburse (or penalize) them on the pend on setting and the population mission risk models for heart failure,55
basis of risk-standardized readmission being studied. The success of some chronic obstructive pulmonary dis-
rates. Others have reached similar con- models in certain populations and the ease,62 and myocardial infarction.63
clusions,55 and also have expressed con- lack of success of others suggest that Our review has certain limitations.
cern that such financial penalties could the patient-level factors associated with We included studies outside of the
exacerbate health disparities by penal- readmission risk may differ according United States, given that portions of US
izing hospitals with fewer resources.56 to the population studied. For ex- health care may resemble other coun-
Still others have argued that readmis- ample, while medical comorbidities tries’ health systems, but applicability
sion rate is an incomplete accountabil- may account for a large proportion of of models from other countries to the
ity measure that fails to consider “the real risk in some populations, social deter- United States may still be limited. Our
outcomes of interest—health, quality of minants may disproportionately influ- classifications of data types, data col-
life, and value.”57 ence risk in socioeconomically disad- lection timing, and the intended use of
Use of readmission rates as a qual- vantaged populations. Our review each model are subject to interpreta-
ity metric assumes that readmissions are found that few models have incorpo- tion, but we attempted to mitigate sub-
related to poor quality care and are po- rated such variables. jectivity by using a dual-review and con-
tentially preventable. However, the pre- Even though the overall predictive sensus process. Finally, few studies
ventability of readmissions remains un- ability of the clinical models was poor, directly compared models within the
clear and understudied. We found only we did find that high- and low-risk same population, and summary statis-
1 validated prediction model that ex- scores were associated with a clini- tics such as the c statistic should not be
plicitly examined potentially prevent- cally meaningful gradient of readmis- used to directly compare models across
able readmissions as an outcome, and sion rates. This is important given re- different populations.
it found that only about one-quarter of source constraints and the need to Additional research is needed to
readmissions were clearly prevent- selectively apply potentially costly care assess the true preventability of read-
1696 JAMA, October 19, 2011—Vol 306, No. 15 ©2011 American Medical Association. All rights reserved.

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

missions in US health systems. Given Conflict of Interest Disclosures: All authors have com- how to use articles about clinical decision rules. JAMA.
pleted and submitted the ICMJE Form for Disclosure 2000;284(1):79-84.
the broad variety of factors that may of Potential Conflicts of Interest and none were re- 7. National Center for HIV/AIDS, Viral Hepatitis, STD,
contribute to preventable readmission ported. and TB Prevention; US Centers for Disease Control and
Funding/Support: This report is based on research con- Prevention. Establishing a holistic framework to re-
risk, models that include factors ducted by the Evidence-Based Synthesis Program Cen- duce inequities in HIV, viral hepatitis, STDs, and tu-
obtained through medical record ter located at the Portland VA Medical Center, and berculosis in the United States, October 2010. http:
funded by the Department of Veterans Affairs and the //www.cdc.gov/socialdeterminants/docs
review or patient report may be valu- Veterans Health Administration, Office of Research and /SDH-White-Paper-2010.pdf. Accessed August 10,
able. Innovations to collect broader Development, Health Services Research and Devel- 2011.
variable types for inclusion in admin- opment. The research also was funded in part by Van- 8. Iezzoni LI, ed. Risk Adjustment for Measuring
derbilt CTSA grant 1 UL1 RR024976 from the Na- Health Care Outcomes. 3rd ed. Chicago, IL: Health
istrative data sets should be consid- tional Center for Research Resources, National Institutes Administration Press; 2003.
ered. Future studies should assess the of Health. 9. Schneeweiss S, Seeger JD, Maclure M, Wang PS,
Role of the Sponsor: The funding organizations had Avorn J, Glynn RJ. Performance of comorbidity scores
relative contributions of different no role in the design and conduct of the study; col- to control for confounding in epidemiologic studies
types of patient data (eg, psychosocial lection, management, analysis, and interpretation of using claims data. Am J Epidemiol. 2001;154(9):
the data; and preparation, review, or approval of the 854-864.
factors) to readmission risk prediction manuscript. 10. Ohman EM, Granger CB, Harrington RA, Lee KL.
by comparing the performance of Disclaimer: The findings and conclusions in this Risk stratification and therapeutic decision making in
models with and without these vari- report are those of the authors who are responsible acute coronary syndromes. JAMA. 2000;284(7):
for its contents; the findings and conclusions do not 876-878.
ables in a given population. These necessarily represent the views of the Department of 11. Hayden JA, Côté P, Bombardier C. Evaluation of
models should ideally be based on Veterans Affairs or the US government. Therefore, the quality of prognosis studies in systematic reviews.
no statement in this article should be construed as an Ann Intern Med. 2006;144(6):427-437.
population-specific conceptual frame- official position of the Department of Veterans 12. Anderson GF, Steinberg EP. Predicting hospital re-
works of risk. Implementation of risk Affairs. admissions in the Medicare population. Inquiry. 1985;
Online-Only Material: eAppendix 1, eAppendix 2, and
stratification models and their effect the eTable are available at http://www.jama.com.
22(3):251-258.
13. Bottle A, Aylin P, Majeed A. Identifying patients
on work flow and resource prioritiza- Additional Contributions: We thank Rose Relevo, MLS, at high risk of emergency hospital admissions: a lo-
tion should be assessed in a broad MS, AHIP, research librarian (Oregon Health & Sci- gistic regression analysis. J R Soc Med. 2006;99
ence University), for constructing and deploying the (8):406-414.
variety of hospital settings. Also, given search strategy, as well as Tomiye Akagi, BA, admin- 14. Krumholz HM, Normand S-LT, Keenan PS, et al.
that many models have limited predic- istrative assistant (Portland VA Medical Center). We Hospital 30-day acute myocardial infarction readmis-
also thank Ed Vasilevskis, MD, Frank Harrell, PhD, Art sion measure: methodology, June 9, 2008. http://www
tive ability and may require some Wheeler, MD, and Italo Biaggioni, MD (all 4 with Van- .qualitynet.org/dcs/ContentServer?c=Page&pagename
investment of time and cost to imple- derbilt University) for critically reviewing a draft of the
=QnetPublic%2FPage%2FQnetTier3&cid
manuscript. Dr Wheeler was compensated by the Van-
ment, future studies should further derbilt CTSA grant. Drs Vasilevskis, Harrell, and Biag-
=1219069855841. Accessibility verified September 22,
2011.
evaluate the relative value of clinician gioni did not receive compensation for their contri-
15. Krumholz H, Normand S-L, Keenan P, et al. Hos-
butions.
gestalt compared with predictive mod- pital 30-day heart failure readmission measure: meth-
els in assessing readmission risk. odology, April 23, 2008. http://www.qualitynet.org
/dcs/ContentServer?c=Page&pagename=QnetPublic
In summary, readmission risk pre- REFERENCES %2FPage%2FQnetTier3&cid=1219069855841.
diction is a complex endeavor with 1. Jack BW, Chetty VK, Anthony D, et al. A reengi- Accessibility verified September 22, 2011.
neered hospital discharge program to decrease rehos- 16. Krumholz HM, Normand S-LT, Keenan PS, et al.
many inherent limitations. Most mod- pitalization: a randomized trial. Ann Intern Med. 2009; Hospital 30-day pneumonia readmission measure:
els created to date, whether for hospi- 150(3):178-187. methodology, June 9, 2008. http://www.qualitynet
2. Coleman EA, Parry C, Chalmers S, Min SJ. The care .org/dcs/ContentServer?c=Page&pagename
tal comparison or clinical purposes, =QnetPublic%2FPage%2FQnetTier3&cid
transitions intervention: results of a randomized con-
have poor predictive ability. Although trolled trial. Arch Intern Med. 2006;166(17):1822- =1219069855841. Accessibility verified September 22,
in certain settings such models may 1828. 2011.
3. Naylor MD, Brooten D, Campbell R, et al. Com- 17. Halfon P, Eggli Y, Prêtre-Rohrbach I, Meylan D,
prove useful, better approaches are prehensive discharge planning and home follow-up Marazzi A, Burnand B. Validation of the potentially
needed to assess hospital performance of hospitalized elders: a randomized clinical trial. JAMA. avoidable hospital readmission rate as a routine indi-
1999;281(7):613-620. cator of the quality of hospital care. Med Care. 2006;
in discharging patients, as well as to 4. QualityNet. Readmission measures overview: pub- 44(11):972-981.
identify patients at greater risk of avoid- licly reporting risk-standardized, 30-day readmission 18. Hammill BG, Curtis LH, Fonarow GC, et al. In-
measures for AMI, HF and PN. http://www.qualitynet cremental value of clinical data beyond claims data in
able readmission. predicting 30-day outcomes after heart failure
.org/dcs/ContentServer?cid=1219069855273
Author Contributions: Dr Kansagara had full access &pagename=QnetPublic%2FPage%2FQnetTier2&c hospitalization. Circ Cardiovasc Qual Outcomes. 2011;
to all of the data in the study and takes responsibility =Page. Accessed May 28, 2011. 4(1):60-67.
for the integrity of the data and the accuracy of the 5. Krumholz HM, Brindis RG, Brush JE, et al; Ameri- 19. Holloway JJ, Medendorp SV, Bromberg J. Risk fac-
data analysis. can Heart Association; Quality of Care and Out- tors for early readmission among veterans. Health Serv
Study concept and design: Kansagara, Englander, comes Research Interdisciplinary Writing Group; Coun- Res. 1990;25(1 pt 2):213-237.
Theobald, Kripalani. cil on Epidemiology and Prevention; Stroke Council; 20. Holman CDAJ, Preen DB, Baynham NJ, Finn JC,
Acquisition of data: Kansagara, Englander, Salanitro, American College of Cardiology Foundation; En- Semmens JB. A multipurpose comorbidity scoring sys-
Kagen, Theobald, Freeman, Kripalani. dorsed by the American College of Cardiology tem performed better than the Charlson index. J Clin
Analysis and interpretation of data: Kansagara, Foundation. Standards for statistical models used for Epidemiol. 2005;58(10):1006-1014.
Englander, Salanitro, Kagen, Theobald, Kripalani. public reporting of health outcomes: an American Heart 21. Howell S, Coory M, Martin J, Duckett S. Using
Drafting of the manuscript: Kansagara, Englander, Association Scientific Statement from the Quality of routine inpatient data to identify patients at risk of hos-
Salanitro, Kripalani. Care and Outcomes Research Interdisciplinary Writ- pital readmission. BMC Health Serv Res. 2009;
Critical revision of the manuscript for important in- ing Group: cosponsored by the Council on Epidemi- 9:96.
tellectual content: Kansagara, Englander, Salanitro, ology and Prevention and the Stroke Council. 22. Naessens JM, Leibson CL, Krishan I, Ballard DJ.
Kagen, Theobald, Freeman, Kripalani. Circulation. 2006;113(3):456-462. Contribution of a measure of disease complexity
Administrative, technical, or material support: 6. McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell (COMPLEX) to prediction of outcome and charges
Freeman. IG, Richardson WS; Evidence-Based Medicine Work- among hospitalized patients. Mayo Clin Proc. 1992;
Study supervision: Kansagara, Kripalani. ing Group. Users’ guides to the medical literature, XXII: 67(12):1140-1149.

©2011 American Medical Association. All rights reserved. JAMA, October 19, 2011—Vol 306, No. 15 1697

Downloaded From: on 04/26/2018

 PREDICTING THE RISK OF READMISSION

23. Philbin EF, DiSalvo TG. Prediction of hospital re- a hospital-based risk screening index. Soc Sci Med. Always Better. Hanover, NH: Dartmouth Institute for
admission for heart failure: development of a simple 1988;27(9):947-954. Health Policy and Clinical Practice; 2009.
risk score based on administrative data. J Am Coll 38. Hasan O, Meltzer DO, Shaykevich SA, et al. Hos- 51. Ashton CM, Wray NP. A conceptual framework
Cardiol. 1999;33(6):1560-1566. pital readmission in general medicine patients: a pre- for the study of early readmission as an indicator of
24. Silverstein MD, Qin H, Mercer SQ, Fong J, Haydar diction model. J Gen Intern Med. 2010;25(3):211- quality of care. Soc Sci Med. 1996;43(11):1533-
Z. Risk factors for 30-day hospital readmission in pa- 219. 1541.
tients ⬎/=65 years of age. Proc (Bayl Univ Med Cent). 39. Boult C, Dowd B, McCaffrey D, Boult L, Hernandez 52. Weissman JS, Ayanian JZ, Chasan-Taber S,
2008;21(4):363-372. R, Krulewitch H. Screening elders for risk of hospital Sherwood MJ, Roth C, Epstein AM. Hospital readmis-
25. Thomas JW. Does risk-adjusted readmission rate admission. J Am Geriatr Soc. 1993;41(8):811- sions and quality of care. Med Care. 1999;37(5):
provide valid information on hospital quality? Inquiry. 817. 490-501.
1996;33(3):258-270. 40. Allaudeen N, Schnipper JL, Orav EJ, Wachter RM, 53. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle
26. Amarasingham R, Moore BJ, Tabak YP, et al. An Vidyarthi AR. Inability of providers to predict un- PG. Systematic review: the evidence that publishing
automated model to identify heart failure patients at planned readmissions. J Gen Intern Med. 2011; patient care performance data improves quality of care.
risk for 30-day readmission or death using electronic 26(7):771-776. Ann Intern Med. 2008;148(2):111-123.
medical record data. Med Care. 2010;48(11):981- 41. Novotny NL, Anderson MA. Prediction of early 54. Colorado Foundation for Medical Care. National
988. readmission in medical inpatients using the Probabil- Coordinating Center for the Integrating Care for Popu-
27. Billings J, Mijanovich T. Improving the manage- ity of Repeated Admission instrument. Nurs Res. 2008; lations and Communities. http://www.cfmc.org
ment of care for high-cost Medicaid patients. Health 57(6):406-415. /caretransitions. Accessibility verified September 22,
Aff (Millwood). 2007;26(6):1643-1654. 42. Charlson ME, Pompei P, Ales KL, MacKenzie CR. 2011.
28. Billings J, Dixon J, Mijanovich T, Wennberg D. Case A new method of classifying prognostic comorbidity 55. Ross JS, Mulvey GK, Stauffer B, et al. Statistical
finding for patients at risk of readmission to hospital: in longitudinal studies: development and validation. models and patient predictors of readmission for heart
development of algorithm to identify high risk patients. J Chronic Dis. 1987;40(5):373-383. failure: a systematic review. Arch Intern Med. 2008;
BMJ. 2006;333(7563):327. 43. Eggli Y. Pre’vision Des Couˆts Hospitaliers Fonde’s 168(13):1371-1386.
29. Coleman EA, Min SJ, Chomiak A, Kramer AM. Post- sur le Profil des Patients [in Swiss]. Chardonne, Swit- 56. Joynt KE, Jha AK. Who has higher readmission rates
hospital care transitions: patterns, complications, and zerland: SQLape sa⬘rl; 2005. for heart failure, and why? implications for efforts to
risk identification. Health Serv Res. 2004;39(5): 44. Tabak YP, Johannes RS, Silber JH. Using auto- improve care using financial incentives. Circ Cardio-
1449-1465. mated clinical data for risk adjustment: development vasc Qual Outcomes. 2011;4(1):53-59.
30. Krumholz HM, Chen YT, Wang Y, Vaccarino V, and validation of six disease-specific mortality predic- 57. Axon RN, Williams MV. Hospital readmission as
Radford MJ, Horwitz RI. Predictors of readmission tive models for pay-for-performance. Med Care. 2007; an accountability measure. JAMA. 2011;305(5):
among elderly survivors of admission with heart failure. 45(8):789-805. 504-505.
Am Heart J. 2000;139(1 pt 1):72-77. 45. Bowen OR, Roper WL. Medicare Hospital Mor- 58. van Walraven C, Bennett C, Jennings A, Austin
31. Morrissey EFR, McElnay JC, Scott M, McConnell tality Information, 1987, Region IX: American Sa- PC, Forster AJ. Proportion of hospital readmissions
BJ. Influence of drugs, demographics and medical his- moa, Arizona, Guam, Hawaii, Nevada. Washington, deemed avoidable: a systematic review. CMAJ. 2011;
tory on hospital readmission of elderly patients: a pre- DC: US Government Printing Office; 1988. HCFA pub- 183(7):E391-E402.
dictive model. Clin Drug Invest. 2003;23(2):119- lication 00651. 59. Mukamel DB, Chou CC, Zimmer JG, Rothenberg
128. 46. Halfon P, Eggli Y, van Melle G, Chevalier J, BM. The effect of accurate patient screening on the
32. Smith DM, Norton JA, McDonald CJ. Nonelec- Wasserfallen JB, Burnand B. Measuring potentially cost-effectiveness of case management programs.
tive readmissions of medical patients. J Chronic Dis. avoidable hospital readmissions. J Clin Epidemiol. 2002; Gerontologist. 1997;37(6):777-784.
1985;38(3):213-224. 55(6):573-587. 60. Society of Hospital Medicine Project BOOST.
33. Smith DM, Weinberger M, Katz BP, Moore PS. 47. Oddone EZ, Weinberger M, Horner M, et al; Vet- Tool for addressing risk: a geriatric evaluation for
Postdischarge care and readmissions. Med Care. 1988; erans Affairs Cooperative Studies in Health Services transitions. http://www.hospitalmedicine.org
26(7):699-708. Group on Primary Care and Hospital Readmissions. /ResourceRoomRedesign/RR_CareTransitions/PDFs
34. Smith DM, Katz BP, Huster GA, Fitzgerald JF, Classifying general medicine readmissions: are they /TARGET_screen_v22.pdf. Accessed May 28, 2011.
Martin DK, Freedman JA. Risk factors for nonelective preventable? J Gen Intern Med. 1996;11(10):597- 61. Vest JR, Gamm LD, Oxford BA, Gonzalez MI,
hospital readmissions. J Gen Intern Med. 1996; 607. Slawson KM. Determinants of preventable readmis-
11(12):762-764. 48. Hernandez AF, Greiner MA, Fonarow GC, et al. sions in the United States: a systematic review. Imple-
35. van Walraven C, Dhalla IA, Bell C, et al. Deriva- Relationship between early physician follow-up and ment Sci. 2010;5:88.
tion and validation of an index to predict early death 30-day readmission among Medicare beneficiaries hos- 62. Bahadori K, FitzGerald JM. Risk factors of hospi-
or unplanned readmission after discharge from hos- pitalized for heart failure. JAMA. 2010;303(17): talization and readmission of patients with COPD ex-
pital to the community. CMAJ. 2010;182(6):551- 1716-1722. acerbation—systematic review. Int J Chron Obstruct
557. 49. Kripalani S, Jackson AT, Schnipper JL, Coleman Pulmon Dis. 2007;2(3):241-251.
36. Burns R, Nichols LO. Factors predicting readmis- EA. Promoting effective transitions of care at hospital 63. Desai MM, Stauffer BD, Feringa HHH, Schreiner
sion of older general medicine patients. J Gen Intern discharge: a review of key issues for hospitalists. J Hosp GC. Statistical models and patient predictors of read-
Med. 1991;6(5):389-393. Med. 2007;2(5):314-323. mission for acute myocardial infarction: a systematic
37. Evans RL, Hendricks RD, Lawrence KV, Bishop 50. Fisher E, Goodman D, Skinner J, Bronner K. Health review. Circ Cardiovasc Qual Outcomes. 2009;
DS. Identifying factors associated with health care use: Care Spending, Quality, and Outcomes—More Isn’t 2(5):500-507.

1698 JAMA, October 19, 2011—Vol 306, No. 15 ©2011 American Medical Association. All rights reserved.

Downloaded From: on 04/26/2018