Hoot 2008
Hoot 2008
         Emergency department (ED) crowding represents an international crisis that may affect the quality and
         access of health care. We conducted a comprehensive PubMed search to identify articles that (1)
         studied causes, effects, or solutions of ED crowding; (2) described data collection and analysis
         methodology; (3) occurred in a general ED setting; and (4) focused on everyday crowding. Two
         independent reviewers identified the relevant articles by consensus. We applied a 5-level quality
         assessment tool to grade the methodology of each study. From 4,271 abstracts and 188 full-text
         articles, the reviewers identified 93 articles meeting the inclusion criteria. A total of 33 articles studied
         causes, 27 articles studied effects, and 40 articles studied solutions of ED crowding. Commonly
         studied causes of crowding included nonurgent visits, “frequent-flyer” patients, influenza season,
         inadequate staffing, inpatient boarding, and hospital bed shortages. Commonly studied effects of
         crowding included patient mortality, transport delays, treatment delays, ambulance diversion, patient
         elopement, and financial effect. Commonly studied solutions of crowding included additional
         personnel, observation units, hospital bed access, nonurgent referrals, ambulance diversion,
         destination control, crowding measures, and queuing theory. The results illustrated the complex,
         multifaceted characteristics of the ED crowding problem. Additional high-quality studies may provide
         valuable contributions toward better understanding and alleviating the daily crisis. This structured
         overview of the literature may help to identify future directions for the crowding research agenda. [Ann
         Emerg Med. 2008;52:126-136.]
interaction of supply and demand. We defined the scope of this         the patient load. Within the groups representing causes, effects,
review to include articles that met 4 criteria: (1) they studied       and solutions of ED crowding, we further categorized articles
causes, effects, or solutions of crowding as a primary objective;      according to common themes that emerged among the primary
(2) they studied crowding on an empirical basis, with a                findings during the data abstraction phase.
description of the data collection and analysis methodology; (3)
they studied crowding in the context of general emergency              Assessment of Study Quality
medicine, rather than a specialty service such as psychiatric              To assess the methodological quality of the studies, we
emergency medicine; and (4) they studied everyday crowding,            applied a previously described 5-level instrument.15,16 Although
reflecting a focus on daily surge rather than exceptional              it was originally developed to judge clinical trials, we applied the
circumstances; in other words, they did not study crowding             instrument consistently to clinical trials, descriptive studies, and
associated with disaster events.                                       surveys by using the following adaptation: Quality level 1
    We identified a broad set of PubMed (MEDLINE) search               included prospective studies that studied a clearly defined
terms to encompass each facet of the inclusion criteria. The           outcome measure with a random or consecutive sample that was
search involved free text and Medical Subject Headings (MeSH)          large enough to achieve narrow confidence intervals and diverse
terms. We described the concept of “ED” by the following               enough to suggest generalizability of the findings. Quality level
search terms: Emergency Medical Services[MeSH] OR                      2 included prospective studies that were more limited in terms
Emergency Medicine[MeSH] OR “emergency.” We described                  of sample size or generalizability. Quality level 3 included
the concept of “crowding” by the following search terms:               retrospective studies that otherwise would have satisfied the
Crowding[MeSH] OR “crowding” OR “crowded” OR                           criteria for quality level 1 or 2. Quality level 4 included studies
“overcrowding” OR “overcrowded” OR “diversion” OR                      that sampled by convenience or other techniques that were
“divert” OR “congestion” OR “surge” OR “capacity” OR                   prone to introduce bias. Quality level 5 included studies that
“crisis” OR “crises” OR “occupancy.” We queried MEDLINE                lacked a clearly defined or validated outcome measure. We did
on June 6, 2006, with the Boolean union of the above queries,          not score articles that lacked necessary methodological details
restricting the search to English-language publications.               for the quality instrument.
Study Selection
    Two reviewers (N.R.H. and D.A.) independently examined             RESULTS
the results returned by the MEDLINE search to identify                     The MEDLINE query returned 4,271 abstracts. The
potentially relevant abstracts. Articles that clearly did not meet     reviewers identified 188 abstracts for full-text retrieval, of which
one or more of the review criteria according to the title and          93 articles satisfied the criteria for inclusion in the review. A
abstract were not considered further. When the 2 reviewers             flow diagram of the selection process is presented in the Figure
disagreed, a consensus was reached through discussion. We              1. The rate of reviewer agreement during the abstract screening
retrieved full-text articles for the potentially relevant abstracts.   phase, before consensus discussion, was 93% overall, 76%
The same 2 reviewers independently examined the full-text              among included articles, and 94% among excluded articles. The
articles to determine which studies met all 4 of the inclusion          statistic for chance-corrected agreement between the 2
criteria. Disagreements were again resolved through discussion         reviewers was 0.47 (95% confidence interval: 0.42 to 0.52),
to reach a final consensus set of articles that met the review         denoting moderate agreement.17
criteria.                                                                  We found that quality level 1 contained 14 articles, quality
                                                                       level 2 contained 12 articles, quality level 3 contained 47
Data Collection and Processing                                         articles, quality level 4 contained 10 articles, and quality level 5
    We used a data extraction form (Appendix E1, available             contained 6 articles. Four articles were not scored because of
online at http://www.annemergmed.com) to record information            inadequate reporting of methodology. The primary findings of
about the methods and results of each relevant article, including      all articles are summarized briefly in the following sections. The
study design, study setting, study population, sample size,            methods and results of each high-quality prospective study are
independent variables, dependent variables, and primary                described in Table 1. A total of 33 articles studied causes, 27
findings. We assigned the articles to nonexclusive groups              articles studied effects, and 40 articles studied solutions of ED
according to whether they investigated causes, effects, or             crowding. This sum exceeds 93 because some articles were
solutions of ED crowding. We attempted to represent the                assigned to multiple categories as necessary.
intentions of the original authors when assigning each article to
a group. For example, an issue such as ambulance diversion may         Causes
be considered a cause, effect, or solution of ED crowding,                Three general themes existed among the causes of ED
depending on the perspective of each study: it might be a cause        crowding: input factors, throughput factors, and output factors.
of crowding at nearby institutions to which patients are               These themes correspond to a conceptual framework for
diverted, it might be an effect of crowding at a single institution    studying ED crowding.18 Input factors reflected sources and
of interest, or it might be a solution of crowding by reducing         aspects of patient inflow. Throughput factors reflected
correlation between ED treatment time and hospital                              patients to receive timely care at their preferred institutions.
occupancy.41 A period of widespread hospital restructuring in                   Provider losses reflected consequences borne by the health care
Toronto independently increased the rate of severe crowding                     system itself. The commonly studied effects of crowding are
from 0.5% to 6% of the time.42 Length of stay in one ED                         summarized in Table 3.
increased substantially when the hospital occupancy levels                         Adverse outcomes. We identified patient mortality to be a
exceeded 90%.43 A survey of Korean EDs linked high hospital                     commonly studied adverse outcome of crowding.
occupancy levels to ED crowding.44 A study in Portland found                       Four articles focused on patient mortality: One study found a
that a decrease in the number of hospital beds was strongly                     significant increase in mortality associated with weekly ED
associated with an increase in ambulance diversion.45 Another                   volume.52 High occupancy in one Australian ED was estimated
study estimated that a hospital closure would affect the nearest                to cause 13 patient deaths per year.53 Another study associated a
ED by increasing ambulance diversion by 56 hours per month                      combined measure of hospital and ED crowding with an
for 4 months.46                                                                 increased risk of mortality at 2, 7, and 30 days after hospital
   Additional themes. Five surveys and interviews identified                    admission.54 In Houston, a statistically insignificant trend was
factors that health care providers and other stakeholders perceive              found for higher mortality among trauma patients who were
to be important causes of ED crowding: increasing patient                       admitted during ambulance diversion.55
volume and acuity, shortages of treatment areas, shortages of                      Reduced quality. We identified transport delays and
nursing staff, delays in ancillary services, boarding inpatients,               treatment delays to be commonly studied effects of crowding
and hospital bed shortages.47-51                                                pertaining to reduced quality.
                                                                                   Four articles examined transport delays: Ambulance diversion
Effects                                                                         was shown to increase transport time and distance in 2
   Four general themes existed among the effects of ED                          studies.56,57 A study focused on cardiac patients found that the
crowding: adverse outcomes, reduced quality, impaired access,                   90th percentile of transport time increased when multiple local
and provider losses. Adverse outcomes reflected health-related                  hospitals were on diversion.58 During 2 years in which
patient endpoints. Reduced quality reflected benchmarks of the                  crowding was exacerbated in Toronto, the 90th percentile of
care delivery process. Impaired access reflected the ability of
of-care laboratory testing, which decreased the length of stay by   Department Overcrowding Scale (NEDOCS) explained 49% of
41 minutes.84                                                       the variation in physician and nurse assessments of crowding.99
    Demand management. We identified nonurgent referrals,           The Real-time Emergency Analysis of Demand Indicators were
ambulance diversion, and destination control to be commonly         designed for real-time monitoring of ED operations, although
studied solutions of crowding involving demand management.          they did not correlate with providers’ opinions on crowding.100
   Four studies tested nonurgent referrals: A survey of ED          The Work Score predicted ambulance diversion at its institution
patients found that 38% would swap their ED visit for a             of origin with area under the receiver operating characteristic
primary care appointment within 72 hours.19 A randomized,           curve of 0.89.101 A comparative validation, which used staff
controlled trial focused on 3 common symptom complexes and          assessments of crowding as the outcome, estimated the area
found that they may be deferred for next-day primary care           under the receiver operating characteristic curve of the EDWIN
without worsening self-reported health status on follow-up.85       to be 0.80 and of the NEDOCS to be 0.83.102 However, an
When following up nonurgent patients who were triaged to            external validation of the NEDOCS in Australia concluded that
receive care elsewhere, one group found that there were no          it was not useful, according to Bland-Altman and  statistics.103
major adverse outcomes, and 42% of the patients received same-      A sampling form consisting of 7 operational measures was
day care elsewhere.86 A similar study found that 94% of             shown to correlate well with staff assessments of crowding.104 A
nonurgent patients who were referred to community-based care        panel of experts described 38 consensus operational measures
reported that their condition was better or unchanged.87            that may be used to assess crowding levels.105
   Five studies investigated ambulance diversion: By one               Two studies used queuing theory: One group illustrated the
calculation, ambulance diversion decreased the rate of              ability of discrete event simulation to model ED operations, and
ambulance arrivals by 30% to 50%.88 A similar calculation           they tested its applicability by analyzing a proposed triage
found that “red-alert” ambulance diversion reduced the arrival      scheme.106 A similar study described a separate discrete event
rate by 0.4 per hour.89 When one hospital committed to              simulation and studied the effects of physician utilization on
avoiding ambulance diversion for 1 week, the need for diversion     patient waiting times.107
at a nearby hospital was almost eliminated.90 Standardized              Additional themes. Five studies described multifaceted
                                                                    administrative interventions that could not be classified
diversion criteria in Sacramento, targeted to decrease “round-
                                                                    separately: A broad intervention consisting of 51 actions
robin” crowding, reduced the rate of ambulance diversion by
                                                                    reduced ED length of stay and ambulance diversion in
74% despite increased patient volume.91 San Diego
                                                                    Melbourne.108 One network deployed several interventions,
implemented a standardized policy for initiating ambulance
                                                                    tuned for the individual needs of 4 hospitals, and reduced the
diversion among all local hospitals and reduced ambulance
                                                                    amount of ambulance diversion by 25% and 34% in
diversion by 75%.92
                                                                    consecutive years.109 A group of hospitals in Rochester deployed
   Two studies proposed destination control: The use of
                                                                    several interventions, and they reported that the most effective
Internet-accessible operating information to redistribute
                                                                    interventions occurred outside the ED.110 Another study
ambulances reduced the need for diversion from 1,788 hours to
                                                                    reported interventions, including more physicians, improved
1,138 hours in one network.93 Another study described a
                                                                    ancillary services, and changes in hospital policy, that reduced
physician-directed ambulance destination control initiative that    length of stay by half.111 One hospital deployed a multipronged
reduced diversion by 41%.94                                         intervention, which involved a short-stay unit, additional
   Three studies considered other aspects of demand                 physicians, and an early warning system, to deal with holiday
management: A trial of paramedic-initiated nontransport found       demand surges.112
that 2.4% of nontransported pediatric patients were later
admitted to the hospital.95 Three social interventions designed     LIMITATIONS
for frequent visitors, which included education and counseling,         This study has a number of limitations that merit discussion.
were associated with decreased ED utilization.96 Another study      First, we may not have captured every article that studied
targeted frequent users with case management interventions, but     causes, effects, and solutions of ED crowding. We limited the
the rate of ED utilization was unchanged.97                         search to English-language articles, so any relevant articles
    Operations research. The studies within the operations          published in foreign languages were not included. We avoided
research theme did not describe direct solutions to ED              searching the grey literature with a general purpose internet
crowding; however, they proposed to support solutions through       query, and we did not hand-search the references of included
improved business intelligence. We identified crowding              articles. If used, these 2 techniques might have impaired the
measures and queuing theory to be commonly studied solutions        reproducibility of our review. We searched a single database;
to crowding according to operations research.                       moreover, it is possible that our search terms did not capture all
   Eight studies described crowding measures: The Emergency         aspects of the topic. The MeSH vocabulary contains a single
Department Work Index (EDWIN) associated well with                  term related to crowding, so we supplemented the search with a
ambulance diversion and less well with secondary outcome            large set of free-text keywords. We attempted to minimize the
measures at its institution of origin.98 The National Emergency     likelihood of missed articles by applying a broad search strategy.
We also used a conservative approach during the abstract               operational changes involve the entire department, rather than
screening phase, retrieving the full-text articles for all abstracts   individual patients who may be randomized to experimental
that could not be clearly excluded. The moderate  value may           and control groups.85 We believe that the crowding literature
be explained because one author was more conservative than the         would benefit from more randomized controlled trials
other in marking abstracts for full-text retrieval. This issue was     examining patient-focused interventions.
identified and resolved during the consensus discussion. We                Although several studies investigated nonurgent and
believe our methodology captured the majority of pertinent             frequent-flyer visits, relatively little evidence suggests they
articles.                                                              independently cause ED crowding.19-23,25 This notion is
    Second, the diversity of methodology, outcome measures,            supported by recent literature.113 More evidence is available to
and reporting among the original articles rendered aspects of          identify inpatient boarding and other hospital-related factors as
this review difficult. We attempted to describe the primary            causes of ED crowding.33,34,38-46 These studies corroborate with
findings of each study as consistently as possible, noting the         successful interventions that reduced crowding by altering the
effect sizes of each study when feasible and in other cases            operation of hospital and community services other than the
describing the nature of the findings in more qualitative terms.       ED.78,79,81,82,90-93 We believe that the crowding literature
In some cases, our descriptions were limited according to the          would benefit from more studies that analyze the ED in the
reporting of the original articles. The brief summaries that we        context of integrated hospital processes and focus on
provide do not capture the full complexity of each study, so our       multicenter community networks rather than single institutions.
review is intended to guide interested readers to the original             The results suggest that standard operations management
cited articles. We did not conduct a formal meta-analysis,             tools, such as queuing theory, have only recently been applied in
because of the breadth of study designs and endpoints                  an effort to improve ED patient flow.106,107 We are aware of
considered. We refrain from making strong conclusions about            few previous reports describing such applications in the ED
which factors are most important because these would be based          setting.114 By contrast, these tools were adopted much earlier by
primarily on judgment rather than numeric inference.                   industries like airlines and manufacturing. This lag is analogous
    Third, the classification of studies into groups and themes        to the gap between basic science and clinical science, which
was partly subjective, so objections may be made regarding how         translational research aims to address. A result of queuing theory
particular articles were categorized. We acknowledge that there        states that a system with varying inputs and fixed capacity will
may be no clearly correct taxonomy for grouping this diverse set
                                                                       become congested for transient periods.115 By consequence,
of articles. For instance, measurement tools and queuing models
                                                                       permanent increases in resources may be neither efficient nor
would not reduce ED crowding unless paired with an
                                                                       adequate to address crowding, given the fluctuating demand.
intervention plan. Regardless, we have classified these articles as
                                                                       The review includes 1 study demonstrating the feasibility of
solutions, insofar as the original authors intended their research
                                                                       deploying additional resources on demand to alleviate ED
to support crowding interventions. Our intention in using this
                                                                       crowding.76 We believe that the crowding literature would
trichotomy of causes, effects, and solutions was to provide a
                                                                       benefit from studies that apply standard management research
structured overview of the relevant literature, which we hope
                                                                       techniques to ED operations and investigate ways to alter
benefits the reader.
                                                                       resource availability dynamically according to demand.
DISCUSSION                                                                 When considered as a whole, the body of literature
    A substantial body of literature exists describing the causes,     demonstrates that ED crowding is a local manifestation of a
effects, and solutions of ED crowding. The major themes                systemic disease. The causes of ED crowding involve a complex
among the causes of crowding included nonurgent visits,                network of interwoven processes ranging from hospital
frequent-flyer patients, influenza season, inadequate staffing,        workflow to viral epidemics. The effects of ED crowding are
inpatient boarding, and hospital bed shortages. The major              numerous and adverse. Various targeted solutions to crowding
themes among the effects of crowding included patient                  have been shown to be effective, and further studies may
mortality, transport delays, treatment delays, ambulance               demonstrate new innovations. This broad overview of the
diversion, patient elopement, and financial effect. The major          current research may help to inform the future research agenda
themes among the solutions of crowding included additional             and, subsequently, to protect the fragile safety net of the health
personnel, observation units, hospital bed access, nonurgent           care system.
referrals, ambulance diversion, destination control, crowding
measures, and queuing theory.                                          Supervising editor: David J. Magid, MD, MPH
    The quality instrument that we used indicated that a large         Funding and support: By Annals policy, all authors are required
number of high-quality articles have been published about ED           to disclose any and all commercial, financial, and other
crowding.15,16 We identified a total of 26 prospective studies         relationships in any way related to the subject of this article,
and 47 retrospective studies that met the criteria for the 3           that might create any potential conflict of interest. See the
highest quality levels. We noted a scarcity of randomized              Manuscript Submission Agreement in this issue for examples
controlled trials in this review, perhaps because many ED              of specific conflicts covered by this statement. Dr. Hoot was
supported by National Library of Medicine grant LM07450-02               18. Asplin BR, Magid DJ, Rhodes KV, et al. A conceptual model of
and National Institute of General Medical Studies grant T32                  emergency department crowding. Ann Emerg Med. 2003;42:
GM07347. The research was also supported by National                         173-180.
Library of Medicine grant R21 LM009002-01. The authors                   19. Grumbach K, Keane D, Bindman A. Primary care and public
declare no conflicts of interest pertaining to the publication of            emergency department overcrowding. Am J Public Health. 1993;
                                                                             83:372-378.
this work.
                                                                         20. Afilalo J, Marinovich A, Afilalo M, et al. Nonurgent emergency
Publication dates: Received for publication July 16, 2007.                   department patient characteristics and barriers to primary care.
Revision received January 26, 2008. Accepted for publication                 Acad Emerg Med. 2004;11:1302-1310.
March 11, 2008. Available online April 23, 2008.                         21. Howard MS, Davis BA, Anderson C, et al. Patients’ perspective
                                                                             on choosing the emergency department for nonurgent medical
Earn CME Credit: Continuing Medical Education for this article               care: a qualitative study exploring one reason for overcrowding.
is available at: www.ACEP-EMedHome.com.                                      J Emerg Nurs. 2005;31:429-435.
                                                                         22. Sprivulis P, Grainger S, Nagree Y. Ambulance diversion is not
Reprints not available from the authors.
                                                                             associated with low acuity patients attending Perth metropolitan
Address for correspondence: Nathan R. Hoot, PhD, 400                         emergency departments. Emerg Med Australas. 2005;17:11-15.
Eskind Biomedical Library, 2209 Garland Avenue, Nashville,               23. Huang JA, Tsai WC, Chen YC, et al. Factors associated with
TN 37232; 615-936-3720, fax 615-936-1427; E-mail                             frequent use of emergency services in a medical center. J
                                                                             Formos Med Assoc. 2003;102:222-228.
nathan.hoot@vanderbilt.edu.
                                                                         24. Andersen RM. Revisiting the behavioral model and access to
                                                                             medical care: does it matter? J Health Soc Behav. 1995;36:1-
REFERENCES                                                                   10.
  1. Yamane K. Hospital Emergency Departments: Crowded                   25. Dent AW, Phillips GA, Chenhall AJ, et al. The heaviest repeat
     Conditions Vary Among Hospitals and Communities. Washington,            users of an inner city emergency department are not general
     DC: US General Accounting Office; 2003. GAO-03-460.                     practice patients. Emerg Med (Fremantle). 2003;15:322-329.
  2. Committee on the Future of Emergency Care in the United States      26. Glaser CA, Gilliam S, Thompson WW, et al. Medical care
     Health System. Hospital-Based Emergency Care: At the Breaking           capacity for influenza outbreaks, Los Angeles. Emerg Infect Dis.
     Point. Washington, DC: National Academies Press; 2006.                  2002;8:569-574.
  3. Gibbs N. Do you want to die? The crisis in emergency care is        27. Schull MJ, Mamdani MM, Fang J. Influenza and emergency
     taking its toll on doctors, nurses, and patients. Time. May 28,         department utilization by elders. Acad Emerg Med. 2005;12:
     1990:58-65.                                                             338-344.
  4. Barrero J. Hospitals get orders to reduce crowding in emergency     28. Schull MJ, Mamdani MM, Fang J. Community influenza
     rooms. New York Times. January 24, 1989:1-2.                            outbreaks and emergency department ambulance diversion. Ann
  5. Goldberg C. Emergency crews worry as hospitals say, “No                 Emerg Med. 2004;44:61-67.
     vacancy.” New York Times. December 17, 2000:39                      29. Andersson G, Karlberg I. Lack of integration, and seasonal
  6. Orenstein JB. State of emergency. Washington Post. April 22,            variations in demand explained performance problems and
     2001:B1                                                                 waiting times for patients at emergency departments: a 3 years
  7. Jeffrey NA. Who’s crowding emergency rooms? Right now it’s              evaluation of the shift of responsibility between primary and
     managed-care patients. Wall Street Journal. July 20, 1999:B1            secondary care by closure of two acute hospitals. Health Policy.
  8. Emergency Medical Treatment and Active Labor Act, established           2001;55:187-207.
     under the Consolidated Omnibus Budget Reconciliation Act of         30. Fatovich DM, Hirsch RL. Entry overload, emergency department
     1985. Pub L No. 99-272, 42USC 1395dd (1986).                            overcrowding, and ambulance bypass. Emerg Med J. 2003;20:
  9. Asplin BR. Tying a knot in the unraveling health care safety net.       406-409.
     Acad Emerg Med. 2001;8:1075-1079.
                                                                         31. Baer RB, Pasternack JS, Zwemer FL Jr. Recently discharged
 10. American Academy of Pediatrics Committee on Pediatric
                                                                             inpatients as a source of emergency department overcrowding.
     Emergency Medicine.Overcrowding crisis in our nation’s
                                                                             Acad Emerg Med. 2001;8:1091-1094.
     emergency departments: is our safety net unraveling?
                                                                         32. Lambe S, Washington DL, Fink A, et al. Waiting times in
     Pediatrics. 2004;114:878-888.
                                                                             California’s emergency departments. Ann Emerg Med. 2003;41:
 11. Kellermann AL. Crisis in the emergency department. N Engl
                                                                             35-44.
     J Med. 2006;355:1300-1303.
                                                                         33. Schneider SM, Gallery ME, Schafermeyer R, et al. Emergency
 12. Hwang U, Concato J. Care in the emergency department: how
     crowded is overcrowded? Acad Emerg Med. 2004;11:1097-1101.              department crowding: a point in time. Ann Emerg Med. 2003;42:
 13. Pham JC, Patel R, Millin MG, et al. The effects of ambulance            167-172.
     diversion: a comprehensive review. Acad Emerg Med. 2006;13:         34. Schull MJ, Lazier K, Vermeulen M, et al. Emergency department
     1220-1227.                                                              contributors to ambulance diversion: a quantitative analysis. Ann
 14. American College of Emergency Physicians. Crowding. Ann                 Emerg Med. 2003;41:467-476.
     Emerg Med. 2006;47:585.                                             35. Lambe S, Washington DL, Fink A, et al. Trends in the use and
 15. Sackett DL, Haynes RB, Guyatt GH, et al. Clinical Epidemiology:         capacity of California’s emergency departments, 1990-1999.
     A Basic Science for Clinical Medicine. 2nd ed. Boston, MA: Little       Ann Emerg Med. 2002;39:389-396.
     Brown; 1991.                                                        36. Polevoi SK, Quinn JV, Kramer NR. Factors associated with
 16. Wang CS, FitzGerald JM, Schulzer M, et al. Does this dyspneic           patients who leave without being seen. Acad Emerg Med. 2005;
     patient in the emergency department have congestive heart               12:232-236.
     failure? JAMA. 2005;294:1944-1956.                                  37. Davis B, Sullivan S, Levine A, et al. Factors affecting ED length-
 17. Landis JR, Koch GG. The measurement of observer agreement               of-stay in surgical critical care patients. Am J Emerg Med. 1995;
     for categorical data. Biometrics. 1977;33:159-174.                      13:495-500.
 38. Andrulis DP, Kellermann A, Hintz EA, et al. Emergency              58. Schull MJ, Morrison LJ, Vermeulen M, et al. Emergency
     departments and crowding in United States teaching hospitals.          department gridlock and out-of-hospital delays for cardiac
     Ann Emerg Med. 1991;20:980-986.                                        patients. Acad Emerg Med. 2003;10:709-716.
 39. Fromm RE Jr, Gibbs LR, McCallum WG, et al. Critical care in the    59. Schull MJ, Morrison LJ, Vermeulen M, et al. Emergency
     emergency department: a time-based study. Crit Care Med.               department overcrowding and ambulance transport delays for
     1993;21:970-976.                                                       patients with chest pain. CMAJ. 2003;168:277-283.
 40. Fatovich DM, Nagree Y, Sprivulis P. Access block causes            60. Liu S, Hobgood C, Brice JH. Impact of critical bed status on
     emergency department overcrowding and ambulance diversion in           emergency department patient flow and overcrowding. Acad
     Perth, Western Australia. Emerg Med J. 2005;22:351-354.                Emerg Med. 2003;10:382-385.
 41. Cooke MW, Wilson S, Halsall J, et al. Total time in English        61. Schull MJ, Vermeulen M, Slaughter G, et al. Emergency
     accident and emergency departments is related to bed                   department crowding and thrombolysis delays in acute
     occupancy. Emerg Med J. 2004;21:575-576.                               myocardial infarction. Ann Emerg Med. 2004;44:577-585.
 42. Schull MJ, Szalai JP, Schwartz B, et al. Emergency department      62. Hwang U, Richardson LD, Sonuyi TO, et al. The effect of
     overcrowding following systematic hospital restructuring: trends       emergency department crowding on the management of pain in
     at twenty hospitals over ten years. Acad Emerg Med. 2001;8:            older adults with hip fracture. J Am Geriatr Soc. 2006;54:270-
     1037-1043.                                                             275.
 43. Forster AJ, Stiell I, Wells G, et al. The effect of hospital       63. Chen EH, Mills AM, Lee BY, et al. The impact of a concurrent
     occupancy on emergency department length of stay and patient           trauma alert evaluation on time to head computed tomography
     disposition. Acad Emerg Med. 2003;10:127-133.                          in patients with suspected stroke. Acad Emerg Med. 2006;13:
 44. Hwang JI. The relationship between hospital capacity                   349-352.
     characteristics and emergency department volumes in Korea.         64. Burt CW, McCaig LF, Valverde RH. Analysis of ambulance
     Health Policy. 2006;79:274-283.                                        transports and diversions among US emergency departments.
 45. Warden CR, Bangs C, Norton R, et al. Temporal trends in                Ann Emerg Med. 2006;47:317-326.
     ambulance diversion in a mid-sized metropolitan area. Prehosp      65. Kyriacou DN, Ricketts V, Dyne PL, et al. A 5-year time study
                                                                            analysis of emergency department patient care efficiency. Ann
     Emerg Care. 2003;7:109-113.
                                                                            Emerg Med. 1999;34:326-335.
 46. Sun BC, Mohanty SA, Weiss R, et al. Effects of hospital
                                                                        66. Weiss SJ, Ernst AA, Derlet R, et al. Relationship between the
     closures and hospital characteristics on emergency department
                                                                            National ED Overcrowding Scale and the number of patients who
     ambulance diversion, Los Angeles County, 1998 to 2004. Ann
                                                                            leave without being seen in an academic ED. Am J Emerg Med.
     Emerg Med. 2006;47:309-316.
                                                                            2005;23:288-294.
 47. Derlet RW, Richards JR. Emergency department overcrowding in
                                                                        67. Baker DW, Stevens CD, Brook RH. Patients who leave a public
     Florida, New York, and Texas. South Med J. 2002;95:846-849.
                                                                            hospital emergency department without being seen by a
 48. Derlet R, Richards J, Kravitz R. Frequent overcrowding in U.S.
                                                                            physician. Causes and consequences. JAMA. 1991;266:1085-
     emergency departments. Acad Emerg Med. 2001;8:151-155.
                                                                            1090.
 49. Richards JR, Navarro ML, Derlet RW. Survey of directors of
                                                                        68. Rowe BH, Channan P, Bullard M, et al. Characteristics of
     emergency departments in California on overcrowding. West
                                                                            patients who leave emergency departments without being seen.
     J Med. 2000;172:385-388.
                                                                            Acad Emerg Med. 2006;13:848-852.
 50. Clark K, Normile LB. Delays in implementing admission orders
                                                                        69. Bindman AB, Grumbach K, Keane D, et al. Consequences of
     for critical care patients associated with length of stay in
                                                                            queuing for care at a public hospital emergency department.
     emergency departments in six mid-Atlantic states. J Emerg Nurs.        JAMA. 1991;266:1091-1096.
     2002;28:489-495.                                                   70. Bayley MD, Schwartz JS, Shofer FS, et al. The financial burden
 51. Bazzoli GJ, Brewster LR, Liu G, et al. Does U.S. hospital              of emergency department congestion and hospital crowding for
     capacity need to be expanded? Health Aff (Millwood). 2003;22:          chest pain patients awaiting admission. Ann Emerg Med. 2005;
     40-54.                                                                 45:110-117.
 52. Miro O, Antonio MT, Jimenez S, et al. Decreased health care        71. Krochmal P, Riley TA. Increased health care costs associated
     quality associated with emergency department overcrowding. Eur         with ED overcrowding. Am J Emerg Med. 1994;12:265-266.
     J Emerg Med. 1999;6:105-107.                                       72. Eckstein M, Chan LS. The effect of emergency department
 53. Richardson DB. Increase in patient mortality at 10 days                crowding on paramedic ambulance availability. Ann Emerg Med.
     associated with emergency department overcrowding. Med J               2004;43:100-105.
     Aust. 2006;184:213-216.                                            73. Rondeau KV, Francescutti LH. Emergency department
 54. Sprivulis PC, Da Silva JA, Jacobs IG, et al. The association           overcrowding: the impact of resource scarcity on physician job
     between hospital overcrowding and mortality among patients             satisfaction. J Healthc Manag. 2005;50:327-340.
     admitted via Western Australian emergency departments. Med J       74. Bucheli B, Martina B. Reduced length of stay in medical
     Aust. 2006;184:208-212.                                                emergency department patients: a prospective controlled study
 55. Begley CE, Chang Y, Wood RC, et al. Emergency department               on emergency physician staffing. Eur J Emerg Med. 2004;11:29-
     diversion and trauma mortality: evidence from Houston, Texas.          34.
     J Trauma. 2004;57:1260-1265.                                       75. Donald KJ, Smith AN, Doherty S, et al. Effect of an on-site
 56. Neely KW, Norton RL, Young GP. The effect of hospital resource         emergency physician in a rural emergency department at night.
     unavailability and ambulance diversions on the EMS system.             Rural Remote Health. 2005;5:380.
     Prehosp Disaster Med. 1994;9:172-176.                              76. Shaw KN, Lavelle JM. VESAS: a solution to seasonal
 57. Redelmeier DA, Blair PJ, Collins WE. No place to unload: a             fluctuations in emergency department census. Ann Emerg Med.
     preliminary analysis of the prevalence, risk factors, and              1998;32:698-702.
     consequences of ambulance diversion. Ann Emerg Med. 1994;          77. Bazarian JJ, Schneider SM, Newman VJ, et al. Do admitted
     23:43-47.                                                              patients held in the emergency department impact the
      throughput of treat-and-release patients? Acad Emerg Med.             96. Michelen W, Martinez J, Lee A, et al. Reducing frequent flyer
      1996;3:1113-1118.                                                         emergency department visits. J Health Care Poor Underserved.
78.   Kelen GD, Scheulen JJ, Hill PM. Effect of an emergency                    2006;17(1 suppl):59-69.
      department (ED) managed acute care unit on ED overcrowding            97. Lee KH, Davenport L. Can case management interventions
      and emergency medical services diversion. Acad Emerg Med.                 reduce the number of emergency department visits by frequent
      2001;8:1095-1100.                                                         users? Health Care Manag (Frederick). 2006;25:155-159.
79.   Moloney ED, Bennett K, O’Riordan D, et al. Emergency                  98. Bernstein SL, Verghese V, Leung W, et al. Development and
      department census of patients awaiting admission following                validation of a new index to measure emergency department
      reorganisation of an admissions process. Emerg Med J. 2006;               crowding. Acad Emerg Med. 2003;10:938-942.
      23:363-367.                                                           99. Weiss SJ, Derlet R, Arndahl J, et al. Estimating the degree of
80.   Ross MA, Naylor S, Compton S, et al. Maximizing use of the                emergency department overcrowding in academic medical
      emergency department observation unit: a novel hybrid design.             centers: results of the National ED Overcrowding Study
      Ann Emerg Med. 2001;37:267-274.                                           (NEDOCS). Acad Emerg Med. 2004;11:38-50.
81.   McConnell KJ, Richards CF, Daya M, et al. Effect of increased        100. Reeder TJ, Burleson DL, Garrison HG. The overcrowded
      ICU capacity on emergency department length of stay and                   emergency department: a comparison of staff perceptions. Acad
      ambulance diversion. Ann Emerg Med. 2005;45:471-478.                      Emerg Med. 2003;10:1059-1064.
82.   Dunn R. Reduced access block causes shorter emergency                101. Epstein SK, Tian L. Development of an emergency department
      department waiting times: an historical control observational             work score to predict ambulance diversion. Acad Emerg Med.
      study. Emerg Med (Fremantle). 2003;15:232-238.                            2006;13:421-426.
83.   Miro O, Sanchez M, Espinosa G, et al. Analysis of patient flow in    102. Weiss SJ, Ernst AA, Nick TG. Comparison of the National
      the emergency department and the effect of an extensive                   Emergency Department Overcrowding Scale and the Emergency
      reorganisation. Emerg Med J. 2003;20:143-148.                             Department Work Index for quantifying emergency department
84.   Lee-Lewandrowski E, Corboy D, Lewandrowski K, et al.                      crowding. Acad Emerg Med. 2006;13:513-518.
      Implementation of a point-of-care satellite laboratory in the        103. Raj K, Baker K, Brierley S, et al. National Emergency
      emergency department of an academic medical center. Impact                Department Overcrowding Study tool is not useful in an
      on test turnaround time and patient emergency department
                                                                                Australian emergency department. Emerg Med Australas. 2006;
      length of stay. Arch Pathol Lab Med. 2003;127:456-460.
                                                                                18:282-288.
85.   Washington DL, Stevens CD, Shekelle PG, et al. Next-day care
                                                                           104. Weiss SJ, Arndahl J, Ernst AA, et al. Development of a site
      for emergency department users with nonacute conditions. A
                                                                                sampling form for evaluation of ED overcrowding. Med Sci
      randomized, controlled trial. Ann Intern Med. 2002;137:707-
                                                                                Monit. 2002;8:CR549-553.
      714.
                                                                           105. Solberg LI, Asplin BR, Weinick RM, et al. Emergency department
86.   Derlet RW, Nishio D, Cole LM, et al. Triage of patients out of the
                                                                                crowding: consensus development of potential measures. Ann
      emergency department: three-year experience. Am J Emerg Med.
                                                                                Emerg Med. 2003;42:824-834.
      1992;10:195-199.
                                                                           106. Connelly LG, Bair AE. Discrete event simulation of emergency
87.   Diesburg-Stanwood A, Scott J, Oman K, et al. Nonemergent ED
                                                                                department activity: a platform for system-level operations
      patients referred to community resources after medical
                                                                                research. Acad Emerg Med. 2004;11:1177-1185.
      screening examination: characteristics, medical condition after
                                                                           107. Chin L, Fleisher G. Planning model of resource utilization in an
      72 hours, and use of follow-up services. J Emerg Nurs. 2004;
      30:312-317.                                                               academic pediatric emergency department. Pediatr Emerg Care.
88.   Lagoe RJ, Hunt RC, Nadle PA, et al. Utilization and impact of             1998;14:4-9.
      ambulance diversion at the community level. Prehosp Emerg            108. Cameron P, Scown P, Campbell D. Managing access block. Aust
      Care. 2002;6:191-198.                                                     Health Rev. 2002;25:59-68.
89.   Scheulen JJ, Li G, Kelen GD. Impact of ambulance diversion           109. Lagoe RJ, Kohlbrenner JC, Hall LD, et al. Reducing ambulance
      policies in urban, suburban, and rural areas of Central Maryland.         diversion: a multihospital approach. Prehosp Emerg Care. 2003;
      Acad Emerg Med. 2001;8:36-40.                                             7:99-108.
90.   Vilke GM, Brown L, Skogland P, et al. Approach to decreasing         110. Schneider S, Zwemer F, Doniger A, et al. Rochester, New York:
      emergency department ambulance diversion hours. J Emerg                   a decade of emergency department overcrowding. Acad Emerg
      Med. 2004;26:189-192.                                                     Med. 2001;8:1044-1050.
91.   Patel PB, Derlet RW, Vinson DR, et al. Ambulance diversion           111. Cardin S, Afilalo M, Lang E, et al. Intervention to decrease
      reduction: the Sacramento solution. Am J Emerg Med. 2006;24:              emergency department crowding: does it have an effect on
      206-213.                                                                  return visits and hospital readmissions? Ann Emerg Med. 2003;
92.   Vilke GM, Castillo EM, Metz MA, et al. Community trial to                 41:173-185.
      decrease ambulance diversion hours: the San Diego county             112. Salazar A, Corbella X, Sanchez JL, et al. How to manage the ED
      patient destination trial. Ann Emerg Med. 2004;44:295-303.                crisis when hospital and/or ED capacity is reaching its limits.
93.   Sprivulis P, Gerrard B. Internet-accessible emergency                     Report about the implementation of particular interventions
      department workload information reduces ambulance diversion.              during the Christmas crisis. Eur J Emerg Med. 2002;9:79-80.
      Prehosp Emerg Care. 2005;9:285-291.                                  113. Schull MJ, Kiss A, Szalai JP. The effect of low-complexity
94.   Shah MN, Fairbanks RJ, Maddow CL, et al. Description and                  patients on emergency department waiting times. Ann Emerg
      evaluation of a pilot physician-directed emergency medical                Med. 2007;49:257-264.
      services diversion control program. Acad Emerg Med. 2006;13:         114. Siddharthan K, Jones WJ, Johnson JA. A priority queuing model
      54-60.                                                                    to reduce waiting times in emergency care. Int J Health Care
95.   Haines CJ, Lutes RE, Blaser M, et al. Paramedic initiated non-            Qual Assur. 1996;9:10-16.
      transport of pediatric patients. Prehosp Emerg Care. 2006;10:        115. Gross D, Harris CM. Fundamentals of Queuing Theory. New
      213-219.                                                                  York, NY: Wiley; 1985.
      Design: ________________________
                               _______________________________________
Institution: _____________________________________________________________
      Population: ________________________
                                ____________________________________
Sample: _______________________________________________________________
Endpoint: ______________________________________________________________
Analysis: ______________________________________________________________
Causes: _______________________________________________________________
Effects: _______________________________________________________________
Solutions: _____________________________________________________________
      Notes: ________________________________________________________________
                                 Appendix E1. Data extraction form