Computers & Industrial Engineering 128 (2019) 526–540

Contents lists available at ScienceDirect

Computers & Industrial Engineering

journal homepage: www.elsevier.com/locate/caie

Discrete simulation-based optimization methods for industrial engineering T

problems: A systematic literature review
Wilson Trigueiro de Sousa Juniora,b, , José Arnaldo Barra Montevechia,
⁎

Rafael de Carvalho Mirandaa, Afonso Teberga Camposa

a
Industrial Engineering and Management Institute, Federal University of Itajubá (UNIFEI), Minas Gerais, Brazil
b
Academic Unit of Industrial Engineering, Federal University of São João del-Rei (UFSJ), Minas Gerais, Brazil

ARTICLE INFO ABSTRACT

Keywords: In recent years, some attention has been driven to modeling, simulation, and optimization techniques capable of
Discrete event simulation representing and improving discrete event systems. These techniques can support decision making helping to
Industrial engineering determine the best scenario on a combinatorial search space with stochastic variables. This paper presents
Optimization findings from a systematic literature review of discrete simulation-based optimization applied to industrial
engineering problems. It indicates the most frequent contexts, problems, methods, tools, and intended results of
discrete-simulation based studies published in the last 25 years (1991–2016) in scientific journals and conference
proceedings. The four research questions presented a scenario to help practitioners and researchers to develop
simulation optimization projects for industrial engineering problems. A conclusion presented the gap and pro-
spects found during the writing of the research.

1. Introduction Dahal, Galloway, Burt, McDonald, and Hopkins (2005) applied a ge-
netic algorithm to the bulk material port handling. Gourgand,
The management of a production system (goods and services) de- Grangeon, and Norre (2003) tested scheduling problems in m machine
mands reliable tools to help the routine of making decisions with the stochastic flow shop with unlimited buffer. Li, Jia, and Wang (2012)
purpose of satisfying customers, minimizing costs and making a profit used DES with the multiple-comparison procedure to define the best
contribution while maintaining competitiveness (Salam & Khan, 2016). average project duration. Moengin, Septiani, and Herviana (2014) op-
Discrete Simulation-Based Optimization (DSBO) is a set of tools and timized the number of hospital beds using DES. With the aforemen-
methods commonly used to help researchers and practitioners, re- tioned studies, it is possible to infer the full range of possible problems
garding analysis and decision making, for investment and resource al- related to the production of goods and services and its variances inside
location in new or already existing systems. DSBO evaluates a specific the same problem and why the need for such different methodologies.
solution space in order to find the best setting that will help to improve Considering DSBO characteristics, related Literature Reviews (LR)
key performance indicators (e.g. service level, delivery lead times, from 1991 to 2016 only refers to methods applied on specific cases
average lateness) in favor to product quality (Gansterer, Almeder, & related to a defined situation and a restrict number of methods, i.e.,
Hartl, 2014; Merkuryeva & Bolshakov, 2014; Merkuryeva, Merkuryev, none of them refers to the industrial sector (manufacturing/service) in a
& Vanmaele, 2010). broad way, to help in the early stages of the optimization process. The
The use of DSBO for stochastic NP-hard problems demands sophis- purpose of this work is to use the systematic literature review (SLR)
ticated methods and for it, knowledge in specific areas of operations methodology, answering the research questions, to present the findings
research such as computational modeling and heuristics/metaheuristics and generate a set of discussion that can help practitioners and re-
optimization algorithms (Laroque, Klaas, Fischer, & Kuntze, 2012). searchers to overview the most used DSBO techniques and contribute
Many articles published in this area refers to the solution of the specific with their projects on industrial engineering. As a result, can help with
problem by one or more methods. Ahmad, Subramaniam, Othman, and the planning for the knowledge that should be managed, created and
Zulkarnain (2011) studied the real-time scheduling problem using DES. considered in such type of project.

⁎
Corresponding author at: Department of Mechanics, Room 4.18MP, Industrial Engineering Course, Praça Frei Orlando, 170, Centro – Campus Santo Antônio,
36307-352 São João Del Rei, Minas Gerais, Brazil.
E-mail address: wilson.trigueiro@ufsj.edu.br (W. Trigueiro de Sousa Junior).

https://doi.org/10.1016/j.cie.2018.12.073
Received 31 July 2017; Received in revised form 11 December 2018; Accepted 31 December 2018
Available online 03 January 2019
0360-8352/ © 2019 Elsevier Ltd. All rights reserved.
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

To accomplish the purpose of the research, four steps were devel- simulation has sufficient data to represent the analyzed system, the
oped related to (1) specify and apply a research methodology com- best-simulated solution can be inferred as optimal, and have good
bining CIMO and SLR to study DSBO, that can be a reference to future chances to be implemented in a real system, performing the goal to be
works; (2) create an up-to-date reference set that relates industrial an excellent tool to help decision making (Taha, 2007).
engineering problems and the solutions tried to resolve using DSBO; (3) To find the optimal solution, a search space made from the com-
answer the CIMO-logic research questions; and (4) introduce future bination of the possible values from the variables is evaluated. The size
work directions pointed out by the articles. of this search space can be a problem regarding the resources necessary
The defined CIMO-logic questions are (A) Which are the main to perform a full search covering all the possible solutions, to find the
problems studied, related to the area of Industrial Engineering? (B) best one. The resources, in this case, are commonly related to the
Which optimization and implementation software methods were the computational power available to perform all the possible solutions that
most used? (C) How were the results measured? (D) Which author, represent a quantity of time that the decision-making person could not
university, publication year and journal were found that compose the have. Those types of problem are considered NP-hard (He, Liang, Liu, &
reference research centers? In the seek to answer these questions, the Hui, 2017; Herrmann, 2013; Nawara & Hassanein, 2013). According to
article contributes to the theoretical development of DSBO, gathering Banks (1998), Chen, Jia, and Lee (2013), Xu, Huang, Chen, and Lee
the work of researches in this specific area for a new methodological (2015) a simulation optimization problem can be formulated as stated
classification, expanding the already existing. Besides the creation of a in Eq. (1).
classification, new perspectives are related to the creation, develop-
ment, and solution of a DSBO project suggesting the already exciting max J (x ) = E [L (x; w )],
x X (1)
methods to be considered and analyzed.
The remainder of the paper is organized as follows: Section 2 lit- where x is an x-dimensional vector with each position representing a
erature review on DSBO, Section 3 research method, Section 4 findings problem variable from the X matrix of restrictions made from the
and discussion, and last Section 5 conclusions and findings. possible values of x . As J cannot be calculated directly, it is an expected
function from the vector x with a random function w that gives the
2. Literature review stochasticity (uncertainty) to the system at each complete run. An es-
timator for the expected value can be obtained by the sample mean.
2.1. Discrete simulation-based optimization: definition of terms used and
research area 1
N
J¯ (x ) L (x ; wj )
N (2)
The term “simulation” refers to a collection of technics to mimic a
j=1

specific behavior from a real or ideal system, using resources (time and
Using the strong law of large numbers and the central limit theorem
knowledge) to answer questions made for the studied structure when
(Fu, 2002), Eq. (2) is a good estimator for the expected value of L (x; w )
real experiments are too costly or impossible to be performed.
with the decrease of sample standard deviation when N . As a
Simulation can be used in a variety of fields, industries, and applica-
result, the better solution is a consequence of a large number of simu-
tions, that mainly consists of data collection, analysis with the help of
lation replications that, depending on the size of the system and the
computers (Banks, Nelson, Carson, & Nicol, 2010; Kelton, Sadowski, &
search space, demand a computer processing power and time that can
Swets, 2010; Law & Kelton, 1991).
be prohibitive.
It is recommended to use simulation when the studied system in-
The overall optimization techniques are developed to find a good
volves variables with stochastic behavior, none or minimal correlation
solution, in a reasonable quantity of time, that can pass through viable
and independent and identically distributed (IID) properties (Bianchi,
or unviable solutions in the search space depending on the method. The
Dorigo, Gambardella, & Gutjahr, 2009). If one of those characteristics
“strength” of the algorithm is measured according to the ability to scape
are not met, the data should be treated, or the decision maker should
local optimums and find a good solution that can be very close to the
consider the use of other types of modeling and optimization techni-
global optimal solution, in a way that the relation of spent time and
ques, such as linear and non-linear optimization.
quality of solution satisfy the expectations of the decision maker.
Another simulation characteristic refers to how the entities change
To solve the problems bounded by the definition of Eq. (2), many
during time. If it only changes at specific points in the system, it is
authors wrote about the subject, such as Banks (1998), Chen, Jia, and
considered discrete (e.g., operations such as cut, weld, paint), in op-
Lee (2013), Dellino and Meloni (2015), Fu (2015) Mujica Mota & Flores
position to variables that change continuously during a period of time
De La Mota (2017), and Pawlewski and Greenwood (2014), discussing
(Rosser, Sommerfeld, & Tincher, 1991). Other types of modeling and
several optimization methods, e.g., heuristics, metaheuristics, gradient-
simulation are based on agents’ behaviors. In these cases, the agents are
based, surrogate models, and others, applied to discrete event simula-
individuals with their behavior and rules, where the modeler can spe-
tion.
cify the condition when the rules will be executed. Agents are con-
Both simulation and optimization can be applied in Industrial
sidered like decision makers with some level of learning and adaptation
Engineering (IE), an area that, according to Maynard & Hodson (2004),
(Collier & North, 2012). For the present SLR, only studies that involve
is concerned to problems related with the production of goods and
discrete-event entities behavior were analyzed, since most of the In-
services, evaluating the effects of design, installation, and improvement
dustrial Engineering problems evolve such kind of problem where en-
of systems that integrate people, materials, and information. Salvendy
tities are transported and modified in a specific way in defined pro-
(2001) states that these problems are associated with technology, per-
cesses.
formance improvement management, management, planning, design,
Simulation projects often aim to answer questions related to the
and control and methods for decision-making. This paper considered
optimization of specific characteristics that represent “what if” sce-
studies that applied discrete event simulation and optimization (i.e.,
narios to the proposed system. Optimization is defined as the mini-
DSBO studies) to problems that involve the production itself, on the
mization or maximization or both related to a one or multi-objective
shop-floor or the related production areas of goods and services, being
function that summarizes, in a mathematical form, the questions made
considered a tool to help the decision making on the Industry 4.0 era
for the system. If so, different combinations of alternatives are con-
(Xu et al., 2016), which demand tools capable of dealing with a large
sidered viable if it satisfies all the restrictions of the problem, or un-
amount of data to transform on information for real-time process con-
viable if at least one restriction is not satisfied. The alternative that has
trol.
the best value for the objective function is considered optimal. If the

527
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

2.2. Literature review on DSBO and correlated themes concerning about vocabulary, theories, key variables, methods, and
history. As a result, the LR can help to avoid problems in all the
During the research, 11 Literature Reviews were found with corre- methodology steps such as problem definition, method selection, data
lated subjects to DSBO, but none comprises a wide range of successfully collection, and analysis, which will lead to research conclusions with
adopted methods that should be considered at the beginning of DSBO less probability to have faults or inquired to be misunderstanding how
on IE projects. This makes this paper, with the use of SLR and CIMO they were obtained.
(explained in Section 3.2.1), a scientific contribution for the area. As stated by Denyer and Tranfield (2009) the Systematic Literature
The earliest LR considered was produced in 1994, presenting mul- Review (SLR) should not be interpreted as a LR, but a research project
tiple-comparison procedures and ranking-and-selection procedures for that at its core use the literature to respond questions, in a way that all
discrete, and gradient-based methods, likelihood ratio method, and the steps are well defined and can be reproduced with minimal bias,
frequency domain experimentation for continuous problems, applied on generating a close result to the original.
(s, S) inventory system and the GI/G/1 queue problems (Fu, 1994a). According to Centre for Reviews and Dissemination (2009) health
Three years later an LR identified six categories and 12 optimization care decisions should be taken with the use of the latest research in-
methods, but it did not associate them with the IE problem type (Carson formation related to the best modern practices with the help of a
& Maria, 1997). The next LR was in 2004 treating only the problems methodology that can unite all the sparse information that exist, that’s a
related to staff scheduling and rostering (Ernst, Jiang, Krishnamoorthy, reason for a large number of SLR presented for this scientific area.
& Sier, 2004). In the year 2009, a survey was conducted based only on Discussing a parallel, the same reasons can be considered to the use of
stochastic combinatorial optimization metaheuristics (Bianchi et al., SLR in Industrial Engineering (IE), with the benefit that, in general, IE
2009). researches do not suffer from the problems related in Hammersley
In 2010 an LR was developed considering only simulation applica- (2001); Learmont & Harding (2006) and Morrell (2008) about the exact
tions, in business and manufacturing, and without considering the intrinsic nature of this science field, that is benefited by SLR char-
origin of the research (Jahangirian, Eldabi, Naseer, Stergioulas, & acteristics to identify, to evaluate, and to summarize the collected data.
Young, 2010). For 2013, two LRs were conducted with the first con- Other modern methodologies such as data mining and machine
sidering the state of the art in Parallel Discrete-Event Simulation (PDES) learning can be performed on a scientific database to search for specific
and the second related nine methods (for continuous and discrete works. The automatic search for words alone demands an initial specific
variables) describing the problem of budget allocation (Jafer, Liu, & knowledge of the desired terms. On a diverse and scattered biblio-
Wainer, 2013; Long-Fei & Le-Yuan, 2013). In the next year, an LR graphy with themes that progress during the time, each author uses
considering the aspects to simulation manufacturing systems design, different ways to present methodology and keywords. The manual
operations and language/package development from 2002 to 2013, search for the present study was chosen, instead of an automatic one,
with the present paper adding the hardware issue for the aforemen- since it provides direct insights on other issues not defined on the start
tioned LR screen method (Negahban & Smith, 2014). The subsequent of the research, which contributes with the development of the research
year other three LR were performed. The first was based in 6 categories: and trends for future works.
ranking and selection, black-box search, meta-model based, gradient-
based methods, sample path, stochastic constraints and multi-objective, 3.2. Application of SLR methodology
explaining each category, with four random examples in total (Xu et al.,
2015). The second studied the DSBO applied to the maintenance pro- Booth, Papaioannou, and Sutton (2012) refer that the word “sys-
blem (Alrabghi & Tiwari, 2015). The third discussed some issues related tematic” implies that the SLR should be performed with the following
to the use of DSBO in transportation (Bierlaire, 2015). Only one paper characteristics: explicit, transparent, methodical, objective, standar-
was found (Oliveira, Lima, & Montevechi, 2016) that used the SLR dized, structured and reproducible. For this reason, the definition of the
methodology relating simulation with supply chain, but not specifically review steps should be done carefully. This paper adopts the steps
the use of optimization techniques. presented in Oliveira et al. (2016), which are planning, searching/
Another six articles used Reviews (R) for correlated DSBO applica- screening, analysis/synthesis, and the presentation of findings.
tions, in a more specific way. In 1994 a paper evaluated discrete and
continuous variables optimization methods applied to two examples 3.2.1. Planning
(Fu, 1994b). In 2009 a study reviewed low-order polynomial regression In the first phase, the goal was to perform a better understanding of
metamodeling (Kleijnen, 2009). In 2013 two papers made reviews on the core issues related to the research itself. An exploratory search was
the subject. The first cited seven approaches and some cases of appli- performed on the Web of Science and Scopus databases. After that, this
cation, without specific criteria for selection (Riley, 2013). The second phase continued with discussions and meetings with the subject of
referenced simulation-based optimization techniques for maintenance DSBO methods. Three professors with experience in DSBO practice and
operations (Alrabghi & Tiwari, 2013). Next year, an R related some theory, Ph.D. and M.Sc. students that research on this same field par-
methods applied for DSBO to build design optimization (Nguyen, ticipated in these events. The meetings defined the search for articles
Reiter, & Rigo, 2014). In 2015 an R talked about metaheuristics applied and conference papers using the terms “Discrete Event Simulation” with
in simulation and stochastic combinatorial problems, and the differ- Boolean logic “AND” with the terms: “Optimization”; “DEA”;
ences about these two fields (Juan, Faulin, Grasman, Rabe, & Figueira, “Metaheuristic”; “Genetic”; “Tabu”; “Design of experiments”;
2015). The last R was developed in 2017 developed and analyzed “Response surface”; “Metamodel”; “Parallel”.
Kriging metamodeling in simulation, with no specific application Fig. 1 represents the early findings for the exploratory screening on
(Kleijnen, 2017). All the aforementioned LR and R composed the base the selected databases, especially regarding the studies of Franceschini,
to create the method for an SLR applied to a theme (DSBO) that is Maisano, and Mastrogiacomo (2014); Jahangirian et al. (2010). It de-
studied over the last 25 years. scribes four primary aspects that researchers or practitioners should
consider for DSBO projects. The design was thought to be both way and
3. Research method iterative, where the researcher or practitioner can start from the desired
problem, choose the best DSBO method that he/her know it. Then, the
3.1. The chose for the SLR methodology software and the hardware are chosen according to the resources
available. The analysis can be both way and iterative because, in the
According to Torraco (2005), a literature review (LR) is a way to the early stage of the DSBO project, the optimization method selection is
researcher demonstrate knowledge about a particular field of study, conditioned to the resources available regarding software license,

528
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

Fig. 1. Different initial levels to consider on DSBO

project.

computer power, and necessary knowledge, demanding time and research.

money for the acquisition and development. Fig. 2 illustrates the three areas that were considered to the data
The idea for the present paper emerged from the observed lack of extraction from the articles. The first category is the nature of the re-
research that joins IE problems and DSBO in a wide range. It can help at search which define the problem category, the industrial sector and if
the initial step for such type of project, considering the state of the art in the project is based on the solution of a real problem or with data ex-
terms of DSBO methodologies. The objectives of this research consist of: tracted from the literature. The second category represents the adopted
methods to perform the DSBO, comprising the optimization method,
(a) Develop an extensive systematic literature review on DSBO appli- the software used, and the results measurement. The last category
cations on IE; considers the available origin information, i.e., author's name, affilia-
(b) Identify and extract the methodologies found in these studies; tion (university or company), country of origin, and journal data (name
(c) Analyze and summarize the methodologies found; and publication year).
(d) Discuss the assumptions according to the results.

To accomplish these objectives, Review Questions (RQ) were for- 3.2.2. Searching/screening
mulated to help in the extraction of valid data from the articles. The To avoid omitting pertinent citations (Franceschini et al., 2014), the
Centre for Reviews and Dissemination (2009) define a frame to shape following 18 databases were selected with their own academic search
the RQ regarding population, interventions, comparators, outcomes, engines: ACM Digital Library; CiteSeerX; dblp Computer Science Bib-
and if the research demands, study design. This method is known as liography; Directory of Open Access Journals (DOAJ); Emerald Insight;
PICO, PICOS or PICOC (Population, Intervention, Comparison, Out- Google Scholar; IEEE Xplore; Microsoft Academic Search; Portal Capes;
comes, Context) acronym, generally used as SLR in medical research. Research Gate; Sage Journals; Scielo; Science Direct; Scopus; Semantic
Other frameworks for medicine were found in Booth, Papaioannou, and Scholar; Springer Link; Web of Science, and Wiley Online Library. The
Sutton (2012) such as SPIDER (Sample, Phenomenon of interest, De- databases were consulted at the same alphabetic order presented.
sign, Evaluation, Research type) (Methley, Campbell, Chew-Graham, After the discussion for the exploratory search, the keywords
McNally, & Cheraghi-Sohi, 2014). According to Denyer, Tranfield, and “Discrete Event Simulation” was selected with boolean logic (AND)
van Aken (2008), the data for organization and management studies with: “Optimization”, “Metaheuristic”; “Genetic”; “VNS”; “GRASP”;
(including IE) is fragmented and need a specific framework. In that “Tabu”; “Particle swarm”; “Ant colony”; “Design of experiments”;
context, the CIMO-logic is proposed to involve the problem Context that “Response surface”; “Factoria”l; “Metamodel”; “Model reduction”;
needs a specific Intervention and use a Mechanism to generate Out- “Parallel”; and “GPU”, performing 15 searches for each of the 18 pre-
comes. Other studies (e.g., Costa, Soares, & De Sousa, 2016; Krause & sented academic search engines, generating a total of 270 searches. The
Schutte, 2016; Pilbeam, Alvarez, & Wilson, 2012; Rajwani & Liedong, search engine results are sorted by the criteria of relevance, meaning
2015; Tanskanen et al., 2017) used the CIMO-logic to express the re- that the result list presents at the first positions articles with all the
search questions. For the purpose of the present paper, the RQ used defined terms, and so on.
were defined by the CIMO-logic question that is divided into four ele- The study selection was conducted with an initial screening of titles
ments: and abstracts regarding two requirements. First, if the article or con-
ference paper (i) seems to be relevant; (ii) uses the English language;

• Context: Which real or theoretical Industrial Engineering pro- (iii) was peer-reviewed; (iv) was published in the last 25 years; and (v)
has the full paper available. Second, a relevance criterion, i.e., the
blems…
• Intervention: …use an optimization algorithm… screening’s stop criterion for each of the selected keyword combination

• Mechanism: …combined with discrete event simulation… (more than 200 K in estimative). These screenings were stopped when

• Outcomes: …to find the best solution in terms of quality defined in the position of the last selected article (according to the first require-
ment) was at least 20 positions above the current article under
the prior objectives and project resources.
screening. Table 1 summarizes the number of downloaded articles.
The Research Questions (RQs) were defined in order to describe Table 1 presents the distribution of the total 663 (271 + 392, ex-
characteristics of each of the dimensions explored by the CIMO-logic, plained on Fig. 3) articles downloaded from 12 databases/search en-
with the “Mechanism” of discrete event simulation been a prerequisite: gines proposed, showing the result ranking the most probable databases
to find articles and conference papers related to DSBO. The result

• RQ1: Which are the main problems studied, related to the area of presented is partly influenced by the fact that the consulting on the
databases was made on the same order presented in Table 1 and the
Industrial Engineering (Context)?
• RQ2: Which optimization methods and implementation software repeated names were discharged, what means that the last names
cannot be considered worse, for example, the Scopus can be better than
were the most used (Intervention)?
• RQ3: How the results were measured (Outcomes)? ACM because had 17 different than the first. The databases CiteSeerX,

• RQ4: Which author, university, publication year, and journal were Google Scholar, Microsoft Academic Search, Portal Capes, Research
Gate, and Wiley Online Library does not return significant results to the
found that compose the reference research centers (Context)?
proposed research keyword combinations. Fig. 3 lists the process for the
In order to answer the RQs, Fig. 2 summarizes the data collected in selected articles.
the papers to couple with the objectives, forming the pillars for the Fig. 3 resumes the process of search and screening with the pro-
posed keyword combination and selected databases. After using the

529
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

Pillars of the Research / Data gathered for DSBO

Nature of the research Adopted method Origin of the research

Author’s name
Defined problem Optimization method
Author’s nationality
Industrial sector Adopted software Author’s filiation

Journal’s name
Real/Theoretical Results measurement
Journal’s year
Fig. 2. Pillar of the research and the data collected in the papers.

Table 1 software development, biological systems, and urban traffic (e.g.

Number of articles downloaded from the databases. Montagna, Viroli, & Roli, 2015; Muta, Raymond, Hara, & Morimura,
Database/search engines Downloaded
2015), but this depends on the application for such areas, for example,
maintenance area that can perform preventive, predictive, and correc-
ACM Digital Library 168 tive studies in which an Industrial Engineer will have little knowledge
Web of Science 130 to work, but related to maintenance workforce timetabling, an In-
IEEE Xplore 125
Sage Journals 73
dustrial Engineer can perform a DSBO with collected data from the real
dblp Computer Science Bibliography 39 case.
Semantic Scholar 31
Emerald Insight 28
Directory of Open Access Journals 21
3.2.3. Analysis/synthesis
Science Direct 18 The synthesis of the findings was performed using a Microsoft
Scopus 17 Excel® spreadsheet to compile all the information extracted from the
Scielo 9 271 articles. In order to answer the RQs, each selected article was
Springer Link 4
evaluated in the concern of 10 items: problem; case study; optimization
Total 663 method, implementation software; results measurement; author’s name;
publication’s year; publication’s name; author’s affiliation and nation-
ality, resuming the proposition illustrated in Fig. 2. After the data ex-
stop criteria and the screening, 793 articles were downloaded and 522 traction, similar terms were identified and consolidated for a better
were excluded for one or more of the following reasons: 62 (15.86%) synthesis, for example, metamodel and metamodeling, or design of
were theoretical studies, 202 (51.66%) do not present an IE problem, experiments and factorial design.
77 (19.69%) do not adopt discrete event simulation or use agent-based After the synthesis of data according to the pillars presented in
approaches, 86 (21.99%) do not use a valid optimization method, 42 Fig. 2, Excel® was used for descriptive statistics to determine the per-
(10.74%) use only deterministic variables, and two (0.51%) were not centage of appearance for each type of problem, method, and research
written in the English language. Moreover, 130 duplicates were ex- origin. This data synthesis was the base for the SLR analysis that con-
cluded. To identify these repeated files, the 793 downloaded articles sisted of presenting the findings and best practices for the development
were put together in the same folder where each file was labeled with of DSBO on IE projects.
the article’s name and arranged in alphabetical order. This approach
was useful as each searched database has its way to label the files,
making difficult to identify the repeated ones. When two consecutive 3.2.4. Presentation (Reporting)
files had the same name and archive size, they were evaluated for The 271 articles were divided into two categories: journals and
possible duplicity. proceedings of international conferences, comprising respectively 150
Theoretical studies, excluded from the review, comprised articles and 121 documents. To present the findings after the analysis and
that only discuss a specific part of the simulation process (e.g., synthesis process, Tables 2–5 and Figs. 4 and 5 summarize the data. The
Adegoke, Togo, & Traore, 2013; Das, 2000; Thomas, Howes, & Luk, results were presented on a sequence to answer the RQs, developing a
2009) and do not present a case study or action research. Articles that discussion by the authors to present the state-of-the-art techniques
did not present an IE problem comprised studies in areas such as applied to DSBO projects on IE problems, showing the past and present
practices and bringing up possibilities to the future of DSBO on IE.

Fig. 3. The search/screening process.

530
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

Table 2 Table 4
Type and economic production sector of the problems. Software used for DSBO.
Total and % of the total Total and % of the total

Problem type Papers Proceedings Total Cum. % Software Papers Proceedings Total Cum. %

Scheduling 58–21.1% 42–15.3% 100–36.4% 36.4 Arena® 34–07.9% 17–04.0% 51–11.9% 11.9%
Industrial Process 54–19.6% 33–12.0% 87–31.6% 68.0 Not specified 29–06.7% 16–08.6% 45–10.5% 22.3%
Logistics 24–08.7% 22–08.0% 46–16.7% 87.4 Matlab® 29–06.7% 10–02.3% 39–09.1% 31.4%
Inventory Control 12–04.4% 14–05.1% 26–09.5% 94.2 C++ 16–03.7% 16–03.7% 32–07.4% 28.4%
Not Specified 03–01.1% 13–04.7% 16–05.8% 100.0 VBA® 16–03.7% 05–01.2% 21–04.9% 38.8%
275–100% Excell® 11–02.6% 12–02.8% 23–05.3% 43.7%
OptQuest® 9–02.1% 9–02.1% 18–04.2% 49.1%
Economic sector
Java 6–01.4% 7–01.6% 13–03.0% 53.3%
Primary 01–00.4% 01–00.4% 02–00.7% 00.7
IBM CPLEX® 8–01.9% 2–00.5% 10–02.3% 56.6%
Secondary 89–32.4% 71–25.8% 160–58.2% 58.9
Promodel® 7–01.6% 3–00.7% 10–02.3% 58.6%
Tertiary 47–17.1% 36–13.1% 83–30.2% 89.1
Others 73–17.0% 95–22.1% 168–39.1% 100%
Not Specified 13–04.7% 17–06.2% 30–10.9% 100.0
238–55.3% 192–44.7% 430–100%
275–100%
DES/Optimization/Communication
Production sector
Programming language 94–21.9% 75–17.4% 169–39.3% 39.3%
Semiconductor 10–03.6% 13–04.7% 23–08.4% 08.4
Modeler and optimizer 85–19.8% 70–16.3% 155–36.0% 75.3%
Health care 13–04.7% 10–03.6% 23–08.4% 16.8
Commercial DES modeler 59–13.7% 47–10.9% 106–24.7% 100%
Automotive 10–03.6% 08–02.9% 18–06.5% 23.3
238–55.3% 192–44.7% 430–100%
Chemical 04–01.5% 01–00.4% 05–01.8% 25.1
Others 16–05.8% 13–04.7% 29–10.5% 35.6 Result measurement
Not specified 97–35.3% 80–29.1% 177–64.4% 100.0 Min Cost 96–22.8% 85–20.2% 181–43.0% 43.0%
275–100% Max throughput 62–14.7% 63–15.0% 125–29.7% 72.7%
Speedup 24–05.7% 18–04.3% 42–10.0% 82.7%
Data origin
Benchmark 14–03.3% 8–01.9% 22–05.2% 87.9%
Real 84–30.5% 66–24.0% 150–54.5% 54.5%
Others 30–07.1% 21–05.0% 51–12.1% 100%
Theoretical 67–24.4% 58–21.1% 125–45.5% 100%
226–53.7% 195–46.3% 421–100%
275–100%

to generate the basis for the discussion.

Table 3
DSBO methods used.
4.1. Nature of the research:
Total and % of the total

Optimization method Papers Proceedings Total Cum. % This section presents the findings related to the first pillar, i.e., the
“nature of the research”, which is related to the information of the
Heuristics problem itself, the industrial sector and the definition if the project is
Local Search 7–11.7% 3–05.0% 10–16.7% 18.2%
based on the solution of a real problem or with data extracted from the
Random Search 5–08.3% 2–03.3% 7–11.7% 30.9%
Hill Climbing 3–05.0% 2–03.3% 5–08.3% 40.0%
literature. Table 2 summarizes the findings to answer the RQ1.
Others 24–40.0% 14–23.3% 38–63.3% 100% In problem type, five categories were generated according to the
39–65.0% 21–35.0% 60–100% significant number of articles related (at least 20): industrial process,
Metaheuristics inventory control, scheduling, logistics, and not specified. Industrial
Evolutionary 60–20.7% 65–22.4% 125–43.1% 43.1% processes comprise problems that occur mainly on the shop floor of an
Simulated Annealing 11–03.8% 12–04.1% 23–07.9% 51.0% industry, where part of the value is generated. Such problems are re-
Tabu Search 14–04.8% 6–02.1% 20–06.9% 57.9%
lated to parameter production definition (e.g., Al-Aomar & Al-Okaily,
VNS 5–01.7% 1–00.3% 6–02.1% 60.0%
Others 87–30.0% 29–10.0% 116–40.0% 100% 2006; Can & Heavey, 2011; Choi, Seo, & Kim, 2014; Creighton &
177–61.0% 113–39.0% 290–100.0% Nahavandi, 2003), buffer (e.g., Amiri & Mohtashami, 2012; Costa,
Surrogate model
Alfieri, Matta, & Fichera, 2015), inspection (e.g., Van Volsem, Dullaert,
DOE 18–16.4% 16–14.5% 34–30.9% 30.9% & Van Landeghem, 2007), and resource allocation (e.g., Lucidi,
Response surface 10–09.1% 9–08.2% 19–17.3% 48.2% Maurici, Paulon, Rinaldi, & Roma, 2016). The sum of the papers is
ANN 9–08.2% 3–02.7% 12–10.9% 59.1% higher than the original quantity (275 > 270) because articles pre-
Kriging 7–06.4% 2–01.8% 9–08.2% 67.3%
sented more than one problem related to IE (e.g., Can & Heavey, 2012).
DEA 5–04.5% 0–00.0% 5–04.5% 71.8%
Regression 4–03.6% 1–00.9% 5–04.5% 76.4% The other types of problems are related to production support systems,
Others 10–09.1% 16–14.5% 26–23.6% 100.0% mainly with respect to inventory control (e.g. Kilmer, Smith, & Shuman,
64–57.3% 42–42.7% 110–100% 1999) for stock level and replenishment; scheduling associated to pro-
Parallel/distributed gramming job shop and dispatching (e.g. Costa, 2015; Jia, Bard,
CPU 6–21.4% 17–60.7% 23–82.1% 82.1% Chacon, & Stuber, 2015; Martins, Fuchs, Pando, Lüders, & Delgado,
GPU 3–10.7% 2–07.1% 5–17.9% 100% 2013; Naderi, Khalili, & Tavakkoli-Moghaddam, 2009; Neto &
9–32.1% 18–64.3% 28–100%
Goncalves, 2010; Saadouli, Jerbi, Dammak, Masmoudi, & Bouaziz,
Proprietary 11–52.4% 10–47.6% 21–100% 100%
Monte Carlo 4–57.1% 3–42.9% 7–100% 100% 2015); logistics (e.g. Baril, Gascon, & Cartier, 2014; Zhen, Wang, Hu, &
Gradient based 6–75.0% 2–25.0% 8–100% 100% Chang, 2014) for allocation, location, layout, supply, and routing pro-
Others 21–44.7% 26–55.3% 47–100% 100% blems; or not specified studies (e.g. Kilmer et al., 1999) which cannot be
determined in one of the previews categories.
The four primary problems namely scheduling, industrial process,
4. Findings and discussion logistics, and inventory control represent 94.2% of the total problems.
This is a sign that these areas have IID aleatory variables that constitute
In order to present the findings to answer the research questions, the a search space which demand a sophisticated methodology, such as
data gathered for the pillars of the research are presented in this section DSBO, to help in the decision process to find the best solution. The

531
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

Table 5 Other
Ireland USA
Number of publications according to the author's name, affiliation, and na- 16%
2% 24%
tionality. Italy
TOP 10 Name - Number of publications 4%
France
Researcher Papers Proceedings
4%
1 Jack P.C. Kleijnen − 4 Amos H.C. Ng − 3 UK Germany
2 A. Azadeh − 3 Hongwei Ding − 3 5%
3 B. Naderi − 3 Lars Mönch − 3
11%
Iran
4 Berna Dengiz − 3 Lyes Benyoucef − 3
5 Christian Almeder − 3 Torsten Hildebrandt − 3
7%
6 Feng Yang − 3 Xiaolan Xie − 3 Brazil China
7 Richard F. Hartl − 3 Alexander Pacholik − 2 7% Canada 11%
8 Wim C.M. van Beers − 3 Alexandre Ferreira de Pinho − 2 8%
9 Wout Dullaert − 3 Andrés Muñoz-Villamizar − 2
10 A. Costa − 2 Anna Persson − 2 Fig. 5. Articles and proceedings according to the top 10 countries for DSBO.
Total − 150 Total − 121

Affiliation present SLR does not try to search for other methodologies to evaluate
1 Amirkabir University of Dresden University of such problems but is possible to infer that in these cases DSBO is a
Technology − 7 Technology − 4
viable tool to be considered and used. The size of the presented pro-
2 University of Tehran − 6 Purdue University − 4
3 University of Vienna − 5 University of Paderborn − 4 blems was in general composed by few stations and/or buffers (1–5),
4 Louisiana State University − 4 University of Skövde − 4 representing only part of the total production systems, showing that
5 Tilburg University − 4 Ilmenau Technical University − DSBO was planned to resolve part of a specific problem, and not to
3 evaluate the role system. Even this problem size has a search space that
6 University of Antwerp − 4 Nanyang Technological
justifies the use of optimization methods.
University − 3
7 Baskent University − 3 Northeastern University − 3 For the economic sector, the four categories were: primary, related
8 Ghent University − 3 Tongji University − 3 to the production or exploitation of natural resources; secondary, that is
9 Islamic Azad University − 3 University of Hagen − 3 responsible for the transformation of natural resources in goods; ter-
10 Shanghai Jiao Tong University Durham University − 2
tiary, associated to the provision of services; and not specified, when
−3
Total − 150 Total − 121 the object of study cannot be detailed in one of the other categories. As
expected, the same production system can have either a perspective to
Nationality
1 United States − 33 United States − 32
supply the customer with goods and services. In this case, only the
2 Iran − 18 Germany − 22 primary purpose of the problem was considered. In this context, a job
3 China − 14 China − 17 shop scheduling and supply chain problem related to the automotive
4 Germany- 9 France − 7 industry would be associated with the secondary economic sector, if the
5 Brazil − 8 UK − 7
main objective is the production and provision of a good.
6 Belgium − 6 Italy − 5
7 Canada − 6 Sweden − 5 Only two papers were identified in the primary sector
8 United Kingdom − 6 Colombia − 4 (Nageshwaraniyer, Son, & Dessureault, 2013a, 2013b), related to a coal
9 France − 5 Brazil − 4 mine. These studies were performed by the same authors, two from the
10 Italy − 5 Ireland − 3 Industrial Engineering and the other one from the Geological En-
Total − 150 Total − 121
gineering. This may be explained by the fact that DSBO problems are
mainly studied on specific engineering courses such as Industrial, Me-
chanical, Electrical, and Computer. Other courses like Agronomy,
Zootechny and Mining Engineering have a different focus and can be

Fig. 4. DSBO on IE summary.

532
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

dependent on the knowledge from the four courses presented before. the studied problem.
This suggests that the primary sector can be explored by future studies Related to the metaheuristics, 43.1% are derived from evolutionary
with the collaboration between the presented courses. The secondary algorithms with different denominations, for example, differential
and tertiary sector comprised most of the studies, 58.9%, and 30.2%, evolution, chaotic differential evolution, genetic algorithm, evolu-
respectively. This unbalance between the secondary and tertiary can be tionary algorithm, and NSGA II. These are mainly population search
an evidence that the production of goods is more suitable to be modeled methods with some modification of the genetic algorithm concepts for
by DES methods, due to the fact that the production of a service is more the individual, gene, population, crossover, etc. The sum of the second,
dependent to the iteration between customer and provider, that have third and fourth most used metaheuristics (simulated annealing, tabu,
human and cultural characteristics that are difficult to be modeled by and Variable Neighborhood Search – VNS) represent 16.9% of the total,
DES and can be a field to be explored by agent-based simulation compared to the 43.1% of the evolutionary. From all the 554 DSBO
methodology (Dorigatti, Guarnaschelli, Chiotti, & Salomone, 2016). methods applied, the 290 metaheuristics represent 52.3%, and the 125
The third aspect presented by Table 2 is the production sector. evolutionary represent 22.6%. The line “other” represents 116 methods
Problems related with the semiconductor, healthcare, automotive, and (40.0% of the metaheuristics) combining methods such as artificial
chemical industries represented 25.1% of the total. These industries are immune algorithm, scatter search, GRASP, particle swarm, and ant
related to the secondary and tertiary economic sectors and commonly colony.
produce goods and services with a high level of aggregate value, The surrogate model represents the second most used DSBO
compared to the primary sector industries. The category others refer to method, corresponding to 110 methods (19.9% of the total). The most
a sparse variety of industries that represent 10.5% of the total, and for used surrogate method was the Design of Experiments (DOE) (30.9% of
64.4% of the articles, it was not possible to specify the production the surrogate methods) followed by response surface (17.3%). The DOE
sector. The fact that the article does not specify the related production method itself provides, in general, a non-linear regression that cannot
sector may be explained by the frequent discretion and confidentiality provide a good solution alone, but together with other optimization
adopted to hide the company problem or strategic information. In this methods such response surface. By this means, all the surrogate
sense, it is recommended that future works specify the object that methods use in some phase a metamodel to be optimized, and in some
generated the problem, helping practitioners and researchers to find cases, the articles make explicit or not the use of them. The line DOE
already implemented solutions for similar problems, and to choose refers to all DOE methods found such as hypercube, full factorial,
suitable methods. This issue was seen both on journal and proceeding Taguchi, robust design, and LHD. The regression is related to more
papers. specific methods, for example, time-series, and the Kriging methods
The last information presented in Table 2 is the origin of the data comprise, for example, kriging metamodeling, detrended kriging, and
used on the examples presented in the articles. Theoretical articles studentized.
comprise papers that used data from other studies, mainly not devel- For DSBO, the standard framework presented on the papers is to use
oped by the same authors, or used classical problems presented in books a computer that uses a single instruction sequential algorithm in the
or specialized literature. In this matter, it is shown a relative balance form of proprietary software or algorithm to generate the DES model
between the number of publications in both directions, on paper and and evaluate the search space. On 28 studies (4.8% of the total), the
proceedings. This may indicate that the development of DSBO has been implementation of specific problems used single machines and parallel
made both on the theoretical and practical ways. instructions or multiple machines with distributed instructions. In that
According to the data presented in Table 2 and the corresponding way, two types of parallelism were found, related to the use of only CPU
discussion, it is possible to answer the RQ1 (“which are the main pro- (Central Processing Unit) on single or distributed machines or hybrid
blems studied, related to the area of Industrial Engineering?”): the in- algorithms that use both the CPU and GPU (Graphics Processing Unit)
dustrial sectors responsible for the productions of goods and services to parallelize the processing of some instruction on the simulation.
with high aggregated value respond for the projects that most invested From the 28 original articles, 23 (82.1%) used the parallelization with
in modeling and searching for the optimized solution of the presented the CPU, and five (17.9%) used the GPU together with the CPU, with
problems, mainly due to the significant costs and benefits related to the earliest publication in 2010 (Park & Fishwick, 2010), which may
scheduling, industrial process, logistics, and inventory control pro- indicate this as a trending topic for DSBO. The parallelization of the
cesses. simulation alone does not characterize as an optimization method. For
that, it is possible to combine other methods (e.g., heuristics and me-
4.2. Adopted method taheuristics) to search for a good solution on a viable period of time
(Costa et al., 2015; Mokhtari & Salmasnia, 2015; Sailer et al., 2013;
In order to search a solution space, looking for the best feasible Uhlig & Rose, 2011).
solution, a variety of optimization methods can be applied to a DES The fifth criteria refer to the use of proprietary optimization pro-
problem. Table 3 summarizes the methods found to optimize the pro- grams, for example, OptQuest® and SimRunner®. The use of proprietary
blems presented in Section 4.1, and adopted by at least five articles. optimization software, by the academic point of view, has limitations
Frequently, articles used more than one optimization method in the that compromise the development and test of new optimization
same article, justifying the total of 571 methods and implementations. methods that can present contributions to the refereed literature. This is
Another factor is that one method/article can use a mix of two different one reason that contributed to the few numbers of papers (21 articles)
modeling types such as integer or binary programming (e.g., Saremi, that utilized proprietary optimization software as the main optimiza-
Jula, Elmekkawy, & Wang, 2013). In those cases, two methods were tion software or a comparison point to relate with other methods.
considered. The methods related to Monte Carlo and Gradient-Based represent
According to Table 3, 40% of the used heuristics are related to 2.6% of the total (15 times used). The category “Other” refers to
Local, Random and Hill Climbing search methods, and the remaining methods that were used less than five times (47 times used, 8.2% of the
60% are related to specific algorithms that do not fit on the first 3 ones, total) for example, model reduction, decision tree, cloning, and fuzzy.
for example multi-start (Lamiri, Grimaud, & Xie, 2009), or do not de- The small number of publications, for the methods presented on the
scribe the adopted heuristic. This fact is partially explained by the Other category, can be a sign for the need of development on these
nature of the heuristic search method that explores a specific problem optimization areas, according to the success of implementation pre-
characteristic. Although it is possible to use a generic method, e.g., the sented on the articles. Table 4 present the software used and DSBO
Hill Climb (Raska & Ulrych, 2015), the application of the heuristics variables considered on the papers.
requires adapting the search algorithm for the specific characteristic of According to Table 4, the criterium “software” presents the

533
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

programs used for DSBO. The most adopted software is the DES measurement. The criteria “Other” is related to statistical measures
modeler Arena®, used on 51 articles (11.9% from the total). The second between results such as Mean Absolute Deviation (MAD) and Mean
category “Not specified” represent the articles that used computer Absolute Error (MAE).
programing language but does not specify which one, cited on 45 ar- It was observed that the results measurement was made to correlate
ticles (10.5%). The third most cited software is the Matlab® that could at least one maximization of throughput and one minimization of cost
be either used for modeling or optimization, on 39 articles (9.1%). The criteria on the objectives and/or restrictions. This is expected to the
three most used software bring on sight the problem related to all DSBO formulation of the optimization problem with a finite and defined space
studies that are the generation of the DES computer model and the solution, with inverse related variables that at some point have a region
recursive call for the evaluation of the results by an optimization that maximizes or minimize the problem response. It was not clear if
method and the parameters to call the new scenarios. the best result presented was implemented on the real systems nor how
Thinking on this question, the second criterium “DES/ far it was from the probable global optimum. According to the authors,
Optimization/Communication” separate on three categories according part of this issue can be explained by the fact that the stochastic nature
to the purpose of the software. The first “Programming language” join of the DES variables makes challenging to guarantee that a good solu-
all the articles that cited, directly or not directly, the used programs or tion found can be used and generate a similar result than the simulated
computer programing languages that demand programming skills such one. This mater can be more explored in future works.
as C++, C#, Java, Cplex, VBA, and CUDA, generating a total of 169 Considering the RQ3 (“how the results were measured?”), it is
(39.3%) programs. The second category “Modeler and optimizer” is possible to infer that the measurement of the DSBO projects is related to
related to the cited software that can be used for both model and op- the initial purpose which stimulates the development of the same. For a
timization, for example, Matlab®, representing 36.0% of the mentioned general DSBO project on IE, the reasons were related to the need for
software. The last category refers for the commercial DES software used evaluation of multiple scenarios that influence the way the organization
for modeling only, for example, Arena®, ProModel®, Witness®, works and the revenue, when a manual simulation is prohibitive.
ExtendSim®, AnyLogic®, and Enterprise Dinamics®, referring to 106 Therefore, the measurement considers how good a solution is and the
cited software (24.7%). time needed for the optimization get on it.
Analyzing the two first criteria on Table 4, it is possible to infer that According to Chwif, Paul, and Barretto (2006), optimization
there is no consent on the academic and practitioner communities to methods and procedures applied to DES can be classified in four cate-
define a framework capable of joining the modeling phase and opti- gories: gradient-based search, stochastic approximations, response
mization on DSBO projects. Considering that it is not a common prac- surface methodology, and heuristic search methods. This categorization
tice to use more than one DES software, it can be said that at least 106 was made in 1999 and was not the purpose of the article to make a
articles (39.1%) from the 271 analyzed, used a commercial program to more precise definition of DSBO methods. For Juan et al. (2015), it
model the DES, and the majority of the studies in some part used a considered the ranking and selection, black-box search methods, meta-
programing language, except the four articles that used commercial model, gradient-based methods, sample path, and stochastic constraints
optimization programs only (e.g., OptQuest® and SimRunner®). The and multi-objective. Fig. 4 summarizes the findings related to the
scatter variety for possible combinations of DSBO methodologies and methodologies applied to DSBO on IE problems, that extend the pro-
adopted “test beds” make challenging to replicate the studies or to posed classification of the aforementioned authors.
compare the results of an optimization method. For the development of Fig. 4 illustrates the best practices for DSBO on IE found in the
a DSBO framework capable of creating a DES model and an open en- present study to help on the planning for the steps of a DSBO project on
vironment to test different search optimizations, the works of Freitag IE, based on the 271 articles. The first step is to define the problem and
and Hildebrandt (2016) and Hildebrandt, Goswami, & Freitag (2015) the questions that will be answered by the DSBO. If the search space or
can be cited. the problem can be limited in a small number of scenarios, intuitive
According to the data presented in Table 4 and the corresponding methods can be used to determine the number of manual changes
discussion, it is possible to answer the RQ2 (“which optimization and needed on the simulation model, escaping from the scope of the present
implementation software methods were the most used?”: in the com- work, that evaluated problems that have a search space that is prohi-
puter modeling phase, in almost half the cases, a commercial DES bitive to be made manually, only with the help of a computer optimi-
software was used, and for the optimization, a programing language zation method. After the evaluation of the problem and the need for an
was adopted, implementing in most of the cases a metaheuristic or automatic search method, it is possible to evaluate the software re-
surrogate model analysis. quired for the computer model and optimization.
The last category in Table 4 is related to the key performance in- The second step constitutes to evaluate the available resources of
dicators that constituted the variables for the objective function and the time and knowledge to spent on the construction and finding the op-
constraints that limit the search space, separated on four criteria. The timal solution. This constitutes an essential issue because more complex
benchmark studies are related to theoretical papers with the study case simulation systems involving distributed/parallel simulation demand
that used data from the literature (books and specialized articles) or more time and equipment investment that are not an assurance to find a
compared the author results with commercial optimization programs, better solution than sequential programming but increases the prob-
responsible for 22 variables, 5.2% of the total studies. The “Max- ability. The last step is to determine the optimization method(s) applied
imization of throughput” comprise the objectives related to process to the simulation model. The selection of the method can be related to
times that influence on the production parameters, for example, queue the findings of previous works or by the implementation and compar-
length, production rate, flow time, wait time, lead time, product ison of different methods, according to the available resources. At this
throughput, and makespan, corresponding for 125 variables (29.7%). point, there is not a consensus for what method is more suitable to solve
The “Minimization of cost” relate the variables that have direct influ- the problem, depending on the previous knowledge and experience of
ence on production costs such as payroll, revenue, performance, effi- the simulation team.
ciency, net profit, capital, resources, cost savings, WIP, lot size, stock
level, buffer size, batch size, and customer demand, showing on 181 4.3. Origin of the research
variables, 43.0% of the total. “Speedup” is related to the time or
number of iterations needed to find good solutions, present on 42 The origin of the research summarizes the data collected on the 271
variables, 10.0% of the total. As it is not common to have more than one articles that refer to the context of the authors and where they produced
speedup variable on the same articles, it is possible to infer that 42 the articles related to DSBO on IE, to help in the answer of RQ 4. It is an
works, 15.5% from the 271 papers, used the speed up as a interest in the present study to find if there is a correlation between the

534
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

authors and their different nationality to construct a research commu- applications of DSBO on IE. This could be partially explained by the
nity in the area of DSBO. Table 5 present the findings. evolution of the Queue Theory and accessibility to computers and DES
From the total of 789 names found on all articles, 126 appear more software. From 2014 to 2016, a decline of 48.5% on the publication is
than one time, and a total of 663 names without repetition were found. noticed, but the reasons are not clear. Moreover, in the same period, the
In the top 10 ranking for the authors, the difference from the most agent-based modeling gained strength and researchers and practitioners
productive author to the others is no more than two articles, which may have concentrated efforts to study hybrid systems or different
shows a panorama that the authors do not have a continuous produc- kinds of simulation that can be characterized as an increase or main-
tion on the field of DSBO applied to IE problems, with a sparse domain tenance of the research produced volume in the area of simulation as a
of this knowledge. This effect demands a search for different authors in whole. Nevertheless, a more extended period and more data are ne-
order to find more details on a specific optimization methodology or cessary for conclusive evidence.
related problem. Finishing with the RQ4 (“which author, university, publication year
The second information on Table 5 is related to the affiliation of the and journal were found that compose the reference research centers?”),
authors. With a similar understanding from the previous conclusion, the it is not possible to infer with the information on Tables 5–7 and Fig. 6
Universities where the studies were conducted does not show a sig- that exist a reference research center which concentrates the publica-
nificant number of publications that justify a cluster or reference re- tion on DSBO on IE. Furthermore, the collected data indicates a scarce
search center related exclusively to DSBO on IE. The last information on orientation to identify a location for research some specific type of
Table 5 join the articles according to the author’s country affiliation. At optimization method and IE problems relating to discrete event simu-
this point is possible to make data segregation, according to Fig. 5. lation.
According to the graphic presented on Fig. 5, is possible to elect the
USA, Germany, China, Canada, Brazil, Iran, UK, France, Italy, and 4.4. Future research directions on DSBO
Ireland, the top 10 countries that concentrate the most productive au-
thors, but with the second information conclusion, even in these According to the papers and proceedings evaluated in the present
countries, the development of DSBO on IE does not have a reference study, three major optimization techniques showed vast improvement
center. Another relevant information extracted from the articles is how and still in development. The first two are related to the use of meta-
the optimization projects have been developed, with the collaboration heuristics and machine learning optimization algorithms, and the last
of multiple research centers from different universities, private sector method with hardware parallelization.
institutions, and countries. Table 6 present the data collected for the To search in an NP-hard problem with a scatter solution space,
author's team. metaheuristics can perform well in terms of finding local or near global
The data on Table 6 is related to the different origins and config- optimum solutions in a reasonable wall clock time. If the discrete event
urations for the author's team of each article. For the first information is simulation is needed, surrogate models (or metamodels) methods can
possible to infer that in 29.2% of the works, members of two or more be used instead of the real simulation. The metamodels are, in general,
universities were involved. The second information is related to the a mathematical representation that gives a result similar to the real
involvement of private and public institution members, different from simulation, in an amount of time less than the needed to run the si-
an academic organization, responsible for 19.9%. This issue signs the mulation. For example, it is possible to cite the use of the Decision Trees
interaction between academics and the industrial sector or govern- machine learning method and the metaheuristic Tabu Search for dis-
mental institutions. The third data correlates authors that have in- patching rules (Shahzad & Mebarki, 2016).
stitutional affiliation in different countries, comprising 12.2% of the The idea of machine learning and metaheuristic is found in more
works. For the remaining of the publications, 38.7%, the authors are recent papers, using Artificial Immune Systems and Genetic Algorithm
from the same academic institution. in a material handling system (Leung & Lau, 2018). The NSGA-II and
The 271 articles were published by 62 journals and 45 conferences. SPEA2 evolutionary algorithms were used to select optimal Information
Table 7 related the top 10 publication journals and proceedings found Technology Infrastructure Library (Ruiz, Moreno, Dorronsoro, &
for DSBO on IE. Rodriguez, 2018), and the use of decomposition-based multi-objective
According to Table 7, the first five journals represent 62 published differential evolution algorithm (MODE/D) compared against NSGA-II
articles for DSBO on IE, comprising 41.4% of the publications. The for inventory replenishment problem (Avci & Selim, 2018). Integrating
other five listed journals published 25 articles (9.2%). Regarding the with the trends on the era of Industry 4.0, machine learning is a tool
conference proceedings, the Winter Simulation Conference (WSC) alone that has been used for a variety of manufacturing prediction issues
represents 57.0% of all articles, showing that the WSC is a reference for (Diez-Olivan, Del Ser, Galar, & Sierra, 2018).
the researches and practices on DSBO applied to IE. Fig. 6 illustrates the The third method is the use of hardware parallelism. Along with
publishing of DSBO on IE along the 25 years considered for the present algorithm development, modern computers have increased their com-
research. putation power, in particular, the capacity of processing more than one
According to Fig. 6, it is possible to infer that from 1991 to 2008 instruction at a time with the advent of multi-core processors, in which
there was a trend of growth and mature of the concepts and discrete event simulation can be benefited (Jafer et al., 2013). A recent
research used NSGA-II and parallelism on bridge construction projects
Table 6 with the time to find the solution of the problem characteristics (Salimi,
Different authors origin. Mawlana, & Hammad, 2018). Fig. 7 illustrates the progress of the
Total and % of the total publication related to the categories of optimization methods classified
according to Table 3. In recent years, articles that adopt hardware
Filiation of the author Papers Proceedings Total Cum. % parallelism present more consistent participation in the DSBO studies
on IE, also indicating a research direction.
More than one university 64–23.6% 15–05.5% 79–29.2% 29.2%
Together with another public 13–04.8% 41–15.1% 54–19.9% 49.1% The three cited methods metaheuristics, metamodel using machine
or private institution learning, and parallel processing have in common the need to find good
More than one country for 23–08.5% 10–03.7% 33–12.2% 61.3% solutions in the least amount of available time. This issue reflects the
authors crescent need to process a large amount of stochastic data and the
Same origin place for all 49–18.1% 56–20.7% 105–38.7% 100%
benefit from the developments on optimization algorithms and hard-
authors
150–55.4% 121–44.6% 271–100% ware parallelization. According to Fig. 7, in the last 25 years, these
three methods represented 64% of all the produced research on the area

535
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

Table 7
Top 10 publications locations for DSBO on IE.
Rank Journal Publications % Cum%

1 European Journal of Operational Research 16 10.7 10.7

2 Computers & Industrial Engineering 12 8.0 18.7
3 International Journal of Production Economics 12 8.0 26.7
4 Simulation: Transactions of the Society for Modeling and Simulation International 12 8.0 34.7
5 Computers & Operations Research 10 6.7 41.4
6 Simulation Modeling Practice and Theory 7 4.6 46.0
7 International Journal of Advanced Manufacturing Technology 6 4.0 50.0
8 Journal of Manufacturing Systems 6 4.0 54.0
9 Applied Soft Computing 3 2.0 56.0
10 Engineering Applications of Artificial Intelligence 3 2.0 58.0
Others 63 42.0 100.0

Rank Proceedings
1 Winter Simulation Conference 69 57.0 57.0
2 International Conference on Automation Science and Engineering 3 2.5 59.5
3 Conference on Manufacturing Modeling, Management and Control 2 1.7 61.2
4 European Conference on Modeling and Simulation 2 1.7 62.8
5 European Symposium on Computer Aided Process Engineering and 9th International Symposium on Process Systems Engineering 2 1.7 64.5
6 International Conference on Industrial Engineering and Engineering Management 2 1.7 66.1
7 International Conference on Service Systems and Service Management 2 1.7 67.8
8 International ICST Conference on Simulation Tools and Techniques 2 1.7 69.4
9 SIGSIM-PADS 2 1.7 71.1
10 World Congress on Intelligent Control and Automation 2 1.7 72.7
Others 33 27.3 100.0

of DSBO applied to IE, and, considering the last five years (2012–2016), area where simulation projects have been developed.
the methods were used on 68% of the studies. Other related issues for
future works are the trends for problem types. Fig. 8 relates the topics 5. Conclusions and findings
presented in Table 2 for the considered period of time.
Observing the information on Fig. 8, the problem types are 68% The purpose of the present study was never to cover all the existing
related to scheduling and industrial process, composing the majority of articles about the theme, but to analyze a significant sample size to give
the studied problems. Evaluating the development through the years, insights about the past and present practices about DSBO on IE, helping
the presented problems have a historic mark between the years 2000 researchers and practitioners with the presentation of the already ex-
and 2002, beginning an exponential growth. The development can be isting projects for future ones. Given the proposed SLR structure and
explained by the ease of computer power access generated on these presented methodology, the coverage of the literature about DSBO on
years, related to the acquisition and development of computer hard- IE was considered enough to answer the RQs with a satisfactory un-
ware and programs for simulation. After the year 2000, the use of DSBO derstanding of the subject. It is worth the recommendation for future
on IE have a consist increase and can be considered an established researchers and practitioners, especially those who seek to enter a Ph.D.
practice on the problem types presented in Fig. 8. program, to understand and adapt the SLR methodology to evaluate
The less researched optimization methods and areas, presented in new topics and trends on the corresponding knowledge, especially
Figs. 7 and 8, are not characterized as inapt but with a few numbers of proven by the present work, on the field of Industrial Engineering.
studies. The aptitude can be an issue to be evaluated in future studies, Thinking on the seek to answer the RQs, the first conclusion is that
aiming to determine which methods and areas are best suited to be used the problems related to the production of goods and services studied on
with DSBO projects on IE. For example, healthcare is a growing study DSBO are related to the optimization of resource mainly related to

35

30
11
13
PUBLICATIONS/YEAR

25

9 9
20

15 8 5
6
10 6 22
7 19 17
5 9 9 16
4 6 11 12
8 10
5
2 2 5
1 2 4 4 6 5 4 2
0 1 1 3 3 1
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016

YEAR
Papers Procedings
Fig. 6. Publications of DSBO on IE.

536
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

80
70
60
50
40
30
20
10
0

2012
2013
2014
2015
2008
2009
2010
2011
2002
2003
2004
2005
2006
2007
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001

2016
Metaheuristic Surrogate model Response surface Others
Heuristic Parallel Interger program Gradient based
Fig. 7. Progress of DSBO methods publication.

40

35

30

25

20

15

10

Scheduling Industrial process Logistic Inventory control Not specified

Fig. 8. Problem types published from 1991 to 2016.

operators, machine and resources, which represent the majority of the identification and selection of possible knowledge necessary to
production costs and are always in the aim of the administrators to be achieve a goal and the application of other resources, with specific
changed and optimized. The second conclusion refers to the optimiza- results for each project step. That is a literature deficiency that can
tion method and implementation software, which is an area with no be considered for future works. Even at the papers that show a DES
consensus because the same problem can be solved by more than one methodology, it was common that some part of the project was not
way, and it was not seen a framework that could be applied to all described and discussed (e.g., verification and validation). In the
problems without a good prior knowledge of the researcher or practi- analyzed articles, it was not found the time dedicated to the project
tioner, being one point to be considered on the development of the nor part of it, or the number of people necessary and in which ac-
project. For the optimization of resources, the time spent to generate tivities they worked;
the solution and the cost of initial and final obtained solutions were • It was common to focus on the optimization methods and a way to
compared, and in general, the results are presented as suitable. The last compare the results. Although this is beneficial information, for
conclusion presents the authors and locations where the DSBO on IE future researches or practitioners that will read the article, it is
were developed. There is no cluster or important research center that is valuable to indicate why other methods were not considered and the
a reference on this area, but the amount of data shows a sparse use on mistakes made during this phase. That is a way to avoid the same
46 countries all over the world with good results. mistakes or to make clearer the science development steps per-
Regarding the answer of the RQs, some issues were observed on the formed;
related works that worth mention to direct future studies: • The pulverized register of the works produced on the field of DSBO
on IE is a sign that this field stays in the interest of many people,
• The selected articles do not present the initial resources necessary with few final rules about the best way to treat some problem. Most
and if they were considered at any moment during the project. It is of it as a result from the essence of the DSBO that put together
only familiar to talk briefly about the computer hardware and problems that always will reflect the new challenges of the in-
software specifications that were used. The DSBO was not con- dustries and the optimization methods that follow and put together
sidered, in overall, a project in terms for the management of time, the discovery on areas such as combinatorial problems and

537
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

computer technological advances; approach for inventory replenishment problem with premium freights in convergent

• There is a demand on the DSBO on IE to the development of pro- supply chains. Omega (United Kingdom), 80, 153–165. https://doi.org/10.1016/j.
omega.2017.08.016.
grams that put together modeling and optimization, in a way that Banks, J. (Ed.). (1998). Handbook of simulation: Principles, methodology, advances,
the decision agent can test and implement different kinds of opti- applications, and practice. John Wiley & Sons, Inc. doi: 10.1016/S0278-6125(99)
mization. This can be a sign to the development of research projects 90111-5.
Banks, J., Nelson, B. L., Carson, J. S., & Nicol, D. M. (2010). Discrete-Event System
between IE and Information Technology to develop such software to Simulation. PrenticeHall international series in industrial and systems engineering (5th
be friendly to the final user and speed up DSBO projects with dif- ed.). Pearson.
ferent methods; Baril, C., Gascon, V., & Cartier, S. (2014). Design and analysis of an outpatient ortho-

• The studies selected do not present if the best solutions were used or
paedic clinic performance with discrete event simulation and design of experiments.
Computers and Industrial Engineering, 78, 285–298. https://doi.org/10.1016/j.cie.
not, even in the real study cases, coupling if the best solution gen- 2014.05.006.
erated the simulated advantages proposed and how close they are Bianchi, L., Dorigo, M., Gambardella, L. M., & Gutjahr, W. J. (2009). A survey on me-
taheuristics for stochastic combinatorial optimization. Natural Computing, 8(2),
from reality, as a final test to prove that the DSBO was efficient as
239–287. https://doi.org/10.1007/s11047-008-9098-4.
the initial proposal; Bierlaire, M. (2015). Simulation and optimization: A short review. Transportation Research
• There are few articles (19.9% of the total cases) where the name of a Part C: Emerging Technologies, 55, 4–13. https://doi.org/10.1016/j.trc.2015.01.004.
Booth, A., Papaioannou, D., & Sutton, A. (Eds.). (2012). Systematic Approaches to a
company appears together with the list of authors, showing little
Successful Literature Review. SAGE Publications.
interaction, related to real cases. That can be a bad sign that com- Can, B., & Heavey, C. (2011). Comparison of experimental designs for simulation-based
panies were not entirely involved during the process of the DSBO symbolic regression of manufacturing systems. Computers and Industrial Engineering,
project, showing that in the future the enterprise could not use and 61(3), 447–462. https://doi.org/10.1016/j.cie.2011.03.012.
Can, B., & Heavey, C. (2012). A comparison of genetic programming and artificial neural
improve the DSBO;
•
networks in metamodeling of discrete-event simulation models. Computers and
The comparison of the optimization scenarios is made comparing Operations Research, 39(2), 424–436. https://doi.org/10.1016/j.cor.2011.05.004.
one specific method against each other. That compromise the gen- Carson, Y., & Maria, A. (1997). Simulation optimization: Methods and applications.
Proceedings of the 29th conference on winter simulation (1997) (pp. 118–126). . https://
eralization of results to that specific case. More robust optimization doi.org/10.1145/268437.268460.
framework can be proposed to test and solve a more significant Centre for Reviews and Dissemination (2009). Systematic reviews: CRD’s guidance for
number of different problems; undertaking reviews in health care. Centre for Reviews and Dissemination, University of

• On the 271 selected articles, a general summary was not found that York, 2008. https://doi.org/10.1016/S1473-3099(10)70065-7.
Chen, C. -H., Jia, Q. -S., & Lee, L. H. (Eds.). (2013). Stochastic Simulation Optimization for
consider together the aspects for problems on IE, optimization Discrete Event Systems. 1st ed. World Scientific. doi: https://doi.org/10.1142/
methods, and software/ hardware, in the first phase, for the con- 9789814282659_fmatter.
Choi, C., Seo, K.-M., & Kim, T. G. (2014). DEXSim: An experimental environment for
ceptual part of the DSBO project, the think about the resources
distributed execution of replicated simulators using a concept of single simulation
available and needed to develop or to search for already im- multiple scenarios. Simulation, 90(4), 355–376. https://doi.org/10.1177/
plemented ones, due to the variety of methods and software’s al- 0037549713520251.
ready existed in the market; Chwif, L., Paul, R. J., & Barretto, M. R. P. (2006). Discrete event simulation model re-
duction: A causal approach. Simulation Modelling Practice and Theory, 14(7), 930–944.
https://doi.org/10.1016/j.simpat.2006.05.001.
It is known that academic papers tend to present, using the scientific Collier, N., & North, M. (2012). Parallel agent-based simulation with Repast for High
method, how a proposed change in the current state of knowledge can Performance Computing. Simulation, 89(November), 1215–1235. https://doi.org/10.
1177/0037549712462620.
lead to a significant improvement on the initial analyzed results. The Costa, A. (2015). Hybrid genetic optimization for solving the batch-scheduling problem in
ideas presented above do not criticize but try to enrich the discussion in a pharmaceutical industry. Computers and Industrial Engineering, 79, 130–147. https://
a way that industrial organizations and overall interested people have doi.org/10.1016/j.cie.2014.11.001.
Costa, A., Alfieri, A., Matta, A., & Fichera, S. (2015). A parallel tabu search for solving the
more information that can be determinant on the success on the im- primal buffer allocation problem in serial production systems. Computers and
plementation of a DSBO on IE project. Operations Research, 64, 97–112. https://doi.org/10.1016/j.cor.2015.05.013.
Costa, E., Soares, A. L., & De Sousa, J. P. (2016). Information, knowledge and colla-
boration management in the internationalisation of SMEs: A systematic literature
Acknowledgments
review. International Journal of Information Management, 36(4), 557–569. https://doi.
org/10.1016/j.ijinfomgt.2016.03.007.
The authors thank CAPES, CNPq and FAPEMIG for supporting this Creighton, D., & Nahavandi, S. (2003). Application of discrete event simulation for robust
system design of a melt facility. Robotics and Computer-Integrated Manufacturing,
research and the anonymous referees for suggestions that contributed to
19(6), 469–477. https://doi.org/10.1016/S0736-5845(03)00057-7.
substantial improvements in the paper. Dahal, K. P., Galloway, S. J., Burt, G. M., McDonald, J. R., & Hopkins, I. (2005). A case
study of process facility optimization using discrete event simulation and genetic
References algorithm. Proceedings of the 2005 conference on genetic and evolutionary computation -
GECCO ’05, 2197https://doi.org/10.1145/1068009.1068372.
Das, S. R. (2000). Adaptive protocols for parallel discrete event simulation. Journal of the
Adegoke, A., Togo, H., & Traore, M. K. (2013). A unifying framework for specifying DEVS Operational Research Society, 51(4), 385–394. https://doi.org/10.1145/256562.
parallel and distributed simulation architectures. Simulation-Transactions of the 256602.
Society For Modeling and Simulation International, 89(11), 1293–1309. https://doi.org/ Dellino, G., & Meloni, C. (Eds.). (2015). Uncertainty management in simulation-optimi-
10.1177/0037549713504983. zation of complex systems. uncertainty management in simulation-optimization of
Ahmad, I., Subramaniam, S., Othman, M., & Zulkarnain, Z. (2011). A discrete event si- complex systems algorithms and applications (Vol. 59). Boston, MA: Springer US. doi:
mulation framework for utility accrual scheduling algorithm in uniprocessor en- https://doi.org/10.1007/978-1-4899-7547-8.
vironment. Journal of Computer Science, 7(8), 1133–1140. https://doi.org/10.3844/ Denyer, D., & Tranfield, D. (2009). Producing a systematic review. The SAGE Handbook of
jcssp.2011.1133.1140. Organizational Research Methods. https://doi.org/10.1080/03634528709378635.
Al-Aomar, R., & Al-Okaily, A. (2006). A GA-based parameter design for single machine Denyer, D., Tranfield, D., & van Aken, J. E. (2008). Developing design propositions
turning process with high-volume production. Computers and Industrial Engineering, through research synthesis. Organization Studies, 29(3), 393–413. https://doi.org/10.
50(3), 317–337. https://doi.org/10.1016/j.cie.2006.02.003. 1177/0170840607088020.
Alrabghi, A., & Tiwari, A. (2013). A review of simulation-based optimisation in main- Diez-Olivan, A., Del Ser, J., Galar, D., & Sierra, B. (2018). Data fusion and machine
tenance operations. Proceedings - UKSim 15th international conference on computer learning for industrial prognosis: Trends and perspectives towards industry 4.0.
modelling and simulation, UKSim 2013 (pp. 353–357). . https://doi.org/10.1109/ Information Fusion, 50(September 2018), 92–111. https://doi.org/10.1016/j.inffus.
UKSim.2013.27. 2018.10.005.
Alrabghi, A., & Tiwari, A. (2015). State of the art in simulation-based optimisation for Dorigatti, M., Guarnaschelli, A., Chiotti, O., & Salomone, H. E. (2016). A service-oriented
maintenance systems. Computers and Industrial Engineering, 82, 167–182. https://doi. framework for agent-based simulations of collaborative supply chains. Computers in
org/10.1016/j.cie.2014.12.022. Industry, 83, 92–107. https://doi.org/10.1016/j.compind.2016.09.005.
Amiri, M., & Mohtashami, A. (2012). Buffer allocation in unreliable production lines Ernst, A. T., Jiang, H., Krishnamoorthy, M., & Sier, D. (2004). Staff scheduling and ros-
based on design of experiments, simulation, and genetic algorithm. International tering: A review of applications, methods and models. European Journal of Operational
Journal of Advanced Manufacturing Technology, 62(1–4), 371–383. https://doi.org/10. Research, 153(1), 3–27. https://doi.org/10.1016/S0377-2217(03)00095-X.
1007/s00170-011-3802-8. Franceschini, F., Maisano, D., & Mastrogiacomo, L. (2014). Scientific journal publishers
Avci, M. G., & Selim, H. (2018). A multi-objective simulation-based optimization and omitted citations in bibliometric databases: Any relationship? Journal of

538
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

Informetrics, 8(3), 751–765. https://doi.org/10.1016/j.joi.2014.07.003. org/10.1109/TASE.2016.2574950.

Freitag, M., & Hildebrandt, T. (2016). Automatic design of scheduling rules for complex Martins, M. S. R., Fuchs, S. C., Pando, L. U., Lüders, R., & Delgado, M. R. (2013). PSO with
manufacturing systems by multi-objective simulation-based optimization. CIRP path relinking for resource allocation using simulation optimization. Computers &
Annals - Manufacturing Technology, 65(1), 433–436. https://doi.org/10.1016/j.cirp. Industrial Engineering, 65(2), 322–330. https://doi.org/10.1016/j.cie.2013.02.004.
2016.04.066. Maynard, H. B. & Hodson, W. K. (Eds.). (2004). Maynard́s Industrial Engineering
Fu, M. C. (1994a). A tutorial review of techniques for simulation optimization. Simulation Handbook. McGraw-Hill.
conference proceedings, 1994. Winter (pp. 149–156). . https://doi.org/10.1109/WSC. Merkuryeva, G., & Bolshakov, V. (2014). Integrated planning and scheduling built on
1994.717096. cluster analysis and simulation optimisation. Simulation and Process Modelling, 9.
Fu, M. C. (1994). Optimization via simulation: A review. Annals of Operations Research, Merkuryeva, G., Merkuryev, Y., & Vanmaele, H. (2010). Simulation-based planning and
53(1), 199–247. https://doi.org/10.1007/BF02136830. optimization in multi-echelon supply chains. Simulation, 87(8), 680–695. https://doi.
Fu, M. C. (2002). Optimization for simulation: Theory vs. practice. INFORMS Journal on org/10.1177/0037549710366265.
Computing, 14(3), 192–215. https://doi.org/10.1287/ijoc.14.3.192.113. Methley, A. M., Campbell, S., Chew-Graham, C., McNally, R., & Cheraghi-Sohi, S. (2014).
Handbook of Simulation OptimizationVol. 216. https://doi.org/10.1007/978-1-4939- PICO, PICOS and SPIDER: A comparison study of specificity and sensitivity in three
1384-8. search tools for qualitative systematic reviews. BMC Health Services Research, 14(1),
Gansterer, M., Almeder, C., & Hartl, R. F. (2014). Simulation-based optimization methods 579. https://doi.org/10.1186/s12913-014-0579-0.
for setting production planning parameters. International Journal of Production Moengin, P., Septiani, W., & Herviana, S. (2014). A discrete-event simulation metho-
Economics, 151, 206–213. https://doi.org/10.1016/j.ijpe.2013.10.016. dology to optimize the number of beds in hospital. Lecture notes in engineering and
Gourgand, M., Grangeon, N., & Norre, S. (2003). A contribution to the stochastic flow computer science (pp. 902–907). .
shop scheduling problem. European Journal of Operational Research, 151(2), 415–433. Mokhtari, H., & Salmasnia, A. (2015). A Monte Carlo simulation based chaotic differential
https://doi.org/10.1016/S0377-2217(02)00835-4. evolution algorithm for scheduling a stochastic parallel processor system. Expert
Hammersley, M. (2001). On ‘Systematic’ reviews of research literatures: A ‘Narrative’ Systems with Applications, 42(20), 7132–7147. https://doi.org/10.1016/j.eswa.2015.
response to evans on ‘Systematic’ reviews of research literatures: A ‘narrative’ re- 05.015.
sponse to evans & bene eld. British Educational Research Journal, 27(5), 543–554. Montagna, S., Viroli, M., & Roli, A. (2015). A framework supporting multi-compartment
https://doi.org/10.1080/0141192012009572 6. stochastic simulation and parameter optimisation for investigating biological system
He, Y., Liang, Y., Liu, Z., & Hui, C. W. (2017). Improved exact and meta-heuristic methods development. Simulation, 91(7), 666–685. https://doi.org/10.1177/
for minimizing makespan of large-size SMSP. Chemical Engineering Science, 0037549715585569.
158(October 2016), 359–369. https://doi.org/10.1016/j.ces.2016.10.040. Morrell, K. (2008). The narrative of “evidence based” management: A polemic. Journal of
Herrmann, F. (2013). Simulation based priority rules for scheduling of a flow shop with Management Studies, 45(3), 613–635. https://doi.org/10.1111/j.1467-6486.2007.
simultaneously loaded stations. Retrieved from Proceedings - 27th european conference 00755.x.
on modelling and simulation, ECMS 2013 (pp. 775–781). . <http://www.scopus.com/ Mujica Mota, M., & Flores De La Mota, I. (Eds.). (2017). Applied Simulation and
inward/record.url?eid=2-s2.0-84900327173&partnerID=tZOtx3y1>. Optimization 2. Cham: Springer International Publishing. doi: https://doi.org/10.
Hildebrandt, T., Goswami, D., & Freitag, M. (2015). Large-scale simulation-based opti- 1007/978-3-319-55810-3.
mization of semiconductor dispatching rules. Proceedings - winter simulation con- Muta, H., Raymond, R., Hara, S., & Morimura, T. (2015). A multi-objective genetic al-
ference, 2015–Janua (pp. 2580–2590). . https://doi.org/10.1109/WSC.2014. gorithm using intermediate features of simulations. Proceedings - winter simulation
7020102. conference, 2015–Janua (pp. 793–804). . https://doi.org/10.1109/WSC.2014.
Jafer, S., Liu, Q., & Wainer, G. (2013). Synchronization methods in parallel and dis- 7019941.
tributed discrete-event simulation. Simulation Modelling Practice and Theory, 30, Naderi, B., Khalili, M., & Tavakkoli-Moghaddam, R. (2009). A hybrid artificial immune
54–73. https://doi.org/10.1016/j.simpat.2012.08.003. algorithm for a realistic variant of job shops to minimize the total completion time.
Jahangirian, M., Eldabi, T., Naseer, A., Stergioulas, L. K., & Young, T. (2010). Simulation Computers and Industrial Engineering, 56(4), 1494–1501. https://doi.org/10.1016/j.
in manufacturing and business: A review. European Journal of Operational Research, cie.2008.09.031.
203(1), 1–13. https://doi.org/10.1016/j.ejor.2009.06.004. Nageshwaraniyer, S. S., Son, Y.-J., & Dessureault, S. (2013a). Simulation-based optimal
Jia, S., Bard, J. F., Chacon, R., & Stuber, J. (2015). Improving performance of dispatch planning for material handling networks in mining. Simulation, 89(3), 330–345.
rules for daily scheduling of assembly and test operations. Computers and Industrial https://doi.org/10.1177/0037549712464278.
Engineering, 90, 86–106. https://doi.org/10.1016/j.cie.2015.08.016. Nageshwaraniyer, S. S., Son, Y.-J., & Dessureault, S. (2013b). Simulation-based robust
Juan, A. A., Faulin, J., Grasman, S. E., Rabe, M., & Figueira, G. (2015). A review of optimization for complex truck-shovel systems in surface coal mines. 2013 Winter
simheuristics: Extending metaheuristics to deal with stochastic combinatorial opti- Simulations Conference (WSC) (pp. 3522–3532). IEEE. https://doi.org/10.1109/WSC.
mization problems. Operations Research Perspectives, 2, 62–72. https://doi.org/10. 2013.6721714.
1016/j.orp.2015.03.001. Nawara, G., & Hassanein, E. (2013). Solving the job-shop scheduling problem by arena
Kelton, W. D., Sadowski, R. P., & Swets, N. B. (2010). Simulation with arena (5th ed.). simulation software. Retrieved from International Journal of Engineering Innovation &
McGraw-Hill. Research, 2(2), 161–166. <http://www.researchgate.net/publication/236631006_
Kilmer, R. A., Smith, A. E., & Shuman, L. J. (1999). Computing confidence intervals for Solving_the_Job-Shop_Scheduling_Problem_by_Arena_Simulation_Software/file/
stochastic simulation using neural network metamodels. Computers & Industrial 60b7d523d5571c3b3e.pdf>.
Engineering, 36(2), 391–407. https://doi.org/10.1016/S0360-8352(99)00139-4. Negahban, A., & Smith, J. S. (2014). Simulation for manufacturing system design and
Kleijnen, J. P. C. (2009). Kriging metamodeling in simulation: A review. European Journal operation: Literature review and analysis. Journal of Manufacturing Systems, 33(2),
of Operational Research, 192(3), 707–716. https://doi.org/10.1016/j.ejor.2007.10. 241–261. https://doi.org/10.1016/j.jmsy.2013.12.007.
013. Neto, A. R. P., & Goncalves, E. V. (2010). A simulation-based evolutionary multiobjective
Kleijnen, J. P. C. (2017). Regression and Kriging metamodels with their experimental approach to manufacturing cell formation. Computers & Industrial Engineering, 59(1),
designs in simulation: A review. European Journal of Operational Research, 256(1), 64–74. https://doi.org/10.1016/j.cie.2010.02.017.
1–16. https://doi.org/10.1016/j.ejor.2016.06.041. Nguyen, A.-T., Reiter, S., & Rigo, P. (2014). A review on simulation-based optimization
Krause, W., & Schutte, C. (2016). Developing design propositions for an open innovation methods applied to building performance analysis. Applied Energy, 113, 1043–1058.
approach for SMEs. South African Journal of Industrial Engineering, 27(3), 37–49. https://doi.org/10.1016/j.apenergy.2013.08.061.
https://doi.org/10.7166/27-3-1625. Oliveira, J. B., Lima, R. S., & Montevechi, J. A. B. (2016). Perspectives and relationships
Lamiri, M., Grimaud, F., & Xie, X. (2009). Optimization methods for a stochastic surgery in Supply Chain Simulation: A systematic literature review. Simulation Modelling
planning problem. International Journal of Production Economics, 120(2), 400–410. Practice and Theory, 62, 166–191. https://doi.org/10.1016/j.simpat.2016.02.001.
https://doi.org/10.1016/j.ijpe.2008.11.021. Park, H., & Fishwick, P. A. (2010). A GPU-based application framework supporting fast
Laroque, C., Klaas, A., Fischer, J.-H., & Kuntze, M. (2012). Fast converging, automated discrete-event simulation. Simulation, 86(10), 613–628. https://doi.org/10.1177/
experiment runs for material flow simulations using distributed computing and 0037549709340781.
combined metaheuristics. Proceedings title: proceedings of the 2012 Winter Simulation Pawlewski, P., & Greenwood, A. (2014). Process simulation and optimization in sustainable
Conference (WSC) (pp. 1–12). IEEE. https://doi.org/10.1109/WSC.2012.6465058. logistics and manufacturingCham: Springer International Publishing. https://doi.org/
Law, A. M., & Kelton, W. D. (1991). Simulation modeling and analysis (2nd ed., vol. 2nd). 10.1007/978-3-319-07347-7.
New York: McGraw-Hill. Pilbeam, C., Alvarez, G., & Wilson, H. (2012). The governance of supply networks: A
Learmonth, M., & Harding, N. (2006). Evidence-based management: the very idea. Public systematic literature review. Supply Chain Management, 17(4), 358–376. https://doi.
Administration, 84(2), 245–266. https://doi.org/10.1111/j.1467-9299.2006.00001.x. org/10.1108/13598541211246512.
Leung, C. S. K., & Lau, H. Y. K. (2018). A hybrid multi-objective AIS-based algorithm Rajwani, T., & Liedong, T. A. (2015). Political activity and firm performance within
applied to simulation-based optimization of material handling system. Applied Soft nonmarket research: A review and international comparative assessment. Journal of
Computing Journal, 71, 553–567. https://doi.org/10.1016/j.asoc.2018.07.034. World Business, 50(2), 273–283. https://doi.org/10.1016/j.jwb.2014.10.004.
Li, S., Jia, Y., & Wang, J. (2012). A discrete-event simulation approach with multiple- Raska, P., & Ulrych, Z. (2015). Comparison of modified Downhill Simplex and Differential
comparison procedure for stochastic resource-constrained project scheduling. Evolution with other selected optimization methods used for discrete event simula-
International Journal of Advanced Manufacturing Technology, 63(1–4), 65–76. https:// tion models. Procedia Engineering, 100(January), 807–815. https://doi.org/10.1016/
doi.org/10.1007/s00170-011-3885-2. j.proeng.2015.01.435.
Long-Fei, W., & Le-Yuan, S. (2013). Simulation optimization: A review on theory and Riley, L. A. (2013). Discrete-event simulation optimization: A review of past approaches
applications. Acta Automatica Sinica, 39(11), 1957–1968. https://doi.org/10.1016/ and propositions for future direction. Retrieved from Simulation Series, 45(11),
S1874-1029(13)60081-6. 386–393. <http://www.scopus.com/inward/record.url?eid=2-s2.0-84880648611&
Lucidi, S., Maurici, M., Paulon, L., Rinaldi, F., & Roma, M. (2016). A Simulation-based partnerID=tZOtx3y1>.
multiobjective optimization approach for health care service management. IEEE Rosser, P. S., Sommerfeld, J. T., & Tincher, W. C. (1991). Discrete-event simulation of
Transactions on Automation Science and Engineering, 13(4), 1480–1491. https://doi. trouser manufacturing. International Journal of Clothing Science and Technology, 3(2),

539
W. Trigueiro de Sousa Junior et al. Computers & Industrial Engineering 128 (2019) 526–540

18–31. https://doi.org/10.1108/eb002973. Tanskanen, K., Ahola, T., Aminoff, A., Bragge, J., Kaipia, R., & Kauppi, K. (2017).
Ruiz, M., Moreno, J., Dorronsoro, B., & Rodriguez, D. (2018). Using simulation-based Towards evidence-based management of external resources: Developing design pro-
optimization in the context of IT service management change process. Decision positions and future research avenues through research synthesis. Research Policy,
Support Systems, 112(June), 35–47. https://doi.org/10.1016/j.dss.2018.06.004. 46(6), 1087–1105. https://doi.org/10.1016/j.respol.2017.04.002.
Saadouli, H., Jerbi, B., Dammak, A., Masmoudi, L., & Bouaziz, A. (2015). A stochastic Thomas, D. B., Howes, L., & Luk, W. (2009). A comparison of cpus, gpus, fpgas, and
optimization and simulation approach for scheduling operating rooms and recovery massively parallel processor arrays for random number generation. In The ACM/
beds in an orthopedic surgery department. Computers & Industrial Engineering, 80, SIGDA International Symposium on Field Programmable Gate Arrays (63–72).
72–79. https://doi.org/10.1016/j.cie.2014.11.021. https://doi.org/10.1145/1508128.1508139.
Sailer, A., Schmidhuber, S., Deubzer, M., Alfranseder, M., Mucha, M., & Mottok, J. (2013). Torraco, R. J. (2005). Writing integrative literature reviews: Guidelines and examples.
Optimizing the task allocation step for multi-core processors within AUTOSAR. Human Resource Development Review, 4(3), 356–367. <http://journals.sagepub.com/
International conference on applied electronics. doi/abs/10.1177/1534484305278283>.
Salam, M. A., & Khan, S. A. (2016). Simulation based decision support system for opti- Uhlig, T., & Rose, O. (2011). Simulation-based optimization for groups of cluster tools in
mization A case of Thai logistics service provider. Industrial Management & Data semiconductor manufacturing using simulated annealing. Proceedings of the winter
Systems, 116(2), 236–254. https://doi.org/10.1108/IMDS-05-2015-0192. simulation conference (pp. 1857–1868). . https://doi.org/10.1109/WSC.2011.
Salimi, S., Mawlana, M., & Hammad, A. (2018). Performance analysis of simulation-based 6147899.
optimization of construction projects using High Performance Computing. Van Volsem, S., Dullaert, W., & Van Landeghem, H. (2007). An Evolutionary Algorithm
Automation in Construction, 87(January), 158–172. https://doi.org/10.1016/j.autcon. and discrete event simulation for optimizing inspection strategies for multi-stage
2017.12.003. processes. European Journal of Operational Research, 179(3), 621–633. https://doi.
Salvendy, G. (2001). Handbook of Industrial Engineering. In G. Salvendyg (Ed.). org/10.1016/j.ejor.2005.03.054.
Handbook of industrial engineering(3th ed.). Hoboken, NJ, USA: John Wiley & Sons, Xu, J., Huang, E., Chen, C.-H., & Lee, L. H. (2015). Simulation optimization: A review and
Inc. https://doi.org/10.1002/9780470172339.fmatter. exploration in the new era of cloud computing and big data. Asia-Pacific Journal of
Saremi, A., Jula, P., Elmekkawy, T., & Wang, G. G. (2013). Appointment scheduling of Operational Research, 32(03), 1550019. https://doi.org/10.1142/
outpatient surgical services in a multistage operating room department. International S0217595915500190.
Journal of Production Economics, 141(2), 646–658. https://doi.org/10.1016/j.ijpe. Xu, J., Huang, E., Hsieh, L., Lee, L. H., Jia, Q.-S., & Chen, C.-H. (2016). Simulation op-
2012.10.004. timization in the era of Industrial 4.0 and the Industrial Internet. Journal of
Shahzad, A., & Mebarki, N. (2016). Learning dispatching rules for scheduling: A sy- Simulation, 10(4), 310–320. https://doi.org/10.1057/s41273-016-0037-6.
nergistic view comprising decision trees, tabu search and simulation. Computers, 5(1), Zhen, L., Wang, K., Hu, H., & Chang, D. (2014). A simulation optimization framework for
3. https://doi.org/10.3390/computers5010003. ambulance deployment and relocation problems. Computers and Industrial Engineering,
Taha, H. A. (2007). Operations research: An introduction (8th ed.). Pearson. 72(1), 12–23. https://doi.org/10.1016/j.cie.2014.03.008.

540