See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/311332657

Teaching ISO/IEC 12207 software lifecycle processes: A serious game approach

Article in Computer Standards & Interfaces · November 2017

DOI: 10.1016/j.csi.2016.11.014

CITATIONS READS

38 14,043

4 authors, including:

Murat Yilmaz Paul Clarke

Gazi University Dublin City University
111 PUBLICATIONS 1,281 CITATIONS 106 PUBLICATIONS 1,870 CITATIONS

SEE PROFILE SEE PROFILE

Rory V. O’Connor
Dublin City University
252 PUBLICATIONS 4,996 CITATIONS

SEE PROFILE

All content following this page was uploaded by Murat Yilmaz on 25 October 2017.

The user has requested enhancement of the downloaded file.

 Teaching ISO/IEC 12207 Software Lifecycle Processes: A Serious Game Approach

Ufuk Aydan
THK University, Ankara, Turkey
ufuk.aydan@ceng.thk.edu.tr

Murat Yilmaz
Çankaya University, Ankara, Turkey
myilmaz@cankaya.edu.tr

Paul M. Clarke
Dublin City University, Dublin, Ireland
paul.m.clarke@dcu.ie

Rory V. O’Connor
Dublin City University, Dublin, Ireland
rory.oconnor@dcu.ie

Abstract
Serious games involve applying game design techniques to tasks of a serious nature. In
particular, serious games can be used as informative tools and can be embedded in formal
education. Although there are some studies related to the application of serious games for the
software development process, there is no serious game that teaches the fundamentals of the
ISO/IEC 12207:1995 Systems and software engineering – Software life cycle processes,
which is an international standard for software lifecycle processes that aims to be ‘the’
standard that defines all the tasks required for developing and maintaining software. “Floors”
is a serious game that proposes an interactive learning experience to introduce ISO/IEC
12207:1995 by creating different floors of a virtual environment where various processes of
the standard are discussed and implemented. Inherently, it follows an iterative process based
on interactive technical dialogues in a 3D computer simulated office. The tool is designed to
assess the novice engineering practitioners knowledge and provide preliminary training for
ISO/IEC 12207:1995 processes. By playing such a game, participants are able to learn about
the details of this standard. The present study provides a framework for the exploration of
research data obtained from computer engineering students. Results suggest that there is a
significant difference between the knowledge gained among the students who have played
Floors and those who have only participated in paper-based learning sessions. Our findings
indicate that participants who played Floors tend to have greater knowledge of the ISO/IEC
12207:1995 standard, and as a result, we recommend the use of serious games that seem to be
superior to traditional paper based approach.

1. Introduction

Undoubtedly, there is an essential need for software systems to continue to satisfy today’s
complex and costly business demands [1]. Sommerville [41] claims, “we cannot run the
modern world without software” (pp. 4). The important point is that in today’s world there is
little chance to sustain all of the work around us without software. Many fields such as
industrial development, logistics, finance, and entertainment (e.g. computer games, music
market, cinema, etc.) rely on computer-based ecosystems to function properly. Without using
the benefits of software systems, it is difficult to build substantive international relations and
trade [1]. Gibbs states that projects and general applications from industry do not consistently
perform without clear definitions and implementations of market demands via software
engineering structures [21].
Software engineering is a complex activity [70] wherein developers should be aware of
their expected responsibilities when implementing software development projects.
Correspondingly, the successful implementation of customer requirements brings challenges,
not least because software systems are abstract and intangible [41]. Furthermore, each project
may vary in its software process [71] as software development settings are of a richly varying
nature [72]. In certain domains, governmental laws and physical materials may act to
complicate software development, for example in areas such as medical device and
automotive software development. In addition, there is also an absence of clear physical
borders for software products. However, with this absence, the complexity of software
systems and their development procedures can reach extreme points. Because of the
difficulties of understanding customer requirements, project costs and development expenses,
the entire process is beset with challenges. Moreover, rapid differentiation of operational
environments necessitates the usage of adoptable and versatile software product [24].
Developing from a simple program to a large-scale information system, there is a need for
a software development methodology [26]. Moreover, software engineers are always in
pursuit of developing new software techniques to make the development processes easier to
build in accordance with the satisfaction basic demands and procedures as well as desired
requirements of complicated and larger systems. As a consequence of these remarks, using a
disciplined approach, software engineering standards should be able to provide for more
effective development, especially for larger software product development initiatives.
Moreover, using such techniques and methodologies creates an essential approach to reaching
high quality products. Without such techniques, development progress may likely to be
interrupted with unwanted errors or defects. However, some companies may still rely on
undisciplined approaches and ultimately produce defective software products. At the end of
the process, their software product may be excessively expensive, and unfit for purpose. To
solve this issue, there is a crucial need for better training concerning software development
processes [38].
According to Calderon et al. [10], despite the general importance and urgency of this issue
there is an increasing demand for qualified personnel to empirically manage software
deliverables. However, software engineering lectures are usually populated with theoretical
knowledge which students are not able to observe empirically during their education. Boehm
[8] states that there is a need for novel frameworks to bridge the gap between software
engineering education and best practices. In addition, software engineering education requires
more interactive approaches to teach how to improve the software product while preserving
customer requirements [37]. Kling and Scacchi [31] agree that there is a need for clear
definitions of techniques in order to foster links between social prospects and software
attributes. Furthermore, without clarifying understandings from a managerial perspective, it
may be problematic to make further predictions about project trade-offs. All of these
observations suggest that there may be considerable importance associated with finding
interactive ways of teaching software development standards.
Games have started to play an important role in individual’s training and development. As
the notion of games evolve, it becomes easier to provide training and simulation of real-world
software development issues to novice practitioners. This study set out to investigate the
usefulness of serious games for teaching the basic content of ISO/IEC 12207. It provides an
exciting opportunity to advance our understanding for adopting interactive methods that are
offered to train novice practitioners. This paper considers the implications of serious games in
software process training particularly for ISO/IEC 12207 Software Life Cycle Processes.
Consequently, the primary hypothesis of the study was that interactive approaches like
serious games are more suitable for novice software practitioners than traditional paper
based learning approaches. To assess this proposition, a serious game was developed, and the
two different style of training sessions were conducted. To revisit the outcomes of our
proposal, we conducted a set of interviews and discuss the findings with a serious game
developer and a software process improvement expert. The contribution of this study has been
to provide a new understanding of ISO/IEC 12207 training.
The remaining part of the paper proceeds as follows. Section 2 includes detailed
information and background of ISO/IEC 12207 Software Life Cycle Processes. In addition,
important features of serious games are presented. Moreover, we have conducted a literature
review about serious games and this section clarifies the main characteristics of these selected
games. Section 3 introduces the research methodology. Additionally, procedure of
methodology is described step by step in every detail. Following this methodology, a serious
game called “Floors” was developed with the help of the development tools that are
mentioned. These design and development tools with their usage area during the development
phase of the game are presented with technical details of their functions and predefined
attributes. After the explanation about the design tools of the game, main features of "Floors"
are described with further descriptions about software engineering details and selected
processes of ISO/IEC 12207:1995 Software Life Cycle Processes. In section 4, firstly, in
order to present our participants' characteristics demographics are considered and then main
features of the sessions are identified individually. The main goal behind identifying the main
virtues of the sessions is exposing the structure of the methodology of this study. By this way
data collection and data comparison parts will become more visible. Moreover, this section
presents the efficiency of this research before starting analysis section where data analysis
and the evaluation of the sessions are described. For the discussion part, threats to validity
and interview results are presented for the sake of offering reliability of this study. Finally,
conclusion part consists of deliberation of the current condition of the game and future plans
which can be potential and possible advancements.

2. ISO/IEC 12207 Software Life Cycle Processes

ISO/IEC 12207 Software Life Cycle Process [27] is a comprehensive framework for
organizations to implement software projects in a more professional and well-planned way.
ISO/IEC 12207 establishes a common framework for software life cycle processes, with well-
defined terminology, that can be referenced by the software industry. It contains processes,
activities, and tasks that are to be applied during the acquisition of a software product or
service and during the supply, development, operation, maintenance and disposal of software
products. However, there may be insufficient awareness about the benefits of software
development methodologies and software life cycle frameworks in Turkish software industry.
According to TSE (Turkish Standards Institute) [45] there is only a single company, which
ability to integrate the structure of ISO/IEC 12207 Software Life Cycle Processes into their
software projects. Due to this fact ISO/IEC 12207 training holds the potential to raise
awareness of software development processes, activities and tasks among future generations
of software developers.
In this study, the main aim is to create a serious game [3,11] for establishing awareness and
improving participants’ knowledge about ISO/IEC 12207 software life cycle processes. The
goal is to help students to improve their understanding and decision-making skills regarding
software development processes across the entire life cycle. Although ISO/IEC 12207
software life cycle processes consist of the detailed definitions of processes, activities, and
related tasks, it does not consist of any mechanisms that may be applied to the task of
education. Rather, in effect the standard is designed to be a reference guide to for the
substantial details associate with the processes, activities, and tasks [74].
2.1. Background of ISO/IEC 12207 Software Life Cycle Processes

ISO/IEC 12207 is an international software engineering standard that defines the software
engineering processes and activities, which are associated with software life cycle process
from conception to end product [54]. It contains definitions for software processes, including
activities and tasks, but it does not provide guidance on the exact implementation of these
concerns [48]. Moreover, it does not measure product or process quality, nor does it prescribe
particular methods, practices or tools, while its modular structure renders it suitable for
tailoring purposes. Therefore, an organization can customize the necessary parts of the
standard that are planned to be used based on the requirements of a software project [30].
Because of the high modularity of the standard, it provides a framework that can help to
address the various factors that affect software development such as complexity, schedule,
cost, etc. In addition, ISO/IEC 12207 can act as an inventory of processes, which give
different perspectives to particular parts of the software life cycle process. These processes
are organized into seven different process groups:
ISO/IEC 12207 offers several different viewpoints as follows: contract, engineering,
operating, quality management, and management views [30]. Firstly, there is a contract view
that includes an acquisition process (i.e. for the acquirer) and supply process (i.e. for the
supplier). Secondly, there is an engineering view which has a development process for
product development and a maintenance process for up keeping the software. Thirdly, the
operating view with the operation process that provides a guideline for operating the software.
Fourthly, a quality management view that has six processes; joint review, audit, verification,
validation, quality, and problem resolution processes [30]. ISO/IEC 12207 Software Lifecycle
processes can be maintained by 7 main phases by any organization which has capability to
support the standard’s views and ability to handle software engineering requirements. These
main phases are; requirements analysis, specification, design, coding, verification &
validation, installation, maintenance & support [26]. Although the standard has explicit
definitions and substantial technical content, professionals and individuals might find it
difficult to fully implement owing to its very comprehensive nature. In light of this, the goal
of this work is to investigate the possibilities of adopting a serious game approach for
teaching the primary content of ISO/IEC 12207.

2.2. Serious Games

A serious game is an interactive approach designed for a purpose other than pure
entertainment [55]. A goal of a serious game is usually improving an educational aspect
where participants certainly attend such activities with such an expectation. These interactive
applications are widely preferred in training and education for medical and military
personnel. Recently, serious games become more popular and therefore they are now found
in any size, complexity and platform similar to casual games. In general, serious games,
consists rules that restrict players’ behaviors and actions during game play because of being
or concerned with more specific themes or subjects rather than commercial games. Due to its
main purpose, which is educating participants and increasing their awareness, serious games
can be used in many different disciplines [56]. In particular, the education aspects of these
interactive applications are heavily depending on the notion of play, which is an important
factor for individuals’ development and learning [54]. In addition, serious games are kind of
simulations of real-world events or processes that are addressed to comprise particular
problems [57]. Therefore, they can be considered as serious activities such as exploring,
training or advertising [58, 59]. However, they still can be entertaining, if their main purpose
covers game elements well. Substantially, games have many attributes which have been seen
in the case of different examples. For instance, serious games allow participants to experience
different learning tasks by using the elements of fun. Another example of attributes is stating
how actions affect the context. Players can create artifacts or complete tasks within in the
orders of a serious game serves and without the effects of real world problems and stress.
This can be interpreted to resembling sandbox type games. Moreover, serious games promote
an active participation while accomplishing its main goal. In fact, games are powerful tools,
because they have the ability to change human behavior [58]. The ambition and the direction
of this power can be useful with the help of incorporation with principles of learning [60].
This incorporation may encourage players’ decision making and viewpoints about
complicated positions in game play of a serious game and at the end of game play session
they clearly distinguish achievements and their success in particular subjects [60].
Furthermore, games can help users with repetitive actions while learning certain subjects.
Because particular tasks and clearly stated objectives of serious games make player easier to
follow certain pathways and play their role for a set of planned behaviors. Such planned
behaviors can be easily linked to the learning process where gaming may assist and
ultimately create a user-oriented learning experience [61, 62].
There are numerous works regarding serious games and applications discussed in the
literature, however only a small number of serious games are related to software project
management. These are; Problems and Programmers [63], SIMSOFT [65], SimSE [65,66],
SESAM [67], DELIVER [69], ProDec [69].
Problems and Programmers is a kind of serious game, which covers educational elements
with card components [63]. The primary ambition while designing the game is being as a
teaching tool to help, improve, and assist the student’s understandings towards software
engineering processes.
SimSE [66] is a type of serious game that is playable with a single player. It is designed to
serve an interactive environment which means there are graphical features and the game
includes visual elements.
ProDec [69] (Project Decision) is a kind of simulation-based serious game. The ambition
behind Calderon et al. is the intention to train and assess students in a software project
management [69]. SimSoft [64] is a type of serious game which consists of two game boards.
For the game there is a printed board and a digital board. DELIVER [68] is another type of
serious game, which consists of a printed board. It helps students to develop controlling
projects performances.

3. Research Design

3.1. Research Techniques

Generally, research methods are divided into two main types: quantitative and qualitative
methods [32]. According to Denzin [17] qualitative researches are carried out with some
traditional ways such as typing important notes, interviews, defining and marking cards,
sorting and shuffling them. Additionally, with the findings of Tesch et al. [44], qualitative
research involves asking typical questions to the participants and then according to their
answers the process of observation and analysis are conducted. Therefore, it is necessary to
understand these observations because it contains collecting data from experiences and
feelings of individuals directly [42].
Besides qualitative research techniques, the other set of research methodologies can be
categorized as quantitative research [25]. Mujis [32] defines that this research type involves
collection and analysis of numerical data via mathematical formulas and statistical tools.
Relying on this analyze period results are becoming relative about the desired subjects that
are waiting for the judgment [23]. Moreover, the whole process is about quantifying the
relationships between the variables, for instance, time, performance, etc. on sample of
subjects such as humans, animals, etc. [22]. To accomplish the research which is related to
this stud, a questionnaire and quiz have chosen. Furthermore, in order to complete the
quantitative aspect of the study suitable analysis were pursued.
Another research technique can be stated as mixed research [15]. According to Johnson et
al. [29] mixed research is a kind of combination that consists of both qualitative and
quantitative research methodologies. Additionally, mixed research has advantages like having
attributes from qualitative and quantitative methods by consisting of both numerical and non-
numerical data for the analysis [29, 43].
In order to maintain the research which is related to this study mixed research technique
has been chosen. Because of collecting numerical data and making interpretations from it,
related analysis methods were followed. In addition to this, interviews and collecting
feedback requires additional analysis. Therefore, all of these procedures required mixed
research methodology and we have chosen to get aid from the attributes of this methodology.
At the beginning of development phase of the serious game environment we have collected
ideas from different perspectives in order to develop adequate tool. These ideas and opinions
constructed the backbone of the serious game environment. After accomplishing the
development phase of the game we have employed two learning sessions in pursuance of
evaluating the efficiency of our tool. At the end of each session game play scale and quiz
questions were subjected to the participants. By this way we have collected numerical data
from participants, who have attended our sessions. This part contains quantitative data and
related analysis method. At the end of the evaluation of quantitative data we have conducted
interviews with regard to get feedback about game via verbal way. This part and the very
beginning of the study contain qualitative data and related analysis method. Consequently,
mixed research technique was conducted in this study.

3.2. Procedure of Methodology

Firstly, the study starts with the literature review about serious games that are related with
software engineering. The main goal was collecting necessary and required information from
various examples in order to construct a suitable framework where the serious game
environment was developed. Secondly, prototype of this virtual office environment is created
via Unity3D. In light of the feedback from experts from academia and students, the proposed
serious game was designed and developed. The proposed serious game’s name is named as
“Floors” to symbolize the different floors of virtual office environment. By this way, various
processes of the standard were discussed and implemented. Finally, to analyze the results
participants who are mostly the students of computer engineering and computer science are
accepted according to groups such as paper-based learning session and virtual learning
session.
To maintain this study, a virtual office environment which contains 9 floors to express
different processes of the standard were created in order to observe participants' experiences
while they are in virtual learning session. It creates more positive and attractive experience
for participants rather than paper-based traditional learning session. According to this
procedure and to conduct the main ambition of the study user experience study is adopted and
related evaluation technique is used.
According to the definition and the statements of Bernhaupt [5], there are various sets of
methods for measuring the experiences in the field of human computer interaction. This
period can contain software system, or software product that includes dedicated goals or
missions. Various methods such as interviewing, observing behaviors of participants,
conducting surveys can be stated. In conformance with these methods in order to conduct this
study surveying has chosen via using adopted version of Game Play scale [34]. It is a kind of
Likert scaling tool as an outcome of project from College University, London and was
developed by Parnell for the usage of assessing and measuring user experience and feedback
from sessions. Additionally, it contains distinctive subclasses and scales, which measures
different characteristic features of approaches. This kind of information can help developers
to get feedback from participants in order to validation of their approaches evaluation [35]. In
pursuance of determining the assessments of results and the differences between surveys of
each participant the adopted game play scale was subjected to the participants in both
traditional paper-based learning session and virtual learning session.
In addition to surveying at the end of each session participants were subjected to a quiz,
which consists combination of questions from ISO [27], IEEE Computer Society Project
Management Quizzes [40] and Software Engineering basics from Sommerville [41]. To
maintain this part, participants were grouped according to their involvement in the study. For
instance, participants who attended in traditional paper-based learning environment were
taken as one group and participants who attended in virtual learning environment were taken
as another group. Each group answers the quiz at the end of their sessions. Furthermore, the
data collected from each group is used to distinguish and analyze how they perform in quiz
according to their different experience environments.
Both the questionnaire and the quiz were reviewed by three academic staff that are
competent in computer science and software engineering in many years. According to the
researcher's presentation and the examinations of one associate professor and two assistant
professors about Game play scale and the quiz, they approved the suitability of the study with
the defined methodology procedure.
With descriptive steps the methodology and the research design was taken place like the
statements below.
1. This study starts with reviewing serious games in various areas such as software
engineering, management, etc.
2. After completing the review of literature a prototype game was designed which
contains the processes of the standard.
3. According to the feedbacks about prototype game the development of proposed
serious game was started via using Unity3D.
4. In this study data collection duration starts with introducing researcher and his
ambition with this study. Additionally, researcher explains the main features of Floors
and its goal. After necessary information is given to participants traditional paper-
based learning session takes place with distribution of official ISO/IEC 12207
Software Life Cycle Processes: 1995 [28]. They are directed to follow definitions of
key terminology and selected processes in order to accomplish learning session with
the basics of the standard. This session takes approximately one hour. After
participants covers the official standard’s terminology and its processes they subjected
to the adopted version of Game play scale questionnaire [34] in order to find out their
experience feedback with the paper-based learning session. Then they were given the
quiz to assess their success level based on their knowledge from this session.
5. In computerized part of this study participants involved the game play duration of
Floors, which is a serious game to teach basics of ISO/IEC 12207:1995 Software Life
Cycle Processes. This game was designed and developed for this study to visualize
the real life organizational scenarios with the help of virtual office environment and
representative character models for explaining the definitions and activities of the
processes of the standard in order to experience more extensive and realistic quests
and goals. Participants involved dedicated quests and dialogues while following the
processes of the standard in a virtual office environment.
6. At the end of the sessions participants were given the adopted version of the game
play scale to get their user experience feedback, and then they were subjected to the
 quiz to assess their success based on their knowledge which was obtained from virtual
learning environment. By this way, participants who play Floors did not use pen and
paper to learn the basics of the standard instead they actively involved in processes
with dedicated quests and dialogues to accomplish the learning journey. Lastly,
analysis of the results started. To clearly determine the results of this study the t-test
for two independent samples were used.
7. The defined steps of this procedure were repeated for every participant.
Consequently, entire procedure of the proposed methodology encompasses the mixed
research method technique which includes both quantitative and qualitative research methods.
In order to validate the effectiveness of the research independent sample t-test were used
while the analysis phase of the study. Following chapter will introduce this methodology via
visual parts and design elements of the proposed game in order to clarify the main goal of the
study.

3.3. Design tools of Floors

Floors is blended by various programming techniques and tools throughout the

development phase with Unity3D, Blender and Mixamo in order to reveal visual elements of
its main scenario.
Unity3D [46] is very popular game development environment that was developed by Unity
Technologies [14]. This development environment contains many supportive predefined
objects and modules in order to help users to focus on their specified design. However,
various kind of simulation which consist these kinds of intense computations and large sized
graphical elements can also be developed with this environment. Additionally, it serves an
environment to users to create both 3D and 2D games with comprehensive predefined
selections of attributes and assets. The main reason for the selection of Unity3D for the
development of Floors is being free and broad asset store, which helps any kind of developer
for finding, related assets for their particular needs. Besides enabling user to find out variety
of particular visual elements it helps developers to find out correct coding schemes and well
defined mapped objects in order to maintain specified objectives that are used in gameplay
sessions. Figure 1 shows a screenshot of the verification and validation processes dialogue
with responsible NPC.

Figure 1 Verification and Validation Processes dialogue with responsible NPC in game
 “Floors” helps players to see the virtual world like their perception in real life. By this way
they feel more immersed during game play. In order to create more responsive game
environment and to get the attention of the user during dialogue sessions and reading tips, we
created a set of real-life like events. By this way users can participate dialogues or reading
tips more focused. Mixamo Fuse [2] is tool, which was utilized to create 3D characters. It is
used to create both human and humanoid characters that can be used in game scenes. In this
study, Blender [7] is used to create basic models of start menu, floor numbers and necessary
office stuff where Unity3D is used to create the virtual environment.
The developed game contains 22 NPCs (Non-Playable Character). 14 of these characters
are actively used in game play duration. They have specifically defined scripts in order to get
interaction with user. Moreover, each of them is responsible one of the selected processes of
the standard and NPC explains required definitions in conformance with the exact
terminology according to the explanations of the standard. These definitions were
transformed into dialogues in order to integrate necessary information via more explanatory
way throughout the whole game play. By this way user followed a defined path between
processes and follow characters and their explanations in pursuance of completing different
floors. The official documentation of the standard contains comprehensive definitions
because of this while coding these definitions in proper format for the game environment
array data structure and nested conditionals were used with C# programming language.
Therefore, every NPC has own dedicated class which contains information about its specified
process and related activities. Beside these NPCs there are 8 more NPCs which contains
animated gifs in order to express the current status and objective of the floor. These NPCs
have no direct interaction with user however they reflect the atmosphere of the specified floor
with animations. These animations were adjusted with the help of Mixamo libraries and
edited with Unity3D environment to locate them adequately in the game scene.
Consequently, in this study every creation and asset were pieced together inside Unity3D in
order to preserve the main theme of Floors and integrity of the assets. Basically, main 3D
objects of Floors were created with Unity and Blender, animations and characters were
created via Mixamo, various office stuff especially PCs, printers, chairs, sofas, desks and
lightening equipment were imported from Unity Asset Store.

3.4. Features of Floors

To follow the main goal of the study, a virtual office environment and an organization is
created where software project management and ISO/IEC 12207 process can be realized.
According to the ISO [27] the standard is available to be used with different life cycle
models. However, organization should choose the model(s), which are going to be used
before the start of the development of whole processes. Based on this statement, Floors
conduct the processes of ISO/IEC 12207 with Waterfall development model sequence via
giving useful information during game play. Additionally based on the statements, which
come from ISO [27], there is no necessity to use all of the processes that are stated in the
standard. They are described as attributes of the standard and they are suitable to be adjusted
according to organization was. According to the requirements and the needs of the
organization during development of the software product they can be shaped. The processes
can be chosen specifically. Moreover, in Floors, there is a structure of virtual firm in order to
conduct representation of related personnel and processes of the standard. Therefore, this
study uses main and most frequently used processes especially in traditional development
models, which are waterfall, incremental, and iterative. Figure 2 shows the diagram of
selected processes and their occurrence sequence during game play (i.e. a task chain [9]). To
make participants active during game play various quests are located in different locations of
floors while continuing processes. By this way participants directly involve the processes and
their required activities with these quests.

Figure 2 the diagram of selected ISO/IEC 12207: 1995 processes and their occurrence
sequence during game play [29]

In order to create a feasible and controllable learning process using a serious game
environment, the primary researcher selected ISO/IEC 12207:1995 as the primary reference.
The goal is to explore the transferability of the knowledge about the standard and to reduce
the learning complexity through the use of a serious game. A later revision of ISO/IEC 12207
was introduced in 2008 (refer to Figure 2) and the latest revision again is presently under
ballot with ISO/IEC JTC1 SC7 WG10. Upon publication of the new revision, we intend to
repeat our exercise again with the new version of the standard.

4. Data Collection

To assess the difference between a paper-based training and interactive training (based on a
serious game), we test Floors on a selected group of participant. The goal is to explore what
kind of advancements in learning can we made by using a serious game. Participants of this
research were undergraduate freshman students who accomplished the same curriculum of
engineering courses. A group of participants (n = 40) was selected for this study. Next, these
participants were randomly divided into two groups that are named as paper-based learning
session and virtual learning session. In paper based learning session participants were
allowed to follow the selected processes of the standard via official documentation of
ISO/IEC 12207 [28]. On the other hand, participants who are in the virtual learning session
were allowed to play our serious game for completing the selected processes of the standard.
At the end of the both sessions participants were subjected a questionnaire, which was,
derived from a set of user experience questions particularly based on game play scale [34]
and quiz questions [40].
Moreover, demographics of the participants were collected via questionnaire. These values
have no direct impact on analysis and do not involve any correlation. Simply, they reflect the
main attributes of participants. In this research, there were 40 (n = 40) participants. 19 (47.5
%) of them were female and 21 (52.5 %) of them were male. Second demographic was age
value which was asked to participants while filling survey. 4 (10 %) of them were at age 19,
19 (47.5 %) of them were at age 20, 10 (25%) of them were at age 21, 4 (10%) of them were
at age 22, and 3 (7.5%) of them were at age 23. The other demographic was department in
order to present the participants’ departments. All of the participants were related with
computer science via their departments or their double major. 27 (67.5%) of the participants
were studying computer engineering, 10 (25%) of them were studying Electrical and
Electronics Engineering, 3 (7.5%) of them were studying industrial engineering.

4.1. Main Features of Sessions

The defined methodology helps collecting user experience feedbacks from two separated
sessions which are virtual learning session and paper-based learning session. The names of
the sessions are given according to their target participants and their assigned task. In virtual
learning session proposed serious game was played in order to find out the how successful the
serious game environment was according to the user experiences and exposing the knowledge
about the main structure of ISO/IEC 12207 processes. In this session participants were
allowed to play the serious game and followed the game scenario about standard's processes
with dedicated NPCs and virtually created environment. At the end of the session participants
were subjected to survey to get feedback about their experiences and the quiz to find out how
successful they were about the concept of the standard with their knowledge from the serious
game.
On the other hand, in paper based learning session participants were allowed to follow the
selected processes of the standard's documentation which was the same as the serious game
(see Figure 2), but this session had no chance to use and get benefit from any digitally created
element like the serious game. They used only pen and paper to follow and understand the
processes. At the end of the session same procedure like the virtual learning session was
conducted in order to compare the results of sessions.

4.2. Paper based learning session

On the side of revealing the learning of basics of the standard there is a need for time and
budget to read and cognizance the official standard terminology and descriptions of processes
with peculiar language. Moreover, there is no dedicated seminar or lecture in order to expose
the definitions of processes of ISO/IEC 12207 for individuals, but in organizational level
demands in conformance with specified criterion such as scale, budget, visions and missions
companies can get or reach the adaptation period of this standard for their related
implementations and project management politics. Unfortunately, finding an appropriate
seminar or lecture for this concept is not possible. Therefore, individuals who want to take,
learn or maintain this standard have to involve waiting limitless amount of time for learning
the processes from the official documentation. In an alternative way of this duration
individuals can follow processes with Software Engineering basics from the Internet via
documentations from experienced employees and academically units, but again it takes some
amount of time and deficiency of real experience to be productive in this field. During this
session in conformance with tailoring process participants followed the selected processes,
which are exactly the same ones with the virtual learning session for preserving the equality.
 Consequently, individuals who have involved in paper based learning session use only pen
and paper in order to understand of the basics of the standard and accomplishing main
structure within a time constraint.

4.3. Virtual Learning Session

According to Bjork and Holopainen [6], computer games may help us to create more
attractive and interesting environments than paper-based versions for today’s generations, and
maintains the costs and values of marketing strategies.
In conformance with the literature review of this study the importance of software
development in serious game environment is getting urgent day by day. Advancements in
such serious topics with a serious game are important aspects of software development with
the help of developing game engines and game design environments. To expose serious topics
to the individuals with using game elements and also to maintain with serious game concepts
can be beneficial to accomplish adequately the desires and goals. In this study one of the
ambitions is getting rid of pen and paper in order to learn the basics of the ISO/IEC 12207. In
virtue of created virtual office environment and modeled characters, participants have a
chance to experience a journey in an office virtually. The virtual office landscape is a 3D
environment enriched with animations, sounds, textures, NPCs, interactive dialogues and
quests. They are all desired attributes for accomplishing the dedicated learning goals in a non-
conventional way. Moreover, their goal is to improve the learning the processes of ISO/IEC
12207 with their non-static attributes and characteristics of paper-based learning session.
During this session, participants followed the processes of the standard (see Figure 2) in
conformance with tailoring process [27].
All of these dynamic contents during this session require some involvement while game
play to express goals and features. In addition to these virtual environment is flexible enough
for changing and adding to conduct various different subjects and theme. The simulation of
office environment with real world examples and scenarios help to create any imaginary
alternative environment. Using convenient tools and techniques for developing appropriate
setting and attributes of the virtual office environment costs of many things were reduced
while conducting the study in a realistic atmosphere.

5. Data Analysis and Evaluation

The main purpose of this research is to detect more beneficial and positive user experience
that participants from the virtual learning session over paper based learning session. With this
ambition participants were subjected to adopted version of game play scale at the end of each
session. Due to different sessions were involved the research independent t-test for two
independent samples was required to conduct the analysis of this study [22]. While
calculating the independent sample t-test IBM SPSS Statistics 23 was used. To accomplish
the test, level of significance was taken as 95% (0.05). The degree of freedom (df = N - 2)
were calculated as 38 since there were 40 (N = 40) participants. Figure 3 presents gameplay
scale score data distribution. Figure 4 presents quiz scores data distribution.
 Figure 3 Data distribution of Gameplay scale score.

Figure 4 Data distribution of Quiz scores

According to t distribution table the demanding t value is 2.021 [22]. All of the
independent t-test calculation was conducted via computer so the probability was too small to
be considered [22]. In conformance with these values t value was calculated as 4.274 (t =
4.274). Participants who involved virtual learning session tend to have more positive
experience and participants who involved paper based learning session tend to have slightly
neutral experience according to Gameplay scale questionnaire scores. To state how exactly
the measurement's variability there is a chance for obtaining this measurement's actual
treatment effect. There is a measure for finding the effect size that is Cohen's d [22]. In
Cohen's d formula 𝑆! and 𝑆! stands for the standard deviations of the independent samples.
The values are respectively 13.20 and 3.49. For the denominator df stands for the degree of
freedom which is 38 due the sample size 40 (df = N - 2). According to these values standard
pooled variance was calculated as 9.65 (Sp = 9.65) and Cohen's d was calculated as 1.35 (d =
1.35). To state how effective the result is there are defined values for d, which are 0.2 (small
effect), 0.5 (medium effect), and 0.8 (large effect) [22]. Due to this scale the effect size of this
study (d = 1.35) is large. The result is 0.324. According to the scale that is defined for the
values of 𝑟 ! 0.01 (small effect), 0.09 (medium effect), 0.25 (large effect). In conformance
with these values this study 𝑟 ! = 0.324 (32.4%) has large effect. Additionally, this research
also sought the assessment of participants via their obtained knowledge from sessions. For the
sake of accomplishing this ambition again independent t-test for two independent samples
were used while the analysis of obtained quiz scores. Again level of significance was taken as
95% (0.05). The degree of freedom (df = N - 2) was calculated as 38. The critical t value
should be 2.021. Under the light of these values t value was calculated as 3.186 (t = 3.186).
Similar to Game play Score analysis for the quiz scores analysis again Cohen's d and 𝑟 ! was
calculated in order to support the results and to clarify how effective the study is from the side
of quiz assessment. Cohen's d value was calculated as1.06 (d = 1.06) which shows the large
effect since it is greater than the value 0.8 and the 𝑟 ! value was calculated as 0.210
(𝑟 ! = 0.210) again it shows the large effect due to it is greater than 0.8 [22].

6. Discussions

This section briefly explains and reflects the findings of the research. According to this
study and based on the finding from literature the ISO/IEC 12207 standard is comprehensive
in identifying the processes, activities and tasks that may be required for software
development. Therefore, conducting processes of the standard requires learning and adoption
durations. Similar to other sciences and engineering fields, software engineering is required
experiments and learning sessions [73]. Enclosing observations while understanding the both
evaluation and evolution phases of this study were required some substantial work as well.
There can be threats that can cause some decrease in the validity of the results. However, in
this research, certain threats were reduced in order to get optimum results. According to the
definition from [48], "threats to validity" can be seen as potential circumstances that can
change usefulness and exactitude of the study in a wrong way. In addition to this definition
threats can be accepted as factors which effects to the results in a bad way [19].
Internal validity refers to whether a serious game makes a difference in outcomes. Here,
our results showed that participants who played floors increased their skills in ISO/IEC 12207
standard. There was therefore some scientific evidence to support claims that floors actually
“worked” as planned. Therefore, we confirm that our approach has an internal validity as a
training tool for ISO/IEC 12207 among the participants. External validity refers to
generalizability of the developed serious game. Our results can be extrapolated to other
settings where novice practitioners might probably prefer a tool like floors to get any training
for ISO/IEC 12207. Taken together, experimental validity (i.e. a combination of internal and
external validity) enable us to make claims that our approach works in light of the evidence
collected.
In this study, we adopted a number of mechanisms in order to reduce the impact of threats
from a validity perspective, including;
• Background of the participants may change or affect the results of data analysis.
o However, in order to preserve the target participants' main characteristics,
demographics were collected and they clearly present age, education and
department data to reflect the similar and familiar backgrounds. Because of
this reason participants were selected from undergraduate students who take
exactly the same courses.
• Time interval between surveys and quiz can cause a possible threat.
o However, there were no extraordinary condition since surveys and quizzes
were conducted immediately at the end of the sessions and the same day.
• Survey scoring can cause a threat
o To avoid this problem both sessions were subjected to same Likert scale
questionnaire and the same adopted version of the Game play scale.
• The participants, who can be in both sessions, may deliberately select incorrect
responses, which are subjected to them in sessions.
 • A participant, who is in the virtual learning session, may also examine and study the
official standard documentation.
• A participant, who is in the paper based learning session, may also play Floors and get
benefit from its features.
This study conducted quantitative research and analyzed quantitative data, but validation
interviews for Floors were also conducted in order to support the outcomes of the research. In
order to accomplish qualitative attitudes of the study these interviews were conducted. With
the help of three academics who are experienced instructors in computer engineering and
software engineering for many years, feedback was obtained. After completing sessions these
interviews were maintained. In these interviews 3 questions were asked to get direct opinions
and feedback about Floors. The questions are listed below;
• Question 1: What do you think about main characteristics of Floors?
• Question 2: What kind of suggestions can you make for further improvements of the game?
• Question 3: What kind of advantages / disadvantages Floors have over paper based learning
session?

According to answers for the first question one of the answer is "the game creates an
atmosphere where you can feel the office environment and do not afraid what will going on
with the stairs, just walk and talk with model and follow the path". They all agree that
designing a process path and interactive involvement while gameplay helps their
understandings of the concept of the standard. For the second question all of the answers were
about adding more features while completing the game scenario. One answer clearly stated,
"Being interactive game needs constant care and Floors have potential for expanding its
features". Lastly, for the third question participants stated there were no doubts about having
advantages as being interactive and graphically endorsed. One answer states that
"Undoubtedly the game has many advantages over reading the processes and trying to
understand the standard. Graphical creations and characters help the observations."
All the answers were collected and classified according to their attitudes towards the
questions. Table 1 presents the collected results of interviewees.

Interview 1 Interview 2 Interview 3

Question 1 Positive Positive Positive
Question 2 Positive Neutral Neutral
Question 3 Positive Positive Positive
Table 1 Collected and classified answers from interviews

According to these interviews, participants granted that our serious game environment has
a potential feature for increasing the decision making skills through the game play duration.
In addition to this the serious game environment helps them to involve the process in a more
visual way with the help of graphically fostered 3D office environment. In conformance with
the considerations and opinion of participants, our serious game environment is a valuable
tool to expose knowledge about the standard.

7. Conclusions

The primary goal of the study is to design and implement a serious game to help
individuals to gain knowledge about ISO/IEC 12207 in an interactive virtual 3D environment.
Rather than just reading the standard, participants are able to involve a live process with
dialogues and quests. Every necessary step was taken to develop the game completely. Every
digital creation was specifically designed for preserving this ambition. Moreover, several
different programs were used in order to make more refined virtual environment with
engaging features. Our essential finding is that those participants who adopted the serious
game approach to learning about ISO/IEC 12207 appear to have benefited from superior
educational outcomes when compared to those participants who applied the more traditional
paper-based approach. This finding has implications for education in general, where it may be
the case that games (as stated in [4]) can offer genuine and lasting improvements over
classical teaching techniques. At the present time, we must acknowledge that virtual 3D
environments such as were adopted in this study remain expensive from a technology
perspective which may currently impact upon the adoption of this technology. However, in
the fullness of time, it is suspected that the costs of this technology will decrease and when
they do, educators and trainers should strongly consider embracing the technology as a means
to improving educational outcomes. In the meantime, we intend to continue our research and
to apply the technology in an educational setting. The contribution of this study has been to
confirm that serious games have been proven effective for teaching ISO/IEC 12207 Software
Lifecycle Processes.
The major limitation of this study is the evaluation of the suitability of serious games to
learning 12207 was based on the 1995 revision. The first reason for this is that primary
researcher was experienced on the standard. Secondly, the research was conducted as an
exploratory study; therefore applicability of game elements to ISO/IEC 12207 was
preliminary tested. Thirdly, like most of the serious games, researchers did not planed for a
commercial release but to teach 12207 for novice developers. Consequently, more research is
needed to better understand the game’s potentials. However, it should be noted that using
games as teaching tools is ultimately serves for improving student’s understanding of the
12207 processes.
Notably, for the future improvement of Floors, Virtual Reality (VR) can be stated. Virtual
Reality is a kind of computer technology that simulates the specified environments to enable
user interaction [49]. Additionally, users get perceptional feelings about their existence in a
virtually created world, which is known as immersion [50]. As a relatively new game
technology, VR can enhance the prominent strength of Floors. Virtual Reality does not
require traditional input devices. Users have chance to control their actions and movements
without any limitations. However, in traditional gaming there is a need for specified input
devices and controllers in order to move and complete specified activities during game play.
However, by using VR technology, participants’ of Floors can examine and explore the
features of the game and the game play scenario by themselves in a more free way.

References
1. Hoskins, J. (2005). Conquering Information Chaos in the Growing Business: IBM
Solutions for Managing Information in an on Demand World. Maximum Press.
2. Smith, M., & Queiroz, C. (2015). Unity 5. x Cookbook. Packt Publishing Ltd.
3. Avedon, E. M., & Sutton-Smith, B. (1971). The study of games. John Wiley & Sons.
4. Bernard Suits, Grasshopper: Games, Life, and Utopia (Boston: David R.
Godine,1990), pp. 34-41
5. Bernhaupt, R. (2010). User experience evaluation in entertainment. In Evaluating
User Experience in Games (pp. 3-7). Springer London.
6. Bjork, S., & Holopainen, J. (2006). Games and design patterns. The game design
reader, 410-437.
7. Thorn, A. (2014). Animating 2D Characters for Games in Blender. Nelson Education.
8. Boehm, B. W., (1976) Software Engineering IEEE Trans. Computer, C-25, pp. 1226–
1241.
9. Wurdel, M., Sinnig, D., & Forbrig, P. (2008). Task-based development methodology
for collaborative environments. In Engineering Interactive Systems (pp. 118-125).
Springer Berlin Heidelberg.
10. Calderon, A., Ruiz, M.: Prodec: a serious game for software project management
training. In: ICSEA 2013, the Eighth International Conference on Software
Engineering Advances. (2013) 565–570
11. Abt, C. (1970). Serious games. New York, NY: Viking Press.
12. Clarke, P., & O’Connor, R. V. (2011). An approach to evaluating software process
adaptation. In Software Process Improvement and Capability Determination (pp. 28-
41). Springer Berlin Heidelberg.
13. Clearwater, D. (2011) What defines video games genre? Thinking about genre study
after the great divide. Loading…, 5 (8).
14. Creighton, R. H. (2010). Unity 3D Game Development by Example: A Seat-of-Your-
Pants Manual for Building Fun, Groovy Little Games Quickly. Packt Publishing
Ltd.pp. 1-115
15. Creswell, J. W., & Clark, V. L. P. (2007). Designing and conducting mixed methods
research.
16. Demirors, O., Demirors, E., Tarhan, A., & Yildiz, A. (2000). Tailoring ISO/IEC
12207 for instructional software development. In Euromicro Conference, 2000.
Proceedings of the 26th (Vol. 2, pp. 300-307). IEEE.
17. Denzin, N. K., & Lincoln, Y. S. (2011). The SAGE handbook of qualitative research.
Sage.
18. Derntl, Michael, Kroncik Milos, Klamma Ralf, Chacon Jonathan, Hernandez-Leo
Davinia (2014).Gamification of Learning Design Environments Workshop. 10th
European Summer School on Technology Enhanced Learning, Malta
19. Fayter, D., McDaid, C., & Eastwood, A. (2007). A systematic review highlights
threats to validity in studies of barriers to cancer trial participation. Journal of clinical
epidemiology, 60(10), 990-e1.
20. Futrell, R.T., Shafer, L.I., Shafer, D.F.: Quality software project management.
Prentice Hall PTR (2001)
21. Gibbs, W.W.: Software’s chronic crisis. Scientific American 271 (1994) 72–81
22. Gravetter, F. J., & Wallnau, L. B. (2016). Statistics for the behavioral sciences.
Cengage Learning.
23. Hopkins, W. G. (2008). Quantitative research design
http://www.sportsci.org/jour/0001/wghdesign.html [Last accessed in September
15,2016]
24. Hosier, W. A. (1961). Pitfalls and safeguards in real-time digital systems with
emphasis on programming. Engineering Management, IRE Transactions on, (2), 99-
115
25. Hoy, W. K., & Adams, C. M. (2015). Quantitative research in education: A primer.
Sage Publications.
26. Selby, R. W. (2007). Software engineering: Barry W. Boehm's lifetime contributions
to software development, management, and research (Vol. 69). John Wiley & Sons.
27. ISO/IEC. ISO/IEC 12207:2008 Systems and software engineering -- Software life
cycle processes (2008)
28. ISO/IEC: ISO/IEC12207-1995- Systems and software engineering Software life cycle
processes. (1997)
29. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of
mixed methods research. Journal of mixed methods research, 1(2), 112-133.
30. Jones, A.: ISO 12207 software life cycle processes fit for purpose? Software Quality
Journal 5 (1996) 243–253
31. Kling, R., and W. Scacchi, The Web of Computing: Computer Technology as Social
Organization, Advances in Computers, 21, 1-90, Academic Press, New York, 1982.
32. Muijs, D. (2010). Doing quantitative research in education with SPSS. Sage.
33. Mullen, T. (2011). Mastering blender. John Wiley & Sons.
34. Parnell, M. J., Berthouze, N., & Brumby, D. (2009). Playing with Scales: Creating a
Measurement Scale to Assess the Experience of Video Games. University College
London, London, UK.
35. Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. MIT
press,pp. 99-113
36. Scacchi, W. (1984). Managing software engineering projects: a social analysis.
Software Engineering, IEEE Transactions on, (1), pp. 49-59.
37. Scacchi, W.: Process models in software engineering, in Marciniak, J.J. (ed.),
Encyclopedia of Software Engineering, 2nd Edition, John Wiley and Sons, Inc, New
York, December 2001.
38. O'Regan, G. (2012). A practical approach to software quality. Springer Science &
Business Media.
39. Singh, R.: International standard iso/iec 12207 software life cycle processes. Software
Process Improvement and Practice 2 (1996) 35–50
40. Software Project Management Knowledge Area Certificate Retrieved from
https://www.computer.org/web/education/project-management
41. Sommerville, I., (2011). Software Engineering 9th Ed. Pearson Education.
42. O’Connor, R. (2012). Using Grounded Theory Coding Mechanisms to Analyze Case
Study and Focus Group Data in the Context of Software Process Research. In M.
Mora, O. Gelman, A. Steenkamp, & M. Raisinghani (Eds.), Research Methodologies,
Innovations and Philosophies in Software Systems Engineering and Information
Systems (pp. 256-270). Hershey, PA: IGI Global.
43. Tashakkori, A., & Creswell, J. W. (2007). Editorial: The new era of mixed methods.
Journal of mixed methods research, 1(1), pp. 3-7.
44. Tesch R., (1990), Qualitative Research: Analysis Types and Software Tools".
Psychology Press, pp. 1-177.
45. TSE Yazılım Yaşam Döngüsü Süreçleri Belgelendirilen Kuruluşlar. Retrieved from
https://www.tse.org.tr/tr/icerikdetay/945/1216/belgelendirilen-kuruluslar.aspx
46. Unity Game Engine Retrieved from https://unity3d.com/
47. Uskov, V., & Sekar, B. (2014). Gamification of software engineering curriculum. In
Frontiers in Education Conference (FIE), 2014 IEEE pp. 1-8
48. Yilmaz, M. (2013). A software process engineering approach to understanding
software productivity and team personality characteristics: an empirical investigation
(Doctoral dissertation, Dublin City University).
49. Gonzales, D., Criswell, D., & Heer, E. (1991). Automation and Robotics for the Space
Exploration Initiative: Results from Project Outreach (No. RAND/N-3284-
AF/NASA). RAND CORP SANTA MONICA CA.
50. Costello, P. J. (1997). Health and safety issues associated with virtual reality: a review
of current literature (pp. 1-23). Advisory Group on Computer Graphics.
51. Aydan, U., Yilmaz, M., & O’Connor, R. V. (2015, June). Towards a serious game to
teach ISO/IEC 12207 software lifecycle process: an interactive learning approach. In
International Conference on Software Process Improvement and Capability
Determination (pp. 217-229). Springer International Publishing.
52. Clarke, P., O’Connor, R. V., & Yilmaz, M. (2012, May). A hierarchy of SPI activities
for software SMEs: results from ISO/IEC 12207-based SPI assessments.
In International Conference on Software Process Improvement and Capability
Determination (pp. 62-74). Springer Berlin Heidelberg.
53. Ritterfeld, U., Cody, M., & Vorderer, P. (Eds.). (2009). Serious games: Mechanisms
and effects. Routledge.
54. Cruz-Cunha, M. M. (Ed.). (2012). Handbook of Research on Serious Games as
Educational, Business and Research Tools. IGI Global.
55. Khosrow-Pour, M. (Ed.). (2008). Encyclopedia of information science and technology
(Vol. 1). IGI Global.
56. Herranz Sánchez, E., Colomo-Palacios, R., de Amescua-Seco, A., and Yilmaz, M.
(2014). Gamification as a disruptive factor in software process improvement
initiatives. Journal of Universal Computer Science, 20(6), 885-906.
57. Yilmaz M., Saran M., O’Connor R.V. (2014) Towards a quest-based
contextualization process for game-based learning. In: Busch C (ed) 8th European
Conference on Games Based Learning, Academic Conferences and Publishing
International Limited, vol 2, pp 645–651.
58. McGonigal, J.: Reality is broken: Why games make us better and how they can
change the world. Penguin Pr (2011)
59. Gee, J.P.: What video games have to teach us about learning and literacy. Computers
in Entertainment (CIE) 1 (2003) 20–20
60. Gee, J.P.: What video games have to teach us about learning and literacy. Macmillan
(2014)
61. Yilmaz, M., Yilmaz, M. and O’Connor, R. V., Clarke P., A Gamification Approach to
Improve the Software Development Process by Exploring the Personality of Software
Practitioners, Proceedings 16th International Conference on Software Process
Improvement and Capability dEtermination (SPICE 2016), Springer-Verlag, June
2016.
62. Gulec, U., & Yilmaz, M. (2016). A serious game for improving the decision making
skills and knowledge levels of Turkish football referees according to the laws of the
game. SpringerPlus, 5, 622.
63. Baker, A., Navarro, E.O., Van Der Hoek, A.: An experimental card game for teaching
software engineering processes. Journal of Systems and Software 75 (2005) pp. 3–16
64. Caulfield, C., Veal, D., Maj, S.P.: Teaching software engineering project
management–a novel approach for software engineering programs. Modern Applied
Science 5 (2011) pp. 87
65. Navarro, E.O., Baker, A., Van Der Hoek, A. (2004) Teaching software engineering
using simulation games. In: ICSIE04: Proceedings of the 2004 International
Conference on Simulation in Education.
66. Navarro, E.O., van der Hoek, A. (2004). Simse: An interactive simulation game for
software engineering education. In Proceedings: 7th IASTED International
Conference on Computers and Advanced Technology in Education, Kauai, Hawaii,
pp. 12–17
67. Drappa, A. Ludewig,J. (2000) Simulation in software engineering training.
In:Proceedings of the 22nd international conference on Software engineering, ACM
pp. 199–208
68. Von Wangenheim, C.G., Savi, R., Borgatto, A.F.: (2012) Deliver! – An educational
game for teaching earned value management in computing courses. Information and
Software Technology 54 pp. 286–298
 69. Calderon, A., Ruiz, M.: (2013) Prodec: a serious game for software project
management training. In: ICSEA 2013, the Eighth International Conference on
Software Engineering Advances. pp. 565–570
70. Clarke, P., O'Connor, R.V., Leavy, B.: (2016) A Complexity Theory viewpoint on the
Software Development Process and Situational Context. In: proceedings of the
International Conference on Software and Systems Process (ICSSP), Co-Located with
the International Conference on Software Engineering (ICSE), pp. 86-90
71. Clarke, P., O'Connor, R.V. (2013) An empirical examination of the extent of software
process improvement in software SMEs. Journal of Software: Evolution and Process,
Vol. 25(9), pp. 981-998.
72. Clarke, P., O'Connor, R.V. (2012) The situational factors that affect the software
development process: Towards a comprehensive reference framework, Information
and Software Technology, Vol. 54(5), pp.433-447.
73. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell and A. Wesslén,
"Experimentation in Software Engineering", Springer, ISBN 978-3-642-29043-5,
2012.
74. Clarke, P., & O’Connor, R. (2010). Harnessing ISO/IEC 12207 to Examine the Extent
of SPI Activity in an Organisation. In European Conference on Software Process
Improvement (pp. 25-36). Springer Berlin Heidelberg.

View publication stats