See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/356472297

Data Mining-Based Metrics for the Systematic Evaluation of Software Project

Management Methodologies

Chapter · January 2022

DOI: 10.1007/978-3-030-77637-4_3

CITATIONS READS

3 209

5 authors, including:

Patricia R. Cristaldo Daniela López De Luise

Universidad Tecnológica Nacional, Argentina, Concepción del Uruguay CI2S Labs - Computational Intelligence and Information Systems Laboratory
3 PUBLICATIONS   3 CITATIONS    318 PUBLICATIONS   241 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Mobile robotics View project

WEBSCIENCE.AADERA View project

All content following this page was uploaded by Daniela López De Luise on 23 May 2022.

The user has requested enhancement of the downloaded file.

 Metrics for the Systematic Evaluation of Project
Management Methodologies
Patricia R. Cristaldo1, Daniela López De Luise1, Lucas La Pietra1, Anabella De Battista1, Jude Hemanth2
(1)
GIBD – National Technological University, Regional Branch Concepción del Uruguay
(2)
CI2S LABS – Buenos Aires, Argentina

Abstract—Metrics allow to evaluate project management management and its effects. Some of these efforts are
methodologies. They can identify aspects related to the quality presented and discussed in this article, along with proposals
of the results and the economy of resources. This article for metrics and indicators in order to measure and evaluate
introduces a proposal for metrics to assess the performance in all aspects and components of project management
the use of project management methodologies, and the first methodologies and guides. It is thus intended to develop a
results on a small number of use cases. From bibliographic measurement framework, through the metrics listed, to
sources, a set of new metrics are defined to include those cross-evaluate different methodologies quantitatively. This
evaluations that up to now are subjective or non-existent. As allows, among other things, to express the degree of
part of the scope, are a detailed presentation of some of the
applicability of the strategies in the different phases of a
metrics, and its application to a case study, along with some
statistics. The applicability of the proposal to real cases is also
project and / or in projects in different contexts.
analyzed. The full derivation of the metrics, the complete
listing are not part of the scope, neither the functional The objective of this work is to present and analyze the
prototype that is being used to apply them. state of the art in the field of generating evaluation metrics
for project management methodologies. A proposal of
Keywords—Software Projects Management, Management original metrics and a certain combination of compatible
metrics , Data Mining, Natural Language Processing metrics extracted from the bibliography is also made. The
I. INTRODUCTION sections that follow show the state of the art in metrics and
evaluation of management control methodologies (section
The management of software projects, includes the II), presentation of a proposal of metrics formulated for
fusion of Science and Management. It includes several managing the scope of a project (section III), case studies
aspects: direction, scope, stakeholders, risks, planning and (section IV), conclusions and future work (section V).
control of activities, project requirements, and business
objectives. It refers to the project manager's abilities to
manage problems related to management and technology. II. LITERATURE REVIEW
To help, there are numerous project management The project management good practices guide (Pmbok)
methodologies and guides on the market. Some o them are determines ten areas of knowledge present in management:
PMBOK [1], PRINCE2 [2][3], APM [4], ISO 21500 [5], scope, time, quality, costs, risks, human resources,
SCRUM [6] [7], KANBAN [8], CRISP-DM [9] [10]. The communications, stakeholders, acquisitions and integration
correct management of projects looks for the conclusion in [1]. With this as a basis, the bibliographic search to evaluate
time and with the desired quality [11]. According to the the state of the matter is translated, in the present analysis, to
2018 CHAOS report, 29% of the projects respect the time, the complete management of a project.
budget, characteristics and functions required. In contrast,
The topic related to project management is extremely
37% do not respect any of these axes. 52% of projects
current and relevant to the sector. In order to show this fact,
experience delays, exceed budget, or implement fewer
an analysis is carried out on the metadata of the indexed
requirements[12]. This is over 10% of what was reported in
scientific publications that contain as search terms: “project
2010. Likewise, the report shows a cancellation of the
management information technologies metrics”. This search
project without product of 19%.
is limited to the Scopus database, of the Library Portal of the
National Technological University. Based on the above, a
Among the reasons for not reaching the objectives in a database with 965 publications corresponding to the period
timely manner can be cited: insufficient planning [13] [14] from 2011 to 2020 is made. The study presented here uses
[15] poor definition of requirements [16] [17] [18], lack of the bibliometrix library [25] and the VOSviewer software
skills, problems with the discipline of management and [26] that allows a graphic analysis to be carried out from the
organization on the part of those in charge of carrying out generation of maps based on co-authorship, citations, co-
the projects [19]. citations and keyword co-occurrence. Although the
analytical work is outside the scope of this work, the initial
Various authors propose the "hybrid" approach, which trend can be observed in Figure 1, which shows the relevant
merges traditional proposals with agile [20] [21] [22] [23] words of the articles in a cloud-style map. For this, the
[24]. In this methodology, the project managers must focus evaluation by dual count is established. That is, how many
not only on the final objective but also on the moment when times the words appear in the articles and how they are
they choose one methodology or another. related to each other.
These changes in management have been accompanied
by numerous efforts to systematically evaluate the quality of

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

 Fig.1. Cloud map of titles and abstracts

As can be seen, there is a high frequency of content presented to store and query data in the management of
around what it does to companies (Copany), metrics in the internet of things.
general (assessment) and to a lesser degree the study of
critical factors for success (critical success factor), Regarding software metrics there are several
strategies (strategy) , modeling (modeling) and services examples. Some allow evaluating the propensity of
(service). It is interesting that effects aspects (effect) are failures [63] [67] with neural networks [29], risk
also reflected. All this indicates factors normally assessment [30] [31] with free review and application of
understood in the processes of modeling and control of indicators, to predict the maintainability of the source
project management. code by applying machine learning techniques [ 58] [59]
[61], for the readability of the code [65] and to estimate
To measure project performance, there are different
metrics around the ISO 25010 product / software quality defects in the source code [55] [56] [62]. There are
model. In [27] a generic environment is proposed that is metrics affected only by software design [57], and others
inspired by the principles of Value-Centered Thinking at each stage: product, process, test, maintenance and
customer satisfaction, using machine learning algorithms
(VFT) and the method Meta-Question-Metric (GQM) to
develop performance criteria regarding the values of the [64].
stakeholders in the project. GOCAME (Goal-Oriented
Context-Aware Measurement and Evaluation) is a metric The metrics presented in the Pmbok [1] evaluate the
system [38] that is combined with GQM [37], C-INCAMI performance of projects according to the EVM method
(Contextual-Information Need, Concept Model, Attribute, (Earned Value Management). But projects with multiple
Metric and Indicator) [39] and associated with the most parallel work paths require special metrics [32]. Other
frequent quality metrics [40] [42] [43] [44] [60]. In [45] a strategies define an index of performance of duration [34].
framework based on predictions is developed to measure Specific metrics are also applied in risk management [33]
the behavior of a data warehouse. In [51] [53] quality [35]. For each case, to support the project manager, the
metrics are established to evaluate business process work of the developers is measured [68]. In [69] agile
models. metrics are defined to monitor and control the best
practices of the ISO / IEC / IEEE 12207 and ISO / IEC
TR 29110-5-1-2 standards.
Among the metrics, some are developed for specific
areas. In banks [28], the possibility of a quantitative
analysis for collaborative businesses is presented. The This work is aligned to the development of metrics,
authors apply Artificial Intelligence with semantics. For but not for project management itself, but for the
SMEs [36], the exploitation of information is measured evaluation of the process of applying management
with metrics that measure different parameters. Three models. The works [33] [35] for risk management and,
categories are defined: data, models and projects. In [68] [69] for management in general are taken as a basis.
medicine, metrics are used to measure the quality of Likewise, when looking for a comprehensive project
reconstructed images from CT scans [41]. In wireless management metric, new metrics of an original nature are
communication systems, flexibility metrics are used for provided, based on concepts mainly from NLP.
mixed numerology in 5G [46]. In natural systems, the
success of biodiversity mitigation strategies is measured III. METRICS FOR PROJECT SCOPE MANAGEMENT
[47]. In videogames, metrics are established to evaluate According to the Guide to Good Practices in Project
interactives with respect to user satisfaction [48]. In Cloud Management, scope management is the set of processes
Computing, metrics are established to measure necessary to ensure that all the work required to
fluctuations in the demand for resources and services [49], successfully complete the project is included [1]. Bearing
scalability [66] and security [50]. In vehicle traffic this in mind, it is important to formalize the initial project
management, complex network metrics are developed for document, Business Scope Statement, with a complete
different urban scenarios [52]. In [54] a metric approach is description of the list of requirements, in order to lay the

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

solid foundations of a quality project [1]. Despite the fact service. It is extracted from the list of requirements. If it
that this work is part of a comprehensive project to does not exist, it is evaluated at 0.
determine comprehensive metrics for the entire process of
applying project management models, due to space issues, RQS= ∑(r)/∑ (RQN, RQI, RQS, RQP, RQC)
only those related to project scope management are (eq.05)
presented. That is, only the metrics that evaluate how the r=count (Z>=0)
project management models manage, the scope RQN Ɛ [0..1)
management aspects, are described. They apply to the
The Project Requirements (RQP): are the set of actions
case study in section IV. The rest of the model and
and processes that the project must provide. It is extracted
indicators will be introduced in future publications and are
from the list of requirements. If it does not exist, it is
not part of the scope of this work.
evaluated at 0.
◦ Degree of Business Compressibility
(GCN) RQP= ∑(r)/∑ (RQN, RQI, RQS, RQP, RQC)
(eq.06)
According to the literature presented in the previous
r= count (Z>=0)
section, it is important to determine the level of RQN Ɛ [0..1)
comprehensibility in the Business Statement (mission,
vision, scope). In order to determine a metric from the The Quality Requirements (RQC): they are defined as the
findings and parameters found, we define: set of conditions or criteria that the product must satisfy. It
is extracted from the list of requirements. If it does not
GCN=relPalCla*relParr (eq.01) exist, it is evaluated in0
RQC= ∑(r)/∑ (RQN, RQI, RQS, RQP, RQC)
With: (eq.07)
 relPalCla=prom(p/parr) r= count (Z>=0)
 relPalCla=prom(p/parr) RQN Ɛ [0..1)
 p keywords (nouns <10% or 5 - 10 words less used)
The next section shows how these metrics should be
 parr paragraphs (number of paragraphs within the applied.
Business Statement)
IV. CASE STUDY
◦ Degree of Scope Completeness (GCA) The study in this section is based on a small set of
Another important aspect is to calculate the degree of organizations in the software industry that work with
completeness of the scope (eq.02). The description should project management. As can be seen in Figure 2, the
profiles of the respondents are varied (project leader, IT
contain the business requirements, stakeholder
project managers, company owner, and administrative
requirements, solution requirements, project requirements, directors without IT training).
and quality requirements. If one of these elements is not
present it evaluates to zero. Determined as GCA metric: At the same time, the type of companies participating
in this case study is also varied (see graph 3).
GCA=pond(RQN)*log2(pond(RQN))+ The database is divided into two samples: one in
pond(RQI)*log2(pond(RQI))+ which the PM previously define the scope of the product
pond(RQS)*log2(pond(RQS))+ and the project, and the other in which they do not define
pond(RQP)*log2(pond(RQP))+ it. In the latter case, PMs are assisted based on the data
pond(RQC)*log2(pond(RQC)) (eq.02) requested in the survey. The sample is protocolized
through a survey, through the Linkedin social network.
In turn, Business Requirements (RQN) is defined as a set Likewise, this section shows the treatment of metrics
of needs and opportunities of the organization (eq.03). It defined in the previous section, when they are applied
must be extracted from the list of requirements. If it does post-mortem to the responses obtained; and some
not exist, it is evaluated at 0. discussions about it in the context of project management.
RQN= ∑(r)/∑ (RQN, RQI, RQS, RQP, RQC)
(eq.03)
r=count (Z>=0)
RQN Ɛ [0..1)

Regarding Stakeholders' Requirements (RQI): set of needs

of those who participate in the project. It is extracted from
the list of requirements. If it does not exist, it is evaluated
at 0.
RQI= ∑(r)/∑ (RQN, RQI, RQS, RQP, RQC)
(eq.04) Fig.2. Respondent Profile
r= count (Z>=0)
RQN Ɛ [0..1)
The Requirements for Solutions (RQS): constitute the set
of characteristics and functionalities of the product or

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

 GCN GCA RQN RQI RQS RQP RQC

ID1 0.3 0.01 0.3 0.2 0.25 0.05 0.05

ID2 0.6 0.02 0.13 0.13 0.38 0.38 0.0

ID3 0.25 0.11 0.1 0.1 0.0 0.28 0.0

ID4 0.33 0.26 0.03 0.03 0.35 0.64 0.02
ID5 0.15 0.01 0.15 0.0 0.0 0.01 0.58
ID6 0.35 0.22 0.31 0.32 0.11 0.08 0.12
ID7 0.63 0.22 0.25 0.29 0.12 0.24 0.17
Fig.3. Company size ID8 0.61 0.21 0.68 0.67 0.70 0.16 0.15
ID9 0.68 0.32 0.51 0.33 0.41 0.62 0.72

◦ Demographic sample ID10 0.74 0.31 0.41 0.55 0.52 0.41 0.75
Of the total of respondents in the sample, the majority
are private sector organizations (80%), while only 20%
are from the public sector. On the other hand, 40% of the The determination of a measure implies the
organizations, at the time of the survey, have between 11 systematization of the collection of the information, but
and 20 employees, 30% more than 30 employees and 20% also the consistent generation of the documents on which
less than 10 employees. Of the validated respondents, one works and, above all, the processing, preserving the
65% are project managers and directors, and 35% are modeling bias in all instances, that is why it is not At no
project coordinators and members of project teams. 20% time does it carry out subjective interpretations or studies
of those surveyed certify that they have knowledge of differentiating the cases.
methodologies and guides to good practices in project
management. Another noteworthy fact is that 60% of the For metrics it is systematically processed and nouns,
respondents confirm that they have defined the scope of verbs, etc. are labeled. In some cases, some responses had
the project and the product, but not the other respondents. to be reorganized, since functional requirements were
confused as business requirements, among others.
◦ Behavior of Metrics
In order to complement the above, the linguistic
This section studies the behavior of the metrics analysis of each scope is performed separately using the
designed in Section III applied to the sample described in Octave NLTK library (c). This allows obtaining numerical
the previous point. complements on the expressiveness and complexity of the
As can be seen in Figure 4, the scopes were not always descriptive language of the documents, and indirectly of
defined despite the fact that they are all operating the information contained.
companies. The metrics of this study are intended to Both studies will be related to establish the adequacy
determine not only the characteristics of the management of the metrics proposed to measure this and other factors.
documents in relation to the type of company and its core Figure 5 shows the behavior of the frequencies of the
business. keywords corresponding to all the responses concatenated
to determine the scope.
a) ID3, ID5

b) ID1,
ID4, ID6
Fig.4. Defined scope
In order to incorporate and interpret from the
subjectivity of the respondent, in the scope of real projects
and, this fact is considered as part of the metrics. From
these documents and the communicated scopes, Table 1 is
c) ID2,
obtained, which results from the application of equations ID7, ID8
01 to 07 that determines the management of the project
scope. It should be remembered that these are part of a set
that encompasses other essential management documents
such as the business mission and vision.

TABLE 1: Preliminary results of application of metrics

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

 d) ID 9,
ID10
The table shows the comparison between the various
characteristics of the text from the linguistic point of
view:
DL: Lexical diversity
SR: Number of words without repetition
Fig.5. Word distribution (ID3, 5) CR: Number of words with repetition
For better comparison during the discussion of results, PMD: Number of words with length greater than
those corresponding to the same characteristics have been 10 characters
placed superimposed.
In that of the GCA metric, in turn, the partial values of
TABLE 2: Lexical analysis of texts its components RQN, RQI, RQS, RQP and RQC are
ID CR SR DL PMD discriminated. Table 3 shows the behavior of the
distribution of the words in each ID. The Ls represent a
1 281 129 0.46 37
linear behavior, where all the words present identical
2 206 113 0.55 37 frequency (value 1). The entries with the value NO,
3 212 121 0.57 24 correspond to the cases in which the respondent does not
4 159 99 0.62 18 answer the question. The rest of the cells present words
5 211 141 0.67 19
(one or more), which exceed the average value of the rest
and therefore are considered of special value for the
6 218 109 0.50 37
analysis.
7 175 90 0.51 30
8 318 153 0.48 29
Note that out of 50 entries, 42% present linear entries,
and 10% are not answered. In total they add up to 52%.
9 415 210 0.51 29

TABLE 3: Frequency of Distribution of the words in the text per question

ID RQN RQI RQS RQP RQC

1 L L PAYMENTS L L
2 L KNOWLEDGE SYSTEM L L
3 PAYMENTS L NO INTEGRATION PLATFORM NO
4 L NO HIGH L L
5 L NO NO NO SHOULD
BE
6 L TECHNIQUES L L L
7 AVAILABILITY L L DRAFT L
BILL
PROCESS
8 SYSTEM SYSTEM MODULE PROCESS L
LOAN
9 MANAGEMENT L MODULE BANK PROCESS DATA
MUST
BE
10 SURVEYS GOVERNMENT APPLICATION COMPLIANCE CREDITS
RESULTS SERVERS SHOULD RULES MODULE
BE LOAN
The next section studies the relationship between these companies present a greater degree of scope of
results, the linguistic analysis and the type of company. understanding of the business, reflected in its scope. They
correspond to companies where the document was defined,
V. DISCUSSION OF RESULTS but each one has a different size (one large, one medium and
The scope of the project must describe the one small). But in neither case is the owner responding but
characteristics, functions and requirements of the product or rather personnel specialized in software development
service and its management. This implies that, in the context management or senior management of projects at scale.
of this proposal, the defined metrics must be able to Note that the general distribution of words (figure 5)
establish the degree to which the project scope management shows the greatest lexical diversity for the 3 cases (number
satisfies the equations of business requirements (RQN), of steps in the curve), reflecting the greatest expressive
stakeholder requirements (RQI), requirements of solutions richness. An indicator with a cutoff of 0.63 is suggested for
(RQS), project requirements (RQP) and quality HIGH.
requirements (RQC).
GCN is minimal in the scope of the ID3 case, as well as
The highest GCN occurs in ID 10, followed by ID 9 and that of ID5. Where the values do not reach 0.30, that would
7 (all greater than or equal to 0.63). This indicates that these be a low rating level. This could be interpreted in terms of

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

lexical complexity, that is, the degree of richness of In these cases, the highest GCA level coincides, showing
expression of the information. It is interesting to see that it is the highest qualification in terms of formalization. It
true that the low levels of word repetition counts occur in corresponds to large companies, where the person
these two surveys (in fact practically the minimum of the responsible for answering the survey is the project leader. In
whole set), while if it is taken from Table 2, the product of both cases, the companies do not have branches and are
DL x SR, gives the smallest values of the whole set, located in an important provincial capital. It is interesting to
indicating less expressiveness of the text. It is also given that note that in one of the cases the scope was not previously
their PMD values are in low to medium values. Figure 5a) defined, and this is reflected in a lower GCA.
effectively shows that the number of "jumps" in the
frequencies are few (2 and 3 respectively). In both cases the companies are established centers, solid
in the industry and with experience in project management
The GCA is also one of the lowest for these same cases. methodologies..
It corresponds to companies where the volunteer who
responds is the director, they have a defined scope at the Table 4 is a summary of the values found and their
organization level, but apparently it is not completely relationship with the cases. It also proposes an indicator
achieved since they cannot answer the typical questions that rating for the associated value ranges.
describe it. They also present problems in determining the TABLE 4: GCA indicators
degree of scope. Both cases correspond to local companies
in Concepción del Uruguay, both dedicated to software ID GCA indicator
consulting.
1,2,3,4,5 <0.2 LOW
The GCN for ID scopes 1, 4, and 6 have a value between
0.3 and 0.5. This constitutes a slightly better value than the 4,6,7,8 [0.2 – 0.3] MEDIUM
previous one, and corresponds both to texts with greater 2,7,8 >0.3 HIGH
lexical diversity and the greater DL x SR product
(0.0275,0.0858, 0.077). We would be facing more elaborate,
expressive and longer texts. This would correspond to more In a similar way to what was done with the GCA values,
information provided and therefore a better understanding of Table 5 presents a summary of GCN and a proposal of
the business. This in turn coincides with the maximum indicators.
levels of PMD in two of the three cases. The counts here are
also somewhat higher for two of the three cases. Note that
the ID 4 company does not have a maximum PMD but one
of the highest DL.
In Figure 5b) it is observed that the frequency jumps are TABLE 5: GCN indicators
greater than before (typically 3 except in case 4, which
reaches its complexity in another way) ID GCN Indicator
These cases all fall in the range of intermediate values of 3,5 <0.3 VERY LOW
GCA, typically they are medium and large companies, 1,4,6 [0.3 – 0.5) LOW
located in different provinces and countries. The branches
correspond to engineering companies with inputs and 2,7,8 [0.6 – 0.68) MEDIUM
personnel of various profiles, typically geographically 9,10 >=0.68 HIGH
spread over more than one city or belonging to the state
level. In these cases, the degree of understanding of the
business is much higher than the previous case, but not In Table 6, a summary of the cases is presented, and the
complete. In contrast the scope is more perfectly defined. dominant characteristics according to GCA, GCN, and the
linguistic counts of their texts. Clear delimitations are
The GCN for ID 2, 7 and 8 have a higher range than the observed that must be statistically confirmed with more
previous ones (from 0.5 to 0.65), where a higher degree of cases.
elaboration and more effective expressiveness is already
assumed. This is reflected in more varied texts than before TABLE 6: GCN and GCA according to the indicators
(the number of jumps in the images in figure 5c exceeds 3
ID GCN CM SRxDL GCA
jumps, and the IMD is greater). It is also noted that the DL x
SR is on average higher. It includes more mature companies 3,5 VERY LOW <= 4 <0.003 LOW
in terms of internal administration. Of the cases studied, two 1,4,6 LOW [3 - 6] [0.03-0.08] LOW -
are small and one is large. In all cases the volunteers with MEDIUM
the most complete knowledge of projects. None of the 2,7,8 MEDIUM [5 - 8] [0.01 – 0.2] MEDIUM
companies have branches but two of them are in major
provincial capitals. In these cases the degree of scoping is 9,10 HIGH [8 – 9] (0.2 - 1] HIGH
balanced and quite good..
As for ID 9 and 10, both present the highest values. Its An interesting fact that the study of table 3 provides has
maximum word count is the highest, the DL x SR value is to do with the accent that the company puts in each instance
also (0.21) and the number of jumps does not fall below 5 in with respect to its line of business. In the case of ID 3, only
graph 5 d), showing a richness and diversity of maximum "NO" appears as notable in the case of services, "Payments"
values. However, the words have a lower PMD than in other appears in the case of business requirements and "Platform"
cases, thus resorting to shorter words and presumably more and "integration" in project requirements. Being a company
frequently used. in the software field, it would indicate that there is a product

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

that is not totally determined, but that payments for the [6] Sutherland, J. Scrum: The art of doing twice the work in half the time.
products are intended. New York: Crown Business. 256 p. ISBN-10: 038534645X, ISBN-13: 978-
0385346450, 2014.
In the case of ID 5 the situation is worse, as no [7] Van Solingen R., Van Lanen R. (Scrum for managers) Scrum voor
significant term is in sight. Managers. Den Haag: Academic Service. EAN: 9789012585903, 2014.
[8] Lei H., Ganjeizadeh F., Jayachandran P., Ozcan, P.A statisc al analysis
Regarding intermediate GCA companies, some more of the effects of Scrum and Kanban on software development projects.
defined terms are observed as relevant, such as the case of Robotics and Computer-Integrated Manufacturing,
http://dx.doi.org/10.1016/j.rcim.2015.12.001, 2015.
ID 7 where "availability" is detected as a factor that the [9] Shearer C. The CRISP-DM model: The new blueprint for data mining.
company has as its key characteristic. Journal of Data Warehousing, no. 4, pp. 13-22, 2000.
[10] Shafique U., Qaiser H. A Comparative Study of Process Models Data
Companies where GCA and GCN are high have much more Mining (KDD, CRISP-DM and SEMMA).International Journal of
specific word differentiations, typically aligned with their Innovation and Scientific Research, pp.217-222, 2014.
core product. [11] Varajão, J. Dominguez, C. Ribeiro, P. Paiva, A. Critical success
aspects in Project management: similarities and differences between the
construction and software industry, Tech. Gazette 21 (3) 583–589, 2014.
[12] TSG. The CHAOS Report. The Standish Group, 2018. Disponible en
VI. CONCLUSIONS AND FUTURE WORK https://secure.standishgroup.com/reports/flyers/CM2018- TOC.pdf, 2018.
This work presents a subset of the metrics for the [13] Lehtinen T., Mäntylä M., Vanhanen J., Itkonen J., Lassenius, C.
Perceived causes of software project failures – An analysis of their
evaluation of project management methodologies, focused relationships. Information and Software Technology, pp. 623–643, 2014.
on managing the scope of the project. Only the Business [14] Ramos P., Mota C. Perceptions of success and failure factors in
Understandability (GCN) and the Degree of Completeness information technology projects: a study from Brazilian companies.
of the Scope (GCA) have been used, since other metrics Procedia - Social and Behavioral Sciences, pp. 349 – 357, 2014.
[15] Montequin S., Fernandez C., Fernandez O., Balsera J. Analysis of the
such as: Requirements Validation Criteria (CVR) and others, Success Factors and Failure Causes in Projects: Comparison of the Spanish
were outside the scope of this document. A case study with Information y Communication Technology (ICT) Sector. Journal
10 companies is also presented, the metrics are applied to Information Technology Project Management, pp. 18-31, 2016.
[16] Chow T., Chao D. A survey of critical success factors in agile software
these cases and certain evaluation indicators are proposed.
projects. Journal of systems and software, Jun. Available: Science Direct,
From the preliminary results presented here, it can be pp.961–971, 2008.
stated that the metrics allow the identification of [17] Elkadi H. Success and failure factors for e-government projects: A
descriptions of scope and requirements at different levels of case from Egypt. Egyptian Informatics Journal, pp. 165–173, 2013.
[18] ElEmam K., Koru A. A replicated survey of IT software project
specificity, which are compatible with simple and traditional failures Software. IEEE Software, pp. 84–90. 2008.
linguistic analyzes. [19] Blaskovics B. The impact of project manager on project success – The
Among the pending tasks are the formulation of metrics case of ICT sector. Assistant Professor, Strategy and Project Management
for the management of the other areas of knowledge that Department, Corvinus University Budapest E-mail: balint.blaskovics@uni-
corvinus.hu, 2016.
must be considered in project management, such as: time [20] Esteki M., Gandomani T., Farsani H. A Risk Management Framework
management, cost management, quality management, for Distributed Scrum Using PRINCE2 Methodology. Bull. Electrical
resource management, stakeholder management , Engineering and Informatics. DOI: 10.11591/eei.v9i3.1905. 2020.
communications management, risk management and [21] Jabar M., Mohd Ali N., Jusoh Y., Abdullah S., Mohanarajah S. A pilot
examination of an improved agile hybrid model in managing software
procurement. projects success. Test Engineering and Management, pp. 3040-3046, 2019.
It also remains to check the linguistic relationships [22] Smoczyńska A., Pawlak M., Poniszewska-Marańda, A. Hybrid agile
between and metrics with the largest number of cases. It also method for management of software creation. Advances in Intelligent
remains to evaluate not only the characteristics of the Systems and Computing, pp. 101-115. 2019.
[23] Włodzimierz Wysockia W., Orłowski, C. A multi-agent model for
management documents but also the impact of quality in the planning hybrid software processes. Procedia Computer Science pp. 1688–
present and future exercise of projects. 1697, 2019.
Another interesting point is to relate the metrics for the [24] Mousaei M., Gandomani T. A new project risk management model
identification of the implicit knowledge found within the based on Scrum framework and Prince2 methodology. International
Journal, 2018.
texts that describe the scope of the project, applying text [25] Cuccurullo C., Aria M., Sarto F. Foundations and trends in
mining techniques, referring to the process of discovering performance management. A twenty-five years bibliometric analysis.
and extracting relevant and non-trivial knowledge from Scientometrics, pp. 595–611, 2016.
unstructured texts. This is relevant given that the complexity [26] Van Eck N., Waltman L. Software survey: Vosviewer, a computer
program for bibliometric mapping. Scientometrics, pp. 523–538, 2010.
of natural language makes it difficult to access information [27] Barclay C., Osei-Bryson K. Project performance development
in texts and it is still far from being able to construct framework: An approach for developing performance criteria & measures
general-purpose representations of meaning from text for information systems (IS) projects. International Journal Production
without restrictions. Economics, pp. 272–292, 2010.
[28] Ivan I., Ciurea C., Zamfiroiu, A. Metrics of Collaborative Business
Systems in the Knowledge Based Economy. Procedia Computer Sciencie,
REFERENCES pp. 379-388, 2014.
[1] Project Management Institute. A Guide to the Project Management [29] Kumar L., Misra S., Rath S. An empirical analysis of the effectiveness
Body of Knowledge. 6 Ed. ISBN- 10: 9781628251845, 2017. of software metrics and fault prediction model for identifying faulty clases.
[2] PRINCE2. An introduction to PRINCE2: managing and directing Computer Standards & Interfaces pp. 1–32, 2017.
successful projects. Office of Government Commerce. Stationery Office, [30] Arar Ö., Ayan K Deriving thresholds of software metrics to predict
123 p. ISBN-10: 0113311885, ISBN-13: 978-0113311880. faults on open source software: Replicated case studies. Expert Systems
[3] Böhm, A. Application of PRINCE2 and the Impact on Project with Applications, pp.106–121, 2016.
Management. ISBN (eBook) 978-3-640-42634-8, 2009. [31] Wanderley M., Menezes J., Gusmão C., Lima F. Proposal of risk
[4] Highsmith, J. Agile project management: creating innovative products. management metrics for multiple project software development. Procedia
2 Ed. Boston, MA: Addison-Wesley. 432 p. 2010. Computer Science, pp. 1001 – 1009, 2015.
[5] ISO. ISO 21500:2012 Guidance on Project Management, Geneva: ISO, [32] Vandevoorde S., Vanhoucke M. A simulation and evaluation of earned
2012. Value metrics to forecast the project duration. Journal Operational Research
Society 58, pp. 1361-1374, 2007.

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

[33] Vanhoucke, M., 2011. On the dynamic use of project performance and [58] Gopal M., Amirthavalli M Applying Machine Learning Techniques to
schedule risk information during project tracking. Omega 39, pp. 416-426, Predict the Maintainability of Open Source Software. DOI:
2011. 10.35940/ijeat.E1045.0785S319, 2019.
[34] Khamooshi H., Golafshani H. EDM: earned Duration Management, a [59] Zagane M., Abdi M., Alenezi M. Deep Learning for Software
new approach to schedule performance management and measurement. Vulnerabilities Detection Using Code Metrics.
International Journal Project Management, 1019-1041, 2014. 10.1109/ACCESS.2020.2988557, 2020.
[35] Wood D. High-level integrated deterministic, stochastic and fuzzy [60] Ismail S., Mohd F., Jalil M., Wan Kadir W. Development metrics
cost-duration analysis aids project planning and monitoring, focusing on measurement level for component reusability evaluation approach (CREA).
uncertainties and earned value metrics. Journal of Natural Gas Science and DOI: 10.11591/ijece.v9i6.pp5428-5435, 2019.
Engineering pp. 303-326, 2017. [61] Oliveira B., Da S. Martins C., Magalhães F., Góes L. Difference based
[36] Basso, D. Propuesta de Métricas para Proyectos de Explotación de metrics for deep reinforcement learning algorithms.
Información. Revista Latinoamericana de Ingeniería de Software, pp. 157- 10.1109/ACCESS.2019.2945879, 2019.
218, ISSN 2314-2642, 2014. [62] Al Mamun M., Berger C., Hanson J. Effects of measurements on
[37] Papa, M. Aseguramiento de la Calidad de un Recurso Organizacional: correlations of software code metrics. https://doi.org/10.1007/s10664-019-
Evaluando y Mejorando una Estrategia Integrada de Medición y 09714-9, 2019.
Evaluación. Tesis Doctoral. Facultad de Informática, Universidad Nacional [63] Aziz S., Kahn T., Nadeem A. Experimental validation of inheritance
de La Plata, 2014. metrics' impact on software fault prediction.
[38] Becker P. Visión de proceso para estrategias integradas de medición y 10.1109/ACCESS.2019.2924040, 2019.
evaluación de la calidad. Tesis Doctoral. Facultad de Informática, [64] Siri D. Machine learning algorithm application in software quality
Universidad Nacional de La Plata, pp. 1 - 2012, 2014. improvement using metrics. DOI: 10.35940/ijeat.F1359.0986S319, 2019.
[39] Rivera M. Enfoque Integrado de Medición, Evaluación y Mejora de [65] Siddiqui T., Ahmad A. Mining software repositories for software
Calidad con soporte a Metas de Negocio y de Necesidad de Información: metrics (MSR-SM): conceptual framework. DOI:
Aplicación de Estrategias a partir de Patrones de Estrategia. Tesis Doctoral. 10.35940/ijitee.J1051.0881019, 2019.
Facultad de Informática, Universidad Nacional de La Plata, 2018. [66] Al-Said Ahmad A., Andras P. Scalability analysis comparisons of
[40] Arvanitou E., Ampatzoglou A., Chatzigeorgiou A., Galster M., cloud based software services. https://doi.org/10.1186/s13677-019-0134-y,
Avgeriou P. A mapping study on design-time quality attributes and metrics. 2019.
The Journal of Systems and Software, pp. 52–77, 2017. [67] Rhmanna W., Pandey B., Ansari G., Pandey D. Software fault
[41] Chillarón M., Quintana-Ortí G., Vidal V., Verdú G. Computed prediction based on change metrics using hybrid algorithms: An empirical
tomography medical image reconstruction on affordable equipment by study. Journal of King Saud University – Computer and Information
using Out-Of-Core techniques. Computer Methods and Programs in Sciences pp. 419–424, 2020.
Biomedicine, 2020. [68] Silva Ferreira M., Almeida Martins L., Júnior P., Costa H. Measuring
[42] Tebes G., Peppino D., Becker P., Matturro G., Solari M., Olsina developer work to support the software project manager: an exploratory
Analyzing and documenting the systematic review results of software study. https://doi.org/10.1145/3364641.3364651, 2019.
Testing ontologies. Information and Software Technology 123, 2020. [69] Mas A., Mesquida A., Pacheco M. Supporting the deployment of ISO-
[43] Jaleel A., Arshad S., Shoaib M., Awais M. Design Quality Metrics to based project management processes with agile metrics.
Determine the Suitability and Cost-Effect of Self-* Capabilities for https://doi.org/10.1016/j.csi.2019.103405, 2020.
Autonomic Computing Systems. Digital Object Identifier
10.1109/ACCESS.2019.2944119, 2019.
[44] Friedman A., Flaounas I. The Right Metric for the Right Stakeholder: a
Case Study of Improving Product Usability.
https://doi.org/10.1145/3292147.3292224, 2018.
[45] Raza B., Aslam A., Sher A., Malik A., Faheem M. Autonomic
performance prediction framework for data warehouse queries using lazy
learning approach. Applied Soft Computing Journal 91, 2020.
[46] Yazar A., Arslan H. A Flexibility Metric and Optimization Methods
for Mixed Numerologies in 5G and Beyond. Digital Object Identifier
10.1109/ACCESS.2018. 2795752, 2018.
[47] Bracy Knigth B., Seddon E., Tommbs T. A framework for evaluating
biodiversity mitigation metrics. DOI: 10.1007/s13280-019-01266, 2019.
[48] Paschali E., Ampatzoglou A., Escourrou R., Chatzigeorgiou A. A
metric suite for evaluating interactive scenarios in video games: An
empirical validation. Proceedings of the ACM Symposium on Applied
Computing 30, 2020.
[49] Benmakrelouf S., St-Onge C., Kara N., Tout H., Edstrom C., Lemieux
Y. Abnormal behavior detection using resource level to service level
metrics mapping in virtualized systems. Future Generation Computer
Systems pp. 680–700, 2020.
[50] Le N., Hoang D. Capability maturity model and metrics framework for
cyber cloud security. DOI 10.12694/scpe.v18i4.1329, 2017.
[51] Kbaiera W., Ghannouchib S. Determining the threshold values of
quality metrics in BPMN process models using data mining techniques.
Procedia Computer Science pp.113–119, 2019.
[52] da Costa J., de Souza A., Rosário D., Cerqueira E., Villas L. Efficient
data dissemination protocol based on complex networks’ metrics or urban
vehicular networks. https://doi.org/10.1186/s13174-019-0114-y, 2019.
[53] Ahmad M., Odeh M., Green S. Metrics for Assessing the Basic
Alignment between Business Process and Enterprise Information
Architectures with reference to the BPAOntoEIA Framework. IEEE 978-1-
7281-0385, 2018.
[54] Beecks C., Grass A., Devasya S. Metric indexing for efficient data
access in the internet of things. IEEE International Conference on Big Data
978-1-5386-5035, 2018.
[55] Kapur R., Sodhi B. A Defect Estimator for Source Code: Linking
Defect Reports with Programming Constructs Usage Metrics.
https://doi.org/10.1145/3384517, 2020.
[56] He P., Li B., Liu X. Chen J., Ma Y. An empirical study on software
defect prediction with a simplified metric set. Information and Software
Technology, pp. 170–190, 2015.

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

View publication stats