Measurement of Current and Potential Non-Financial Business
Value Delivery of IT Investments
Oscar Gonzlez-Rojas, Guillermo Beltrn and Dario Correal
Systems and Computing Engineering Department, School of Engineering, Universidad de los Andes, Colombia
E-mail: {o-gonza1, ga.beltran66, dcorreal}@uniandes.edu.co
Abstract
Most organizations trust their performance and continuity on Information Technology (IT)
investments that are expected to declare their business value delivery. We present a method to fill a gap
in the current literature pertaining to the measurement of current and potential value delivery focusing
on non-financial benefits. First, a metrics map with specific parameters for each metric was constructed
for measuring non-financial value delivery. Second, mapping artifacts were defined to guide the
selection and instantiation of the above metrics. These artifacts were used to measure non-financial
benefits of an enterprise information system investment on an educational organization.
Key Words: Business value delivery, Potential value, Measurement, IT investments, Non-financial
benefits, Intangible assets
1. Introduction and Motivation
The main goal of an organization is to deliver value to its stakeholders. Quantifying the
value delivered by IT investments becomes a critical organizational issue to accept costs and
risks associated with them. However, while several approaches [1] are focused on measuring
business value in function of financial returns, the measurement of current non-financial
benefits (e.g. improved quality, product variety, customer satisfaction) has been left aside due
to lack of methods to define and quantify specific metrics that are qualitatively treated.
Quantifying non-financial benefits becomes critical since they represent approximately 80%
of the benefits from an IT investment [2].
Current approaches lack of specific metrics in which their computing variables can be
selected for measuring non-financial benefits according to the available business information.
Moreover, there is a lack of methods to guide the measurement of potential value throughout
the long-term investment implementation [3].
This paper presents three artifacts designed to measure current and potential non-financial
business value. First, a metrics map was constructed by defining specific parameters required
to build non-financial metrics. Second, a mapping between the metrics map and value chain
activities was built to prioritize how to measure value delivery in terms of critical business
elements. Third, a mapping between the metrics map and the different execution stages of the
IT investment life cycle was built to guide the measurement of value delivery when the
investment is not in use. A particular IT investment in an educational organization was used
as case study to evaluate the usability of the proposed artifacts. The main results of this case
study and a final discussion are presented at the end of this paper.
2. Artifacts Created for Measuring Intangible Value
Artifact 1: A Metrics Map. This map provides organizations with specific metrics and
parameters that they can use to measure the intangible benefits of the investments. To create
this map we gathered metrics from multiple sources [4] [5] [6] [7] and focused on those
intended to measure intangible value. From this exercise we identified 287 different metrics.
We then rationalize this first set of metrics through aggregation of similar metrics and
identification of duplicated metrics. At the end of this stage we had a map with 31 intangible
metrics 1. We classified these metrics into five categories regarding how to deliver value:
Strategic Alignment (SA), Innovation (IV), Structural and organic (SO), Relations and
Communications (RC) and Human and Organizational Resources (HR). Specific parameters
were defined to these metrics in order to allow organizations to use them. For instance, to
measure the Product Quality metric, organizations can use the number of product defects
identified (e.g. at the end of a production line) or the number of complains presented by their
customers. By using this map, organizations can customize the usage of non-financial metrics
according to the available information or by implementing the most convenient mechanisms
to acquire the required information.
Artifact 2: Current Value Mapping. Fig. 1 illustrates this mapping by relating the
metrics map with a generic value chain (either a manufacturing or a services value chain).
Product Value Chain Service Value Chain
Inbound Operations Outbound Marketing After-sales Service Knowledge Delivery Moment of Service Legend
logistics logistics and sales services design management systems truth competition
management management management AI: Strategic alignment
5 AI 0 EC 5 AI 0 EC 5 AI 0 EC 7 AI 3 EC 7 AI 3 EC 5 AI 0 EC 5 AI 0 EC 5 AI 0 EC 7 AI 3 EC 7 AI 3 EC IV: Innovation
3 IV 1 OE 2 IV 3 OE 2 IV 3 OE 0 IV 2 OE 0 IV 4 OE 3 IV 1 OE 2 IV 3 OE 2 IV 3 OE 0 IV 2 OE 0 IV 4 OE RC: Relations and
5 RC 2 RO 5 RC 1 RO 5 RC 1 RO 5 RC 1 RO 6 RC 2 RO communications
5 RC 2 RO 5 RC 1 RO 5 RC 1 RO 5 RC 1 RO 6 RC 2 RO
EC: Econometric
Organization 4 AI 0 EC 3 OE 1 IV 4 RC 1 RO Physical aspects 4 AI 0 EC 3 OE 1 IV 4 RC 1 RO OE: Structural and
People 3 AI 3 EC 3 OE 0 IV 5 RC 6 RO Human resources 3 AI 3 EC 3 OE 0 IV 5 RC 6 RO organic
Technology Process information 5 AI 0 EC 1 OE 5 IV 5 RC 1 RO RO: Human and
5 AI 0 EC 1 OE 5 IV 5 RC 1 RO
organizational
Purchasing 3 AI 3 EC 0 OE 1 IV 4 RC 0 RO Punctuality and reliability 3 AI 3 EC 0 OE 1 IV 4 RC 0 RO
resources
Fig. 1. Mapping between non-financial metrics and value chain macro-processes
To use this artifact is necessary to identify the macro-processes of the organization that will
be supported by the IT investment or by identifying the business units that are aligned with
the investment stakeholders. Once these macro-processes are identified, the related metrics
1
The entire metrics map and their relationships with business and IT goals is available on:
http://minsky2.uniandes.edu.co/publico/ITGovernancePublic/ITValue/mapmetric.pdf
within the current value mapping are selected to measure business value in terms of
operational performance. An importance degree must be assigned to the selected metrics
(related with more business and IT elements) to scope the critical ones that must be
computed. Finally, the set of critical metrics can be grouped into the implementation stages
defined in the following artifact to limit the amount of metrics that must be measured at
different stages of the investment.
Artifact 3: Potential Value Mapping. Fig. 2 illustrates the IT investment lifecycle
proposed for supporting the measurement of potential value. We took the project
development phases (i.e., training, transition, change in work performance, balance, stability)
[8] and overlap them with the common states associated with organizational change [9], and
also with common activities expected on an investment. The artifact relates the metrics
defined in the metrics map with each phase in the constructed life-cycle where they should be
measured and analyzed depending on the status of the project (i.e., already executed, to be
executed). This can help an organization that is implementing an investment to dynamically
consider metrics and to identify relevant issues along its implementation. For example, a bad
result on Employee Satisfaction metric in the Transition phase may represent failures to
advance as expected in the change management and investment management curves.
Project development phases through time
Learning
Preparing
Negation
Competence on change
future
Equilibrium
Benefits
Uncertainty
Resistance
Expertise
acquisition
Investment results
Exploration IT improvement
Technological
diffusion Reinvention Tasks redesign
Launch System ability
Installation
Communication
Social integration
Learning
Training Transition Change in work Balance / Stability /
performance Equilibrium Improvement
Fig. 2. Project development lifecycle phases associated with non-financial metrics
Some of the metrics related to training (first phase) were knowledge development, know-
how and internal communication while for stability/improvement (last phase), some of the
metrics were informed decision making, IS effective management and organizational
learning. Organization is then able to select the best suited metrics to the phase it wants to
measure, and refer to the metrics map to identify the required parameters to compute those
metrics. Since the metrics are defined by phase, organizations can apply metrics to past
phases to identify issues, as well as to future phases to know where it has to focus to get the
expected benefits.
3. Measuring Non-Financial Value Delivery: A Case Study
The proposed artifacts were used on an educational organization (EO) in Latin America to
measure the intangible benefits of an Enterprise Resource Planning (ERP) investment named
TEA. This IT investment had impact on 7 of the 27 macro-processes within its value chain.
Once the processes being affected by the investment were identified, they were mapped to
the processes on artifact 2, and then to the appropriate metrics in the metrics map. Through
this process we identified six metrics that were used to measure the intangible benefits of the
investment (cf. Table 1). Measurement deviations between the obtained values and the
expected values from the organization can be used to plan an investment differently at early
or even latter stages. For instance, the organization decided to implement more workflows
after the investment as a corrective action to reach the 6 processes expected to be executed
near to real time.
Table 1. Selected metrics and value delivered
Selected Metric Parameters Value
Process Process execution time, Automated Response time in 4 processes near to real time, 5 new
Performance processes, Quantity of workflows processes automated, 2 new workflows
Human Assets Percentage of employees retained New department for TEA project (3 person), Increase
by 16 % of employees in EO
Labor Utilization Percentage of integration between IT TEA is integrated in 1 of 9 primary macro-processes
and the organization and 6 of 6 support macro-processes
Headcount Activities executed by an IS that in Activities executed by IS were quantified in function of
reduction or the past were executed by employees work templates used for activities execution: Before IS
avoidance implementation: 19 work templates; After IS
implementation: 25 work templates (1 replaced, 13
added, 7 removed, 7 changed and 4 kept)
Training Training hours, Percentage of 316 training hours (95 theoretical and 221 practical),
employees trained, Percentage of Increase by 16 % of employees in EO, 39 new experts
expertise with specific technologies in IS (616 employees trained)
Internal Quantity of processes unified, Time Process integrated with activities from different
communication required to execute an activity dependencies, Time reduction near to real time (hiring
process, goods requests, purchase requests, project
invoicing, legalization and project control ) Reports
delivered in online records (paperless)
4. Discussion
We identified that the definition of the proposed artifacts may represent benefits for the
organizations since they facilitate the agile selection and parameterization of metrics required
to measure value delivery on IT investments. These artifacts integrate and specialize metrics
of multiple IT frameworks created for measuring intangible value and relate them with
business criteria required to prioritize them (value chain activities, investment phases).
From the application of the proposal we identified the existence of a secondary value
(intangible value) that was recognized during the implementation of the investment, but that
is not represented by any of the metrics in the map. As future work it is important to improve
the metrics map by including metrics for the secondary value (e.g. establishment of policies,
definition of new roles and responsibilities, etc.). Documenting the artifacts created when
measuring value delivery generate skills and capabilities in the organization.
Some of the difficulties presented in the application of the proposal are the necessity to
have early access to business information and a continuous evaluation of the progress made
by the organization in the IT investment implementation. This is mandatory to avoid a
negative impact at the time of measuring the intangible value. Thus, it is necessary to add
metric selection criteria (e.g., availability of data sources) for increasing the accuracy of the
metrics that are by an organization to measure its non-financial value.
When an investment project is part of an investment program, potential and current value
may change depending on the cost of making one investment before another one, so that if
another investment is not executed before the current one, the potential value may be lower
or it may take longer to be achieved. This proposal can then be summarized as the ability to
consider these particular dependency scenarios so that the potential value and the current
value of an investment can be calculated in a precise way.
References
[1] Symons, C., Orlov, L. M., & Sessions, L., Measuring the business value of IT: a survey of
IT value methodologies. Forrester Research Inc. (2006).
[2] Sajko, M., Rabuzin, K., & Baa, M., How to calculate information value for effective
security risk assessment. J. Inf. Organiz. Sci., 30(2006), 263-278.
[3] Brynjolfsson, E., Malone, T. W., Gurbaxani, V., & Kambil, A., Does information
technology lead to smaller firms? Management Science, 40 (1994), 1628-1644.
[4] IT Governance Institute, Enterprise Value: Governance of IT investments. The Val IT
framework 2.0. IT Governance Institute. (2008).
[5] Keyes, J., Implementing the IT Balanced Scorecard: Aligning IT with corporate strategy
(1st ed.). Auerbach Publications. (2006).
[6] Carty, M. M., & Lansford, R., Using an IT business value program to measure benefits to
the enterprise. Intel Information Technology. (2009).
[7] Schryen, G., Revisiting IS business value research: What we already know, what we still
need to know, and how we can get there. Eur. J. Inform. Syst., 22(2013), 139169.
[8] Yee, J. T., & Oh, S.-C., Technology integration project planning and execution.
Technology Integration to Business. London: Springer London, 2013, pp 169-236.
[9] Elrod II, P. D., & Tippett, D. D., The death valley of change. Journal of Organizational
Change Management, 15(2002), 273-291.
