State of the Art of the Maturity Models

to an Evaluation of the Enterprise Architecture

Jihane Lakhrouit, Karim Baïna
AlQualsadi research team on Enterprise Architecture
Ecole Nationale Supérieure d’Informatique et d’Analyse des systèmes, ENSIAS Mohammed V- Souissi University
Rabat(10000) , Morocco
jihane.lakhrouit@gmail.com, baina@ensias.ma

Abstract — the evaluation of enterprise Architecture EA The questions of essential research studied in the document
allows companies to evaluate their current situation. More are the following:
specifically, it optimizes the alignment, the business strategy, the
organization, the resources, the processes and the technology. 1.What an enterprise architecture?
This optimized architecture provides a cost-efficient solution and
2. What is an evaluation?
also contributes to ensure the successful implementation of the
purposes of the organization. However it is very difficult to find a 3. Why should we evaluate the enterprise architecture (EA)?
complete method of evaluation in term of indicators of evaluation 4. What are the existing maturity models?
and which ensures the follow-up of all the stages of development 5. Can these methods be used for an evaluation?
of the architecture of company. This article presents the
possibility of evaluation starting from the methods of maturity of The article is structured as follows: the second section
existing enterprise architecture. presents the definition and the goal of enterprise architecture,
the third section describes the maturity methods, then the
Keywords — Enterprise Architecture, Maturity Models of fourth section presents a comparative study of maturity
Enterprise Architecture, Evaluation of Enterprise Architecture.
methods, with an evaluation model of the enterprise
I. INTRODUCTION architecture and the fifth section is to conclude and propose
the perspectives.
Today, the management of the organizations based on
strategic management is the only solution for the creation of II. ENTERPRISE ARCHITECTURE
value. Consequently, the performance must be measured by an
evaluation and measuring success during the progression of A. Definition of enterprise architecture
the strategies [15].
The EA entered in the world of the company in the years 1980
The wide range of definitions of enterprise architecture in when Zachman created the framework of architecture of IS [2].
the literature indicates that not only the enterprise architecture The framework of EA of Zachman consists of a table contains
is a new subject, but also that its emergence is closely linked 6 lines and 6 columns; as shown in the figure below in which,
with business prospects and information technologies [16]. In for each cell identifies a kind of model can be developed to
general, enterprise architecture can be defined as "a plan that document the company and its IS.
describes the structure of the mission, the necessary
information for the company, and technologies needed to
support them and and also defines the transition process of
the implementation of these technologies"[17].

During the implementation process of the EA, we proceed

to an evaluation regarding to the needs. The results of this
evaluation will be a useful basis for improving the system in
terms to achieve the goal and vision of the organization.

This is important because they help to show the strengths

and weaknesses of the company and led to the capture and
recognition of gaps. This can be used to guide business
strategies. This document aims at presenting the possibility to
evaluate EA starting from the models of maturity. Figure 1: The Zachman framework

978-1-4799-3392-1/13/$31.00 ©2013 IEEE

 This framework is seminal from its way of establishing a manner to improve the organization of the different aspects.
common vocabulary and a set of prospects to build a holistic The following chapter will present and compare these models
description of the complex systems for the company. of maturity by explaining the differences and the similarities.

Another definition is proposed by the Rational Edge (IBM) A comparative study of the various models of maturity was
which says that enterprise architecture can be defined as the carried out in 2009 which proposes a classification according
model that documents all information systems in the company, to Van Steenbergen [12] who classifies the models of maturity
their relationships, and how they interact to achieve the in three categories which are Stage, Continuous and Focus.
company mission [7].
A. Staged model
B. Goal of enterprise architecture These are models that have five maturity levels or more.
According to several studies [5][8][9][11] the enterprise Each level of maturity contains a certain number of indicators
architecture is practiced by a large number of international which must be satisfied to reach this level and to pass on the
organizations which adopted it like a tool to define their next level.
structures and their operations with an aim of achieving their
current and future goals. These studies showed that the EA is
practiced to solve 9 great problems:

x Supporting the alignment of business, information

technology (IT) and their integration
x Helping to manage the change Figure 2: Presentation of the staged maturity model

x Reducing the « time to market »

The most known model of maturity of this type is (CMM)
x Improving the availability of documentation of the and the model of maturity of management EA which we will
company detail is: Maturity Framework EAMMF from US Government
x Helping to unify and integrate the business processes General Accounting Office [12].
trade and the data through the company and with the
external partners US GAO is an independent organism which works for the
American Congress. He studies the way in which the federal
x Reducing the time and cost of development by government spends the taxes and gives advices for a more
maximizing the reuse of business models. effective use. In this context, GAO developed the framework
x Improving the agility related to the reduction of EAMMF maturity «It is a reference tool for planning and
complexity evaluation efforts of the enterprise architecture» [13].

x Helping to create and maintain a common and shared The EAMMF was determined to improve the EA
vision of the target with the business stakeholders and IT management in federal agencies. [13] The figure below
x Ensuring continuous alignment of business and IT presents the model EAMMF.
From these goals one can note that enterprise architecture
is not only a process of modeling or an alignment method, but
allows companies to model their business, their organizations,
their processes, and to urbanize their information system.
It formalizes the company and its IS to better control their
evolution, to manage the risks related to changes, also to
establish a plan of urbanization and to optimize the return on
investment on existing.

III. MATURITY OF ENTERPRISE ARCHITECTURE

The maturity of the enterprise architecture is an important Figure 3: GAO - EA Management Maturity Framework (U.S. Governance
aspect in the domain of development.There are several models Accountability Office (GAO)
of maturity enterprise architecture that can be used to monitor
the current status (level of maturity), to make a diagnosis and The levels of maturity: Each stage of maturity is
to propose changes and improvements in organizations. Some represented by conditions that are described in their
of these models show only the current state of EA in the corresponding evaluation items. A certain stage of maturity
organization, whereas others give also suggestions on the can only be achieved if all previous steps have been respected.
The stages function as a roadmap for an organization which B. Continuous model
increases its maturity. The seven stages of maturity ranging These models distinguish also five levels of maturity or
from the minimum maturity (stage 0) to a maximum maturity more and a certain number of indicators. The difference with
(step 6) are presented in the above model. the first type is that the indicators are not specific to only one
level, but for all the levels of maturity as shown in the figure
The indicators: The table below watches four layers of below. The model of maturity of the EA that we will study of
indicators. Each indicator has its own specific attributes of this type is: EAMM – NASCIO. Enterprise Architecture
success with their particular characteristics .The four Maturity Model from national Association of State Chief
representations with their attributes of success are presented in Information officers [19].
the table below.

TABLE1: THE INDICATORS OF EAMMF

Indicator Description
EA Management Demonstrates commitment
Action Provides capability to meet commitment
Representation Demonstrates satisfaction of commitment Figure 5: Presentation of the continuous maturity model
Verifies satisfaction of commitment
EA Functional Area Governance
The National Association of State CIOs (NASCIO) offers
Representation Content
Use
EA Maturity Model EAMM which is particularly useful to
Measurement evaluate the function of the architecture. It is designed for
Office of Completion governments, but it is general enough to be used by non-
Management & Use governmental organizations.
Budget Capability Results
Area Representation

EA Enabler Leadership
Representation People
Processes
Tools

The elements of evaluation: The 59 core elements are

distributed over the maturity stages, except for Stage 0, which
has no core elements allocated. The reason is that stage 0
merely represents the phase of making an organization EA
aware and the EA activity is mainly ad hoc and unstructured
Figure 6: Model of NASCIO - Enterprise Architecture Maturity Model
[15].The core elements represent the EA practices, structures, EAMM
activities and conditions. If they are performed or met
properly, an organization is able to advance to a higher
maturity state. The figure below represents the indicator « The levels of maturity: The maturity levels range from no
Provides capability to meet commitment» with its elements of documented EA (level 0) to a mature EA (level 5) with
evaluation. ongoing refinements and improvements. Each level contains
the elements that are relevant for that particular maturity level.
A certain maturity level can only be reached if all other
previous levels have been fulfilled. The five maturity levels
are shown in Figure 6.

The indicators: EAMM proposes eight indicators to evaluate

the enterprise architecture. The indicators are projected on the
six levels of maturity, they are presented below:

Administration: Governance Roles & Responsibilities

Planning: EA program road map and implementation plan
Framework: Processes and templates used for Enterprise
Architecture Blueprint: Collection of the actual standards and
Figure 4: The evaluation elements of the indicator « Provides capability to specifications
meet commitment»
Communication: Education and distribution of EA and
Blueprint detail Compliance: Adherence to published
standards, processes and other EA elements, and the processes
to document and track variances from those standards
Integration: Touch-points of management processes to the EA
Participation: Support of the EA Program throughout the
organization

The elements of evaluation: Each level of maturity contains

one or more elements for each of the eight indicators. We
detail below some indicators for level 1:
Administration: The need for committees to define the
standards and processes has been identified
Communication: The need to create greater awareness about
EA has been identified. Little communication exists about the
EA process or possible process improvements. Figure 8: Dynamic Architecture maturity matrix.
Participation: The organization has identified a need to make
staff throughout the enterprise aware of the benefits and For example the key area development of architecture has
concepts of Enterprise Architecture. EA awareness efforts are level A positioned at Scale 1. The key area “architectural tools”
informal and inconsistent. Some groups are unsupportive of has level A positioned at Scale 6. This means that level A for
the efforts and may cause unrest in the organization development of architecture should be reached before level A
of the key area architectural tools. This is because, according
C. Focus model to DyAMM, certain key areas should be developed first and
These models based on the idea that each indicator has its reaches a certain maturity, before other key areas should get
own specific levels of maturity. The overall maturity of an their attention [15].
organization is expressed as a combination of levels of
maturity indicators. The maturity model that we will study is The indicators: DyAMM defines 18 indicators for the
Dynamic Architecture Maturity Matrix — DYAMM [18]. evaluation of the maturity of enterprise architecture, we
explain them below: Development of architecture: The
approach to architecture development, varying from isolated,
autonomous projects to an interactive process of continuous
facilitation. Use of architecture: The way architecture is used:
merely as a conduit for information, as a means of governing
individual projects or even as a tool for managing the entire
organization. Alignment with business: The extent to which
Figure 7: Presentation of the focus maturity model the architectural processes and deliverables are in tune with
The Dynamic Architecture Maturity Matrix was developed what the business wants and is capable of. Alignment with the
by Marlies von Steenbergen [1]. It identifies 18 indicators as development process: The extent to which architecture is
you can see in the figure below, each one of these indicators embedded in the existing (business and IT) development
has its own trajectory of growth which is labeled with capital processes of the organization. Alignment with operations: The
letters of A to a maximum of D. extent to which architecture is both used in and built on the
operations and maintenance discipline. Relationship to the as-
The levels of maturity: Each indicator has a number of is state: The extent to which the existing situation is taken into
maturity levels. In these cells the letters A to D represent the account by the architecture processes and deliverables.
maturity levels for each indicator. The number of possible Roles and responsibilities: The distribution of responsibilities
maturity levels may differ for each key area, varying from two concerning both architecture processes and deliverables within
to four levels. This means that some key areas have two the organization. Coordination of developments: The extent
maturity levels (e.g. Relation to the as-is state), while others to which architecture is used as a steering instrument to
have three or even four maturity levels (e.g. Monitoring). In coordinate the content of the many developments that usually
total the matrix contains 54 characters, each reflecting a take place concurrently. Monitoring: The extent to which and
certain maturity level corresponding to a key area. Each the manner in which compliance of projects with the
maturity level (character) is covered by one or more of the architecture is guaranteed. Quality management: The extent to
137 assessment questions. which quality management is applied to the architecture
The positions of the characters of the maturity indicators practice. Maintenance of the architectural process: The extent
reflect the prioritization that is given in the maturity matrix. to which the architectural process is actively maintained and
improved.
Maintenance of architectural deliverables: The extent to which
and the manner in which the architectural deliverables are
kept up to date. Commitment and motivation: The extent to the EA are evaluated independently for each other, which are
which commitment is attained from and shown by the different compared to NASCIO and GAO where the indicators
organization. Architectural roles and training: The don’t receive an individual score, which creates the possibility
acknowledgement and support of the architectural roles and for DyAMM of identifying the individual scores for 18
the extent to which architects can educate themselves. Use of categories EA by giving the possibility of having more
an architectural method: The extent to which a (common) detailed analysis of maturity.
architectural method is used Consultation: The extent to
which communication among architects and between It still offers a score of additional total maturity founded
architects and their stakeholders takes place on a structural on the scale of the 14 levels and It shows the individual force
basis. Architectural tools: The extent to which architects are or misses maturity indicator, which means that the
supported by tools. Budgeting and planning: The extent to organizations can prioritize their resources for the indicators
which architectural activities are budgeted and planned which count more. Based on this method we propose three
types of evaluation elements (Questions, Check-list and
The elements of evaluation: Contrary to other models, AMM Simulations)
elements are simple, they are made from questionnaires yes /
B. The dimensions of the evaluation indicators of enterprise
no and they contain exactly 137 issues related to the EA. Thus,
architecture
the achievement of a level of maturity can be easily
determined by answering all the questions in a certain stage of Evaluation indicators of enterprise architecture are divided
maturity with yes, the table below provides examples of into 5 types:
evaluation questions. x The business architecture defines the business
strategy, governance, organization, and key
business processes.
TABLE 2: EXAMPLE OF QUESTIONS FOR THE EVALUATION OF THE EA USING
DYAMM x The data architecture describes the structure of
an organization's logical and physical data assets
Questions Response and data management resources.
Is exchange of best practices supported by the Y x The application architecture provides a blueprint
organization? for the individual application systems to be
Is frequent and systematic feedback on the Y deployed, their interactions, and their relationships
architecture process given by the organization? to the core business processes of the organization.
Are there any architecture tools in use that ensure N x The technology architecture describes the logical
the consistency between the various architecture software and hardware capabilities that are
artefacts? required to support the deployment of business,
data, and application services. This includes IT
IV. FROM MATURITY MODEL TO A MODEL OF infrastructure, middleware, networks,
EVALUATION communications, processing, standards, and so on.
A. Comparison of maturity models x Communication and deliverables defines the
awareness of the EA, the sharing of information in
According to the state of the art carried out concerning the
the organization and the pace and maintenance of
evaluation of the enterprise architecture we noted that there is
the documentation and the deliverables.
a lack of the evaluation methods, however there are several
We present in table 3 the classification of indicators according
methods in the world of industry concerning calculation of the
to the types defined above, after presenting the various
level of maturity of the EA which can be also used for the
indicators that we will use to evaluate the enterprise
evaluation and from the comparative study presented below
architecture and their types we present an evaluation model
we see that DyAMM method is most suitable for evaluation.
and the various concepts and terms of our approach.
The uses of EA categories are different for each of the
models in both, numbers and structure. DyAMM, NASCIO
and GAO have respectively 18, 8 and 4 EA categories. GAO
uses only four categories, but also differs in structure by
adding 15 additional sub categories. DyAMM covers by far
the most aspects with regard to EA categories. By using a
relatively large number of categories (with a broader scope on
EA)[15]. We will use the indicators of DyAMM to evaluate
the enterprise architecture.

Regarding the elements of evaluation, DyAMM has an

additional aspect of maturity, it proposes 137 questions which
are divided on the 18 indicators of the EA. The indicators of
 TABLE 3.EVALUATION INDICATORS AND THEIR TYPES Element specifies how to evaluate each indicator. It may vary
Indicator BA DA AA TA CM from one organization to another one. We introduce this
Development of architecture * artifact as an intermediate layer as not to limit the flexibility
Use of architecture * of evaluations. Each company sets its own elements to
Alignment with business * validate the evaluation indicators. We can use 3 types of
Alignment with the * * attributes:
development process
Alignment with operations * x Questions: We define a series of questions to
Relationship to the as-is state * * * * evaluate the indicator. The result of each element
Roles and responsibilities * can vary from 0 to 3.
Coordination of * x Checklists: Checklists are the quickest and
developments cheapest way to evaluate anything, including
Monitoring * enterprise architectures. Each evaluator must be
Quality management * * * * assigned a particular role and prepare questions
Maintenance of the * * * * relating to the position in question, for example,
architectural process might evaluate the architecture or effort in terms
Maintenance of architectural * of their role only. The affected metric is 0 or 3.
deliverables
x Simulations and prototypes: This method
Commitment and motivation *
consists in developing simulations, “What if”,
Architectural roles and *
and scenarios to validate the hypotheses and
training
conduct experiments about the existing
Use of an architectural *
method architecture. Sometimes this is the only way to
Consultation * deal with specific issues. The result of each
Architectural tools * simulation can vary from 0 to 3.
Budgeting and planning *
The difference between the 3 types of the elements is the type
C. Evaluation model of enterprise architecture of the attribute “result_element” that precise the response of
Stakeholders are people involved in developing or using the element: the “result_element” of checklist element is a
EA, and therefore have a direct relation with these criteria [5]. boolean result and the attribute “result_element” of question
Each stakeholder can participate to evaluate one or more and simulation elements is list of five responses and each
indicator and each indicator is evaluated by several actors. response is corresponding to one metric from 0 to 3.
There are two main groups of stakeholders in our model:
After developing and measuring the elements of the indicator,
x Direct reviewers: These represent direct users of the we calculate the value ’’value_indicator’’of each indicator as
indicators. As direct users of the metrics, they follows: Value_indicator = The sum of value of elements in
were involved as participants in the research and relation with the indicator / the number of these elements.
in the interpretation of the results. The attribute
"task_ reviewers" specifies the task of the D. Aggregation functions
evaluator in the organization.
In this section we precise the method of calculating the
x Indirect reviewers: These represent indirect users of
value of the result of evaluation of EA and :
the criteria, as they received the criteria
information as feedback on their work. These
EVAL_EA: the final result of evaluation.
were project analysts working on application
EVAL_DATA: the result of evaluation data architecture.
development projects, who were responsible for
EVAL_BUSINESS: the result of evaluation business
developing the enterprise architecture. The
architecture.
attribute "function_ reviewers" specifies the
EVAL_TECHNOLOGY: the result of evaluation technology
function of the Reviewer in the development of
architecture.
enterprise architecture in the organization. EVAL_APPLICATION: the result of evaluation application
architecture.
Each of these stakeholder groups is likely to have a different EVAL_COMMUNICATION: the result of evaluation
perspective on the utility of each indicator communication.
EVAL_EA = (EVAL_DATA + EVAL_ BUSINESS +
Metrics precise the scale measuring the attribute, the result is EVAL_TECHNOLOGY + EVAL_APPLICATION +
attributed to “value_element” specified in the element. It is EVAL_COMMUNICATION) / 5.
important to define the indicator and the attributes as well as
the belonging metric clearly [7]. In our approach we use 4
metrics (0: poor, 1: acceptable, 2: good, 3: very good)
EVAL_DATA = the sum of “value_indicator’’ of all data
architecture indicators/ the number of data architecture
indicators.

EVAL_BUSINESS = the sum of “value_indicator” of all

business architecture
indicators / the number of business architecture indicators.

EVAL_TECHNOLOGY = the sum of “value_ indicator” of

all technology architecture indicators / the number of
technology architecture indicators

EVAL_APPLICATION = the sum of “value_ indicator” of

all application architecture indicators / the number of
application architecture indicators.

EVAL_COMMUNICATION = the sum of “value_

indicator“ of all communication indicators / the number of
application architecture indicators.

V. CONCLUSION ET PERSPECTIVES

In this paper we have discussed a very important and

timely subject which is the evaluation of the enterprise
architecture. From the state of the art carried out we noted that
there exist several methods in the world of industry and of the
company for the calculation of the level of maturity of the EA
but there is a lack of the solutions of evaluation. For this
reason we proposed a model of evaluation of the EA while
basing ourselves on indicators, but we should not neglect that
the EA is carried out by stages (As-Is, To-Be, Plan of
transition) and we must take them into account on the
evaluation of the EA

x The evaluation of the current EA (As-Is) :

current architecture comprises a set of descriptions Figure 9: Enterprise Architecture evaluation model
which show the actual position of the organization in .
terms of missions, process and technological On the one hand we must consider these steps in our
infrastructures evaluation, on the other hand for the evaluation of any
x The evaluation of desired architecture (To-Be): architecture we must validate and verify because these two
desired architecture includes a set of descriptions principles represent the key success of any evaluation and EA
which show the desired future state of the should not be the exception.
organization.

x The evaluation of the transition plan: REFERENCES

The transition strategy is a plan that shows the time [1] Benjamin S., Analysis and Classification of Maturity Models in
required to transfer the organization of the current Enterprise Architecture Management
[2] Zachman J.A., The Framework for Enterprise Architecture, ZIFA
state to the desired state, the estimated effects of the report. 2003.
implementation of the desired architecture of the [3] Amir Hossein Amirkhni1, Seyed Aliakbar Ahmadi1, Ahmaderza Haghi
organization, planning the creation of the necessary Performance Evaluation of Enterprise Architecture Using Balanced
infrastructure, planning the implementation of the Score Card: A Case Study
[4] Zachman, J. A. (1987) A Framework for Information Systems
components of the desired architecture, and risk Architecture. IBMSystems Journal 26 (3), pp 276-292.
identification. Therefore, the evaluation of the [5] Moody D, Measuring the Quality of Data Models: An Empirical
enterprise architecture should also assess the Evaluation of the Use of Quality Metrics in Practice, 2006
components of the transition strategy. [6] Schekkerman J., Trends in Enterprise Architecture 2005: How are
Organizations Progressing? Institute For Enterprise Architecture
Development (IFEAD) report, 2005.
[7] ADDICKS J., ‘’Enterprise Architecture Dependent Application [15] Jurjen Schipper, The Dynamic Architecture Maturity Matrix
Evaluations’’, 2009. Assessment analysis and instrument validation 2002.
[8] West, D., Bittner, K., Glenn E. (2002). Ingredients for Building [16] Namazi , Mohammad , 2003, evolution of balanced evaluation ,
Effective Enterprise Architectures, 2002, The Rational Edge. presented in the 17th Iranian Accounting Seminar , University of
[9] Noran. An Analysis of the Zachman Framework for Enterprise Mazandaran , Babolsar
Architecture from the GERAM perspective, Annual Reviews in
Control, 27, 163-183 OMG, 2013. [17] Winter, R. and Fischer, R. (2006) Essential Layers, Artifacts,
[10] Brown T., The Valeur of Enterprise Architecture, ZIFA report, 2005. andDependencies of Enterprise Architecture. In EDOC Workshop on
[11] Salinesi C., Thevenet L., “Enterprise Architecture, des problèmes Trends in Enterprise Architecture Research (TEAR 2006), 17October
pratiques à l’innovation”, Nouveaux challenges dans les SI, Paris, 2008. 2006, Hong Kong (Society, I. C. , Ed), IEEE Computer Society Press,
[12] Van Steenbergen, M., Van den Berg, M., & Brinkkemper, S. (2007a). Los Alamitos, CA, USA.
An Instrument for the Development of the Enterprise Architecture [18] C. I. O. Council (1999) Federal Enterprise Architecture Framework
Practice. In (Cardoso, J.; Cordeiro, J.; Filipe, J., Ed.): International Version 1.1. http: //www. cio.gov/archive/fedarch1. Pdf.
Conference on Enterprise Information Systems (3). Pages 14–22. [19] National Association of State Chief Information Officers (NASCIO)
Berlin, Germany. 2007. Springer. USA:.Enterprise Architecture Maturity Model.
[13] Governance Accountability Office (GAO) USA:. A Framework for www.nascio.org/publications/documents/NASCIO-EAMM.pdf.
Assessing and Improving Enterprise Architecture Management.
[14] Steenbergen, M., Architecture Maturity Matrix.
http://www.swean.com/dokument/Architecture_maturity_matrix_Intro.
pdf