0% found this document useful (0 votes)
74 views40 pages

White Paper CICD

This white paper discusses how Continuous Integration and Continuous Deployment (CI/CD) enhances customer value creation in the financial services sector, particularly through Commerzbank's implementation. It outlines key practices and principles of CI/CD, emphasizing the importance of automation, cross-functional teams, and a shift towards a technology-driven approach. The document also anticipates future developments in agile software development over the next five years.

Uploaded by

danhuynh17102000
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
74 views40 pages

White Paper CICD

This white paper discusses how Continuous Integration and Continuous Deployment (CI/CD) enhances customer value creation in the financial services sector, particularly through Commerzbank's implementation. It outlines key practices and principles of CI/CD, emphasizing the importance of automation, cross-functional teams, and a shift towards a technology-driven approach. The document also anticipates future developments in agile software development over the next five years.

Uploaded by

danhuynh17102000
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 40

CI/CD

White Paper
Beyond Banking: How Continuous Integration & Continuous
Deployment (CI/CD) Accelerates Customer Value Creation
in the Financial Services Industry

The bank at your side


This White Paper was developed in a collaborative research project between
Commerzbank and the Business Engineering Institute St. Gallen.

Impressum

Publisher: Design and Layout


Commerzbank AG Steffen Rädle, Lil-Britt McBrien
60261 Frankfurt am Main, Germany
+49 69 136-20
info@commerzbank.com Image Credits
Logo by courtesy of Business Engineering Institute St. Gallen (page 2),
Group Services Unsplash: C. Käppeli (1) , A. Pettitt (21), A. Ismail (27), P. Richier (39);
Technology Foundations Shutterstock (30); Pixabay (16, 40)
Cluster CI/CD CHAMP All other images: Commerzbank

1
Beyond Banking: How Continuous
Integration & Continuous Deployment
(CI/CD) Accelerates Customer Value
Creation in the Financial Services
Industry
Insights on Commerzbank’s Holistic CI/CD Approach

Dear Reader,

The market has changed. Digitization is our experience and best practices. Toge-
in demand everywhere today. While in ther with various experts from
the past it was mainly about hardware in different companies and industries we
the financial industry, today it is primarily took a look into the future of CI/CD.
about software. And the way how to deli- I hope that you will enjoy reading this
ver software defines the success of every white paper and gain interesting insights
player in the market since today a bank is into the concept of CI/CD and how Com-
a software-driven company. Because at merzbank applies this concept to develop
Commerzbank, we accept that our DNA superior products for its customers.
has changed. We are now a technology-
driven innovation bank. Best regards,
The concept of Continuous Integration
and Continuous Deployment (CI/CD) has
gained momentum and changed the way
software is developed within the financial
services industry. The driving force behind
this momentum is acceleration which is
key to deliver high-quality and customer-
centric software products. At Commerz-
bank, we have started our CI/CD journey
around 3 years ago alongside with a dra-
matic organizational change to agility. We
are looking forward to sharing with you Florian Meiser (Commerzbank AG)

2
3
Content
Executive Summary 06

Introduction08

Continuous Integration & 09


Continuous Deployment –
What is it About?

CI/CD CHAMP Use Cases  18

Hypotheses About the Future 23


of Agile Software Development
until 2026

Summary and Conclusion 29

Appendix31

4
Executive Summary

In this white paper, we aim to describe how the software development process can be
accelerated by pinpointing the CI/CD concept: its cornerstones, how it affects customer
value creation in general and in financial services in particular, how Commerzbank is le-
veraging CI/CD to drive innovation and shorten release cycles, and which developments
we can expect in the upcoming five years in agile software development.
We concluded that CI/CD is a set of best practices and tools which enable organizations
to accelerate customer value creation, time-to-market, and the delivery of software qua-
lity. We would like to share with you the five key take-aways we identified during our CI/
CD journey and the research process of this white paper:

01 02
CI/CD accelerates customer value crea- The implementation of a CI/CD pipeline
tion in the financial services industry needs to follow a set of technical
The objective of CI/CD is to be able to daily release high- principles
quality software that creates value for the customer. At The CI/CD pipeline is the backbone of every CI/CD ap-
Commerzbank, we believe this especially for organizations proach. It reflects the automation engine alongside the
in the financial services industry which are still undergo- software development lifecycle and is governed by the
ing a radical shift driven by increasing competition, high technical principles of full automation through highly
regulatory requirements, structural low interest rates, and integrated tools connected via APIs. What`s more, a
fast changing customer needs, CI/CD is key to being closer standardized and scalable pipeline, high reliability and
to the customer. CI/CD enables greater automation in the availability, and a technology agnostic pipeline are incor-
whole software development and delivery process. This all- porated. Although an effective CI/CD pipeline is built upon
ows us to release daily, new digital banking solutions to our these principles, there is no one-size-fits-all approach to
customers and in turn results in a higher customer value. implementing a CI/CD pipeline. Since its setup is determi-
ned by company-specific internal processes, resources,
and infrastructure. The approach of continually gathering
feedback during the implementation phase and adjusting
the process accordingly is best-practice we discovered.
Furthermore, organizations should accentuate those tech-
nical elements in their CI/CD pipelines that fit their organi-
zational conditions best, in accordance with regulations to
successfully deliver value to their customer.

5
03 04
CI/CD is not just about tools and proces- An empowering CI/CD approach lever-
ses, it is also about teams that have end- aging the strength of cross-functional
to-end responsibility teams is key
A successful CI/CD approach significantly depends on Building on the foundation of agile software development,
giving teams end-to-end-responsibility of the software cross-functional teams comprising of development and
development lifecycle so that they can create software operational functions, alongside business and security
features without significant dependencies on other teams experts ensure the targeting of key business outcomes and
to accelerate speed, quality, and customer value creation 1. compliance with security standards throughout the CI/CD
This requires organizations to enable their teams technolo- lifecycle. We are convinced that “BizDevSecOps” 2 teams
gically and culturally with the right tools and methods. Our will be the new form of collaboration and co-innovation
experience also shows that the combination of a self-ser- and a decisive factor for organizations to deliver software
vice platform and availability of an ongoing technical and more quickly, safe and responsive to customer demand.
methodological support for teams, if necessary, lays the To implement this new team constellation, organizations
foundation for teams to take over end-to-end responsibili- need to empower their teams with the appropriate training
ty within the whole CI/CD software development lifecycle. and coaching solutions to bridge the several functions and
Although, organizations in the financial services industry enable them to carry on the different tasks throughout the
need to comply with specific regulations concerning the CI/CD lifecycle. We think that the alignment of software
separation of development and operations entities. Esta- development and business objectives in accordance with
blishing end-to-end responsibility of teams further accele- security standards and built upon shared business metrics
rates the CI/CD software development process. significantly contributes that the whole team. This enables
working towards the shared goal of accelerating the soft-
ware development process.

05
Everything as Code (EaC) as the new
norm, meaning automation via software
will be everywhere
We expect that the EaC development will accelerate in the
upcoming five years leading to the practice that everyt-
hing such as operations, infrastructure and configuration
management adheres to the same software development
practices 3. The EaC principle plays a central role for other
developments such as enhanced cross-functional collabo-
ration or a shift left approach of security, compliance, and
documentation. Additionally, EaC provides the technical
conditions for establishing a common CI/CD platform that
serves as single point of truth for data, documentation,
and tracking points that can be established to measure the
speed or waste in certain phases of the software develop-
ment lifecycle. Cross-functional teams using this CI/CD
platform in their daily business will use a common langua-
ge of code that enables collaboration and co-innovation
among the different functions. We defined EaC as one im-
portant component of our future CI/CD approach and will
follow a zero-documentation approach which ensures an
automated and compliant documentation processes along
the CI/CD lifecycle.

1
For more information, please see our learnings from CI/CD Use Cases on p. 18.
2
For a detailed explanation of BizDevSecOps, please see p. 24.
For a detailed explanation of software development practices, please see p. 17. 6
3
Introduction

As organizations continue their journey of accelerating solutions through the CI/CD “Commerzbank Hyper Acce-
their agile software development process, Continuous Inte- leration Master Pipeline” (CHAMP 4) for all other clusters 5
gration and Continuous Development (CI/CD) has become within Commerzbank,” says Florian Meiser (Commerzbank
crucial to deliver customer value faster [1]. With the need AG). Our objective is to support Commerzbank`s digital
to deliver quickly high-quality and customer-centric soft- transformation and the delivery of innovative solutions for
ware, organizations focus on CI/CD to develop and release customers by providing a fully automated self-service pipe-
software in short iteration cycles supported by cross-func- line that comprises all software development steps from
tional collaboration, automation, and tools. Today, CI/CD the initial idea to the final product while assuring compli-
forms the operational backbone in a modern DevOps envi- ance with regulations and enforcing the spread of modern
ronment while combining automation and cross-functional software development techniques.
collaboration from customer needs and business require-
ments to software development, testing, and operations to With this white paper, at Commerzbank we aimed at crea-
improve the overall software development process. ting a common understanding of the CI/CD concept, espe-
cially about the way how it affects customer value creation.
CI/CD as a set of practices and tools enables a faster time Moreover, we want to give you a detailed insight into our
to market through rapid, continuous integration of code holistic CI/CD approach and would like to share with you
changes, automated tests and continuous deployments our experience on our CI/CD journey so far – things we
of every code change to production after successfully have learned and our best practices that might hopefully
passing all quality gates [2]. The concept rests on the in- support you in your upcoming agile software develop-
terrelated cornerstones of the right tools, streamlined and ment endeavors. Additionally, we discussed with 14 subject
automated processes, and skilled people that if synchroni- matter experts from different industries in which direction
zed drive collaboration, innovation, and automation within agile software development is expected to heading to in
the organizations which ultimately improve customer the next five years. We will provide you with the result in
experience. “CI/CD offers organizations immense value: the form of five central hypotheses about the future of
faster release of products and services, stronger customer agile software development until 2026.
engagement, quicker feedback on new features, higher-
quality code, and ultimately higher customer value,” says Today, CI/CD has become an essential technology enabler
Jörg Oliveri del Castillo-Schulz (Commerzbank AG). for creating customer value, and we expect that the way in
which software is developed within companies will beco-
Specifically, the financial services industry might benefit me an even more decisive success factor. We hope that
from these advantages since the industry faces significant this white paper will provide insightful information on our
challenges due to the macroeconomic situation, fierce Commerzbank CI/CD CHAMP approach, contribute to the
competition, and high customer expectations [3]. However, discussion of the future of agile software development, and
the way in which CI/CD is applied in the financial service trigger new questions that will drive all of us towards new
industry is significantly impacted by national and interna- horizons of agile software development.
tional regulatory requirements, that need to be considered
when designing and introducing CI/CD in agile software
development process. Additionally, in the time of the digi-
tal transformation of the whole financial services industry,
a financial institution is a software-driven organization that
needs to use state-of-the-art technologies to provide cus-
tomers with an exceptional user experience.

At Commerzbank, we have started our work to build a


CI/CD Pipeline in 2018 to technically support our vision
of being the bank at the side of our customers using an
integrated approach. “With the commitment to provide
superior customer experience by offering digital, indivi-
dualized and highly valuable products and services, at
Commerzbank we have the mission to deliver end-to-end

4
For a detailed explanation, please see “How Commerzbank is living CI/CD” on p. 13.
7 5
For a detailed explanation refer to “Glossary”, page 36.
Continuous Integration &
Continuous Deployment –
What is it About?
As software continues to eat the world, organizations are tools [1], which will be connected throughout all essential
required to accelerate their software development process. software development steps [2].
In this context, CI/CD has become a key technological ena-
bler for the fast delivery of high-quality software features The concept of CI/CD forms the operational backbone of
that satisfy customer expectations. But what is CI/CD and DevOps practices and enables the automation and ac-
how does it benefit organizations? And which aspects are celeration of the whole software development process,
underlying? The following chapter aims at demystifying the from planning software over testing it to finally deploying
concept of CI/CD by providing the theoretical foundations software to production [11]. As seen in figure 1 CI/CD is
of this concept including its cornerstones and possible composed of two interrelated value streams – a continuous
business impact. Additionally, we explain why CI/CD is one integration and continuous deployment cornerstone cover-
crucial technology foundation for Commerzbank’s agile ing the respective activities of the software development
transformation and how we leverage this set of technical process.
practices to maximize value for our customers.
In accordance with the overarching idea of agile software
CI/CD Explained development, CI facilitates merging of software chan-
CI/CD has its roots in agile software development [1], ges from multiple contributors to a shared source code
which aims at a software development process being lean, repository frequently, often many times a day [12]. After
collaborative, goal-oriented, and responsive to change and each merge, the chunk of code is automatically built and
customer feedback [4]. In essence, the objective of agile tested to ensure the code is still functional after each
software development is to produce high-quality soft- code change, to avoid potential integration issues and to
ware that best satisfies customer needs [4]by cultivating guarantee that it is safe for other developers to write code
and promoting organizational capabilities such as sensing, on top of the new changes [13]. Companies that start with
learning, adaptability, resilience, speed, innovation, colla- introducing CI usually face two challenges comprising
boration, and efficiency [5]. One approach to operationali- changing the habits of developers to do smaller frequent
ze these capabilities is following the paradigm of DevOps, code commits and ensuring the needed automation to
which aims at bridging the gap between the functions of support the software development flow and reduce the
development and operations and aligning their activities time to integrate and validate code changes. The validation
across the organization [2][6]. On the one hand, DevOps itself comprises not only the automated unit tests but also
combines functional skills from business, software de- the automated static code analysis and security scans to
velopment, testing, and operations in one integrated team early detect code vulnerabilities. The practice of shifting
improving cross-functional collaboration and communica- this validation while performing unit or integration tests
tion [2]. On the other hand, it empowers teams with end- and code quality checks early in the software development
to-end responsibility which increases the ability to faster process is referred to as “shift left” and can result in signifi-
deliver high quality software. cant cost reduction as well as in increased software quality
since potential pitfalls in the software code are identified
This stands in contrast to the formerly applied software de- early during the development process [12].
velopment processes, where functional and informational
silos or step-by-step sequential processes lead to long de- CD picks up where CI ends comprising the software de-
velopment times: “in enterprise software 1-2 years, in new velopment process steps of deploying and releasing to
microprocessors 2-4 years, in automobiles 3-5 years, and in production. Within the CD phase the software artefact is
aircraft a decade” [7]. Before DevOps, software was often automatically launched and distributed to end-users after
developed isolated as a single piece and, after finishing it has successfully passed automated tests to validate if
the complete development, tested as a whole [8]. Now, changes to a codebase are correct and stable for immedia-
crucial activities, such as testing, are parallelized with the te automated deployment to the production environment
development phase [9] and software is developed in itera- [14]. In living customer centricity, it is important that new
tive cycles, building on intermediary results and feedback products or features are deployed smoothly and without
loops [10]. Hence, possible pitfalls can be detected faster, delay. CD complements the process stage of CI leading to
and solutions can be implemented more quickly. Another a faster and smoother release of new software and makes
important strength is the increasingly integrated technolo- it possible to continuously integrate customer feedback
gy stack comprising development, testing, and integration early in the product lifecycle.

8
Feedback Loop

Figure 1: The CI/CD Lifecycle and its Process Steps

According to the CI/CD lifecycle view, all steps of the soft-


ware development process are continuously integrated
and automated to ensure a seamless and holistic software The Cornerstones of CI/CD
development process. The automation of the CI and CD The concept of CI/CD rests on the cornerstones of the
process steps is typically referred to as “CI/CD pipelines”, right tools, streamlined and automated processes, and
an analogy of traditional factory product automation skilled people, as illustrated in figure 2, to succeed by
pipelines. CI/CD pipelines are often complemented by an continually providing value to the customer. It is imperative
integrated customer feedback loop which allows orga- to take a holistic view when implementing CI/CD since it
nizations to measure customer reactions, collect metrics is the multi-dimensional aspect of all three cornerstones
and actionable insights from software in production. This that drive collaboration, innovation, and automation. In the
feedback loop closes the CI/CD cycle and ensures quality following we want to further describe the characteristics of
of the software product or feature beyond deployment and each cornerstone and how they interact with each other.
a smooth customer experience.

1. DevOps Teams
“Overall, the concept of CI/CD is an important enabler Excelling in CI/CD requires the right people with the ap-
for organizations to accelerate the software develop- propriate skills and mindset as it applies to the successful
ment process leading to a potential reduction of the implementation of almost every agile software develop-
time-to-market of new products and features to cus- ment practice [16] accumulated in one DevOps team. A
tomer,” says Sabine Vigelius (Commerzbank AG). In this recent study of the DevOps Institute [17] elaborates that
context, the benefits of CI/CD are manifold. First, CI/CD people skills such as collaborating, problem solving, know-
leads to a shorter time-to-market since development teams ledge transfer and adaptability are key for success in living
can release software builds faster due to incremental soft- DevOps and CI/CD. Additionally, individuals with an open
ware development [1]. Second, CI/CD ensures an early and innovative mindset are needed to further develop
detection and fixing of software vulnerabilities and defects existing products and services to deliver customer value.
due to the shift left approach which ultimately results in According to the State of DevOps Report 2021 [18], orga-
higher code quality [15]. Third, the automation inherent in nizations that have successfully adopted the concept of CI/
the CI/CD lifecycle ensures that crucial process steps of CD ensure that their DevOps teams have strong identities,
the software development process are less prone to human clear responsibilities with a high degree of autonomy in
error and easier to manage by the team [16]. Fourth, CI/ their own function as well as distinct interaction ceremo-
CD reduces the cost of communication as it supports the nies and communication channels with other teams.
DevOps practice of aligning cross-functional teams. Fifth,
CI/CD allows for integrating customer feedback early in
the software product lifecycle enabled by short feedback- A corporate culture that promotes a workforce living CI/CD
loops [1]. principles and successfully applying the technical practices

9
of the CI/CD concept is based on a collaborative environ- lize full potential of an organization’s CI/CD pipeline. Since
ment [19]. Another important aspect is to impose end-to- agile software development is built on fast feedback loops
end responsibility among DevOps teams that work with and constant collaboration between different stakeholders,
CI/CD practices and tools ensuring that all team members the processes within the CI/CD pipeline need to foster this
together are fully accountable for the software artefact environment [21]. The process along the CI/CD pipeline
across the whole software development cycle. This sup- typically comprises the steps of plan (setting up the soft-
ports a collaborative team spirit while providing the possi- ware project with its requirements), code (the stage where
bility for each team member to contribute his or her ideas developers implement the IT product requirements), build
and strengths during the whole software development (the stage where the IT artefact is created), test (the stage
life cycle [17]. Last but not least, encouraging continuous where code is tested), deploy (the stage where code is
learning. This helps us support the development of DevOps deployed to production), and release (the final delivery of
culture. This can be achieved by promoting employees to the software product to the customer) [2]. A powerful CI/
embrace failure and learn from it by creating a psychologi- CD pipeline that increases speed and quality of the who-
cal safe environment through meaningful and constructive le software development process is based on an optimal
feedback [17]. orchestration and seamless connection of the beforemen-
tioned process stages.

Developing an appropriate organizational culture that


promotes the required CI/CD skills and mindset requires A relevant aspect in this context is the shift left approach 6
cultural change. In this context, the ability of employees to that leads to a parallelization of process steps within the
embrace change is one important, but challenging factor. software development workflow since instead of sending
A study of Atlassian and CITE Research [20] states that multiple changes to a separate test team, DevOps teams
almost 35% of the interviewed professionals consider the working with CI/CD practices and tools perform continu-
corporate culture change as well as the lack of required ous automated tests throughout the whole development
skills as the most pronounced barriers to realize the full process [11]. This allows to detect and fix defects early in
potential of CI/CD. Thus, the DevOps teams cornerstone the process resulting in an improvement of code quality
is a decisive factor for a sustainable CI/CD transformation and higher velocity in software delivery [22].
since it significantly determines the application of CI/CD
principles and technical practices.
The integration of CI/CD principles related to the corners-
tone Software Development Processes leads to an iterative
2. Software Development Processes process workflow that offers possibility for continuous
Besides the DevOps teams cornerstone, there is the need testing and feedback and thus to the consequent imple-
to establish the right software development processes mentation of the paradigm of agile software development.
promoting agile software development techniques to rea- The streamlined process along the CI/CD pipeline lays

6
See sub-chapter “CI/CD Explained” for a detailed explanation. 10
the foundation for teams to collaborate and communicate might add its special features to the specific process stage.
throughout the CI/CD process workflow to maintain align- It is important to create interfaces between separate tools
ment, velocity, and quality [17]. Moreover, teams using a to enhance the seamless and automated interaction bet-
CI/CD pipeline benefit from more freedom in the way they ween them. Moreover, each tool comes with its very own
conduct business and collaborate since the process along- limitations that it brings to the whole process which need
side the CI/CD pipeline reduces processual limitations [18]. to be considered to avoid potential negative effects on the
functionality of the CI/CD pipeline [22]. Additionally, too
many tools used in the CI/CD pipeline increase complexi-
Although, adapting processes with sight to the CI/CD pipe- ty which could negatively impact their usability as well as
line offer valuable benefits, this also might come with some accuracy. These factors might negatively affect the smooth
costs since changing processes from existing well-esta- process workflow of the CI/CD pipeline and could there-
blished processes to CI/CD-ready processes requires an fore decrease quality delivered to the customers. Hence, it
extensive change effort within the whole organization [18]. is important to find the right balance between the number
In a running organization it would be almost impossible to of tools with their special features for each CI/CD process
stop the existing process approach and start completely stage and ensuring manageability of the whole tool chain.
new from scratch. In addition, it is a costly and time-con-
suming process to integrate an organization to a CI/CD
pipeline since the long-grown technology stack is charac- Another important aspect in the context of CI/CD corners-
terized by a high degree of complexity [18]. tone CI/CD tools is the question of build or buy. Although,
the open-source community is gaining more and more
traction nowadays regarding powerful DevOps Tools and
3. CI/CD Tools the development in cloud-computing enables a shift of
The process on the CI/CD pipeline is supported by various the process towards a more powerful environment [21],
tools that ensure each process step is seamlessly integra- at Commerzbank, we believe that an organization should
ted within the software lifecycle and automated to reach assess its strength and weaknesses before choosing a CI/
the maximum acceleration potential of the CI/CD pipeline. CD toolchain. An off-the-shelf tool or even a pre-made
These tools connect different systems and ensure the ref- pipeline including all necessary tools might be simpler
lection of any change in the source code across different than building a tool itself, but it might sacrifice flexibility.
environments [21]. Additionally, the cloud native tools of In our experience, organizations can combine the best of
the CI/CD pipeline technically support the agile software both approaches individually deciding about which tools
development process by accelerating the building of new should be collected from a vendor or if a tool built by the
applications and integration of user feedback and by con- organization itself will more fit the individual organizational
tributing to dismantling borders between development and requirements. We believe this is more pronounced in the
operation teams [23]. Each tool used in the CI/CD pipeline Financial Services Industry since financial institutions have

● Skills such as collaborating, problem solving,


knowledge sharing and adaptability
● Innovative and open mindset
● Corporate culture promoting change and
collaboration
DevOps
Collaboration Teams Innovation

● Fast feedback loops and constant Software ● Seamless connection of individual


collaboration between stakeholders CI/CD tools on the CI/CD tool chain
Development
● Shift left approach to test software Tools ● Appropriate tools for different
early in the lifecycle Processes process stages
● Iterative process workflow ● Supporting the right fit of tools for
the organisation

Automation

Figure 2: The Three Cornerstones of CI/CD

11
oftentimes specific requirements due to their long-grown on is key. Releases should happen daily, ideally on-de-
software stack and regulatory parameters that need to be mand, whenever a new feature is ready. Only with this
considered. Thus, the possibility that financial institutions approach we can realize our goals - deliver delightful
can simply leverage the advantages of a pre-made CI/CD software products to customers and get ahead of
pipeline including a ready-to-go tool chain might be illusio- competition.
nary and therefore they usually focus on building their CI/
CD pipelines integrating relevant tools on their own. • Next, the Pipeline: It is mostly a technical solution that
executes CI/CD at Commerzbank and enables teams
to deliver to production with high quality. We provide
To summarize, the realization of the whole potential pro- various tools in each stage of the software develop-
vided by CI/CD pipelines requires a holistic view of this ment process - from integrated development environ-
practice. At Commerzbank, we believe that the reflection ments (IDE) to deployment tools. Since all tools are
of the CI/CD components and their interconnectedness in seamlessly integrated in one pipeline, synergy effects
the respective CI/CD endeavors ensures that CI/CD leads are achieved. On top of this, at Commerzbank we offer
to a more collaborative culture, an innovation-driven mind- coaching solutions - training teams in efficiently using
set, and a higher-level of automation. In the following we the tools and methods. This is complemented by a set
will present how we are living CI/CD at Commerzbank and of internal policies that govern the software develop-
in which way we have integrated the CI/CD cornerstones. ment process, implemented as a set of documents and
tools that guide users in fulfilling regulatory require-
ments.
How Commerzbank is Living CI/CD
The way in which the DevOps paradigm and the CI/CD • Lastly, a Master (Pipeline), with all the breadth of tools,
concept are applied in the financial services industry is sig- platforms, target environments, middleware compo-
nificantly impacted by national and international regulatory nents, programming languages, frameworks and high
requirements. Regulators such as the European Central national and international regulatory requirements
Bank oversee not only the business model of financial insti- coming at Commerzbank, we think that it is crucial
tutions but also impose regulations on their IT Infrastruc- that there is one single master pipeline that fulfils all
ture and thus the banking specific software development. those requirements. This pipeline leads to a high level
These regulatory requirements need to be considered of confidence since everything runs smoothly if well
when financial institutions apply CI/CD practices and prin- orchestrated.
ciples, build their CI/CD pipelines as well as set up relevant
methods and processes in their DevOps teams.
Since CI/CD CHAMP is focused on the acceleration of
Commerzbank’s delivery organization, we prefer to employ
At Commerzbank, we started with an integrated approach the analogy of a racetrack in Formula E. As in Formula E,
to build our CI/CD pipeline in 2018 as a technology enabler a successful CI/CD pipeline is about speed, engineering
of our agile transformation journey to implement our con- discipline and outstanding teamwork to facilitate the com-
tribution of being “the bank at your side 7 ” . “What drives plete automation of the delivery process from the initial re-
us is our commitment to provide a superior customer quirement to delivering the final product to our customers.
experience by offering digital, individualized and highly As seen in figure 3, CI/CD CHAMP is composed of three in-
valuable products and services,” says Dr. Alena Kretzberg terrelated services: (1) Frameworks assuring the complian-
(Commerzbank AG). Today, Commerzbank has established ce with regulatory requirements during the whole software
the cluster “Continuous Integration/ Continuous Deploy- development process; (2) Automated Pipeline comprising
ment CHAMP” (CI/CD CHAMP) as a technology foundation all technical steps and tools from development over testing
for the fast and automated delivery of new functionalities to production, and (3) DevOps Coaching promoting state-
to the organization’s customers. of-the-art development methods and practices within the
cross-functional agile teams.

The mission of the cluster CI/CD CHAMP is to deliver


end-to-end solutions through the CI/CD “Commerzbank It is important to mention that the interplay between the
Hyper Acceleration Master Pipeline” (CHAMP) for all other three CI/CD CHAMP services makes our CI/CD approach
clusters within Commerzbank. “Our objective is to signi- so unique. “We as Commerzbank offer the CI/CD CHAMP
ficantly accelerate the development of new digital and automated pipeline with its tools and integration as a
customized solutions for our private and corporate cus- standard for all Clusters in the Delivery Organization so
tomers,” says Damian Waszak (Commerzbank AG). CI/CD that DevOps teams working on new software features can
CHAMP is our approach at Commerzbank to realize CI/CD flexibly configure their own pipeline template based on
and DevOps. As with every acronym each word has certain their individual needs,” says Fonlinda Frasheri (Commerz-
meaning. Let us take a moment to explain it: bank AG). It is the aspiration of the CI/CD CHAMP team to
provide a CI/CD pipeline that allows for the necessary flexi-
• First of all, it is Commerzbank, it is our way of imple- bility to be adapted to diverse IT application architectures.
menting CI/CD and DevOps. We are not claiming that Additionally, the CI/CD CHAMP team gives technical and
this is how everyone should do it. But we think this is methodological support and guidance during the configu-
the best way how it works for us considering Com- ration and final working phase with the CI/CD pipeline at
merzbank’s history and objectives. any time.

• Then there is Hyper Acceleration: We want to support


DevOps teams to speed up, being able to deliver more To provide you with more insights about how Commerz-
and more financial features to Commerzbank’s private bank lives CI/CD on a daily basis, we would like to explain
and corporate customers. As this objective should be the three services of CI/CD CHAMP in more detail:
ambitious, at Commerzbank, we think hyper accelerati-

7
Our proven brand promise “Commerzbank. The Bank at your side” puts our brand posi-
tioning in a nutshell, describes what we stand for and what we promise to our customers. 12
CI/CD

Frameworks Automated Pipeline DevOps Coaching

Tool supported end-to-end Fully automated CI/CD State-of-the-art develop-


documentation process pipeline covering all tech- ment practices and coa-
to comply with regulatory nical steps from develop- ching to further strengthen
requirements during the ment to testing and pro- DevOps technical and
whole software process. duction. mindset excellence.

Figure 3: Services of Commerzbank Hyper Acceleration Master Pipeline (CHAMP)

• CI/CD CHAMP Frameworks: As in a race situation The PO-Cockpit serves as the regulatory single point of
having the necessary safety equipment, for instance a truth for any audits and controls and provides an overview
racing helmet or racing suit, is crucial when leveraging of all CI/CD endeavors. Since the required documentation
the maximum speed of your racing car while knowing depends on the planned software solution, the PO-Cockpit
that all necessary conditions are met to bring you allows employees to tailor the documentation workflow ac-
safely to the finish line. Like in our racing example, the cording to the software artefact type. Additionally, the tool
service “Frameworks” provides our DevOps teams with offers interfaces to other documentation systems ensuring
a tool-supported end-to-end documentation process that DevOps teams can navigate through the workflow
that covers the entire IT product development work- using one single tool. The PO-Cockpit allows users to tailor
flow to ensure compliance with regulatory standards the required documentation to their respective software
while ensuring our DevOps teams can fully focus on development endeavor using a standardized set of questi-
developing the right software for our customers. This ons to automatically generate the needed documentation
added value is achieved by two interconnected solu- template. Another feature is the embedded delta docu-
tions: the Commerzbank Delivery Guide (CDG) that mentation function that allows to use existing documenta-
contains a description and relevant documentation re- tion templates of the completed projects if the respective
quirements for each step in the software development users want to document a change or new feature of the
process and a Product Owner Cockpit (PO-Cockpit existing endeavor. Moreover, the PO-Cockpit ensures that
CHAMP) that guides the Product Owner (PO 8) and the documentation contains all necessary approvals. For
other involved colleagues through the development instance, the tool automatically sends the documentation
workflow and necessary documentation according to template to the respective responsible person to be com-
the CDG (as seen in figure 4). pliant with the double check principle ensuring security
standards. Additionally, the technical illustration of the
CDG in the PO-Cockpit serves as a guiding tool throughout

13 8
For a detailed explanation refer to “Glossary”, page 36.
all process steps of the software development workflow frameworks, and integrated development environments to
while facilitating the management of documentation re- developers to implement the user stories.
quirements through transparent navigation. DevOps teams
always have an overview of the next process step and how 3. “Source Code Control” provides a central source code
they can find the right balance between accelerating the repository, where changes from all software developers
software development process and documentation. can be stored, merged, and reviewed. Each code change is
tracked to ensure traceability.

• CI/CD CHAMP Automated Pipeline: At the heart of CI/ 4. “Build” tools enable an automated build process with
CD CHAMP is our automated pipeline, the racing car, each commit, such as change in source code repository.
which turns power into motion, resulting in a signifi- During the build process step, automated unit tests, func-
cant acceleration of Commerzbank’s software delivery tional tests, and code analysis checks are executed ensu-
process. The objective of the automated pipeline is to ring that the code is of high quality before being integra-
support DevOps teams in their CI/CD endeavors with ted.
the necessary technical tools and consulting on how
these tools can be applied to implement their envisio- 5. “Code Analysis” represents another quality gate for the
ned products or features to enhance the experience of code artefact created in “Build” comprising an automated
our private and corporate customers. On the technical checkup of the code for programming errors and adher-
tool side, the automated pipeline offers different tools ence to internal policies, e.g., programming guidelines. It is
currently comprising of up to 15 different applications used to detect outdated, insecure, or error-prone depen-
with partially automated interfaces along the CI/CD dencies used in the software project. In addition, security
software delivery workflow from the initial require- scans are conducted to detect vulnerabilities.
ment to production.
6. In the step “Delivery”, the code artefact is automatically
As seen in figure 5, the technical tools support DevOps handed over to the artefact repository and archived secu-
teams to successfully navigate on the CI/CD CHAMP rely.
racetrack completing all development stages towards the
finish line of deploying the final product to production. The The next steps on the CI/CD CHAMP racetrack are part of
first six stages comprise tools used to realize continuous continuous deployment:
integration:
7. “Configuration Management” represents a solution for
1. “User Story Design” which provides POs, using agile soft- the configuration and management of the stage specific
ware development methods, such as Scrum or Kanban 9 , configuration parameters.
with an automatically created tool space to document the
requirements for their product visions in the form of user 8. “Deploy Test” provides tools to support DevOps teams
stories. with test quality criteria (e.g., functionality) of artefacts in a
target environment with a low number of manual tests and
2. “Coding”, provides standardized functions, libraries, a central overview of all test cases. These tools enable a
full automation of the deployment process.

Commerzbank Delivery Guide


Description of the whole Software Delivery Pro-
cess including necessary documentation to ensu-
re regulatory compliance

PO-Cockpit
Tool that supports the Product Owner and
other colleagues through the documentation
process based on regulatory requirements

Figure 4: Solutions Provided by CI/CD CHAMP Frameworks

9
For a detailed explanation refer to “Glossary”, page 36. 14
CI/CD

Build Code Analysis Delivery Configuration Management Deploy Test

Source Code Control Coding User Story Design Deploy Prod

Figure 5: Steps of the CI/CD CHAMP Automated Pipeline

9. “Deploy PROD” provides the appropriate tools for an teams using the CI/CD CHAMP pipeline are provided with
automated release of the artefact in the target system. reusable templates for all included tools that can be tailo-
There are two conditions that needed to be considered for red to the specific project needs leading to a simplification
the design of the automated pipeline. First, the regulatory of the whole software delivery process. Another area in
requirement of “Segregation of Duties” (SoD), that requi- which DevOps teams benefit from CI/CD CHAMP is the
res that the person changing the code of an application self-service aspect of the pipeline. Traditionally, DevOps
cannot bring the same change live, effectively separating teams that would like to use multiple CI/CD tools, would
the functions of development and operations, influenced have to talk to multiple internal teams that manage those
the way the deployment process was automated. At Com- tools leading to time lags in granting access and configura-
merzbank, we established safeguard measures to ensure a ting user rights. Therefore, at the core of the CI/CD CHAMP
consisting SoD throughout the entire deployment process. pipeline is a self-service platform where all required tools
Second, additional complexity of deployments arose from can be requested on demand, and DevOps teams can get
the extent of middleware, programming languages, and access to them without delay which increases the overall
cloud technologies used at Commerzbank. To ensure CI/ satisfaction of DevOps teams that are more in control of
CD CHAMP is as open and extensible as possible, the CI/ their software development environment.
CD CHAMP Automated Pipeline can deploy to all of those
target systems. Additionally, the user experience for de- • CI/CD CHAMP DevOps Coaching: Besides the techni-
velopers and operators is the same, no matter what the cal factors, the success of a racing team also depends
underlying target is. on the skills and execution of the driver. A successful
The automated pipeline offers several advantages to fur- racing driver who leverages the full potential of his
ther accelerate Commerzbank’s CI/CD activities. DevOps or her car, needs an experienced pit crew as well as a

15
team of racing strategists which support him or her off at further strengthen the DevOps mindset of agile teams.
the racetrack on the pit lane. We think that also applies The selection of the appropriate training and coaching sec-
to our CI/CD pipeline since a successful application of tions is based on the individual need of the respective agile
all technical features requires the best possible metho- team as well as existing challenges they face. Therefore,
dological and cultural support of DevOps teams. The- CI/CD CHAMP DevOps Coaching is highly individualized to
refore, the CI/CD CHAMP service “DevOps Coaching” support agile teams in the best possible way in their daily
offers training and coaching for modern software business. The DevOps teams benefit from this approach
development methods to further establish software in several ways. First, they can leverage new practices and
craftsmanship within Commerzbank’s cross-functional skills to further increase their software quality through a
agile teams. The training and coaching solutions inclu- shift left approach. Second, they are optimally prepared to
de a two-stage procedure comprising “DevOps Techni- use our CI/CD CHAMP pipeline faster and easier to imple-
cal Excellence” to further strengthen agile techniques ment their intended solutions and features.
and practices within the DevOps teams and “DevOps
Agile Empowerment” to support DevOps teams in
applying and living the DevOps concept. Since the “We believe that our CI/CD CHAMP pipeline offers a good
successful execution of DevOps depends both on practice of how organizations can holistically design their
technical excellence and a collaborative and inclusive CI/CD pipelines leveraging the CI/CD cornerstones of
mindset, the focus of DevOps Coaching is on offering DevOps Team, Software Development Process, and CI/CD
training and coaching solutions comprising these two Tools. Although, we successfully integrated CHAMP in our
interrelated components as seen in figure 6. daily business, we are still working with high intensity to
bring it to the next level,” says Florian Meiser (Commerz-
bank AG). We are driven by the feedback of the DevOps
On the methodological side, the component DevOps teams and Commerzbank’s customers – because our
Technical Excellence provides trainings comprising state- software is just as good as they think it is. Like teams in
of-the art methodological strategies (e.g., Value Stream the Formula E are constantly searching for improvement,
Mapping), software development practices (e.g., Test we are working rigidly to further innovate CI/CD CHAMP
Driven Development, Continuous Integration, Clean Code) and its services. Please have a look at chapter 4 where we
and advanced testing techniques (e.g., Behavior Driven share our new features and vision for CHAMP 2.0 with you.
Development, Integration Testing). On the other side, these
methodological trainings are complemented by DevOps
Agile Empowerment coaching in which a DevOps coach
accompanies DevOps teams beyond the respective trai-
ning sessions to support them in adopting the practices in
their daily business. Additionally, the coaching sessions aim

Component 1 Component 2
DevOps Technical DevOps Agile
Excellence Empowerment
• Provide training for state-of-the-art • Support agile teams to adopt a
practices related to technical agility DevOps mindset
• Leverage DevOps methodologies to • Accompanying agile teams to further
support agile teams in daily business strengthen a DevOps culture

Establish agile technical practices to Adopt a DevOps mindset to master


master the technical side of agility DevOps practices in daily business

Figure 6: Components of Commerzbank’s CI/CD CHAMP DevOps Coaching

16
CI/CD CHAMP Use Cases

At Commerzbank, we would like to give you a deeper for restructuring the API deployment process to enable
insight into how CI/CD CHAMP and its three interrelated process speed, scalability, and security. The resulting pro-
services support the teams of our Delivery Organization cess artefacts were delivered to the API Banking cluster
in their daily business. Hence, we are proud to share with that adjusted these artefacts accordingly. Although, the
you three of our CI/CD CHAMP use cases as depicted in DevOps team from API Banking and CI/CD CHAMP had
figure 7 to explain the implementation of the different CI/ separate tasks to adjust the API deployment process, they
CD CHAMP solutions and how our CI/CD CHAMP team both worked hand in hand. “The collaboration with the
collaborates with their internal customers. In addition, we colleagues from CI/CD CHAMP was excellent during the
want to share our learnings and experiences on our CI/ whole process. Our weekly meetings and the possibility to
CD journey so far. Thus, we provide you at the end of the always contact one of the colleagues highly contributed
chapter with our best practices on how to master the im- to the success of our project,” says Volker Sulzbach (Com-
plementation of CI/CD in a large corporation. Further, we merzbank AG).
are looking forward to opening up the discussion in the CI/
CD and DevOps community to also hear your learnings and
best practices. During the implementation of the target API deployment
process the joint team also faced some challenges. For
instance, the tasks related to the adjustment of the API de-
CI/CD CHAMP Use Case 1: Commerzbank API Banking ployment process were initially seen as additional effort for
the API Banking unit. However, the clear identification of
Commerzbank’s Application Programming Interfaces (API) milestones and the incremental delivery of improvements
Banking cluster enables co-creation of valuable solutions in the deployment process, continuously enabled the team
with business partners, customers and FinTechs. In essence to speed up deployment, which in turn did free up time
APIs are technologies that enable communication between for other tasks and ultimately lead to the mitigation of the
IT-systems resulting in a closer collaboration among diffe- initial problem.
rent parties resulting in better solutions for our customers
[24]. The API Banking cluster is responsible for providing
a platform for managing and operating the APIs of Com- Additionally, API Banking was using a test automation
merzbank and was in 2018 one of the first cases where CI/ tool, that was not included in the setting of Commerzbank
CD CHAMP was used to enable a faster software develop- and hence not part of the standardized, automated CI/CD
ment process while further increasing the software code CHAMP pipeline. Accordingly, even though the deploy-
quality, both factors being crucial for delivering APIs. ment time could be reduced significantly, the high quality
of the releases needed to be ensured. To solve this issue,
CI/CD CHAMP supports the cluster API Banking by crea- the CI/CD CHAMP team implemented a new interface
ting an automated pipeline in close collaboration, provi- between the automated pipeline and the tool API Banking
ding the appropriate technical tools and consulting, as well already used for integration tests and demonstrated that
as by offering DevOps coaching comprising methodologi- the automated pipeline can be extended and individually
cal and mindset training components. configured based on the needs of the users. Through this
interface the automated CI/CD CHAMP pipeline triggers
test automation and reads result reports, ensuring only
CI/CD CHAMP Automated Pipeline high-quality releases are deployed to the production en-
Initially the API deployment processes were carried out vironment.
manually which led to challenges regarding a prompt
delivery of the API and scalability. Complexity arose from
the fact that a relatively small team had to deal with a The cluster API Banking benefitted from the automated CI/
plethora of various APIs provided daily by all other soft- CD CHAMP pipeline in different ways. “The automated API
ware teams within Commerzbank. Thus, the API Banking deployment process is a success for our Cluster. Deploy-
unit partnered with the automated pipeline team of CI/CD ing an API has become much easier and faster compared
CHAMP to further improve the API deployment process. to the initial process,” says Christian Pfaff (Commerzbank
To identify potential for improvement, the API deployment AG). As a result of adjusting the API deployment process,
process was analyzed in detail according to the methodo- the time needed to deploy an API in production was re-
logy of Value Stream Mapping (VSM 10) . The design of the duced to 15 to 20 minutes assuming a well-designed API.
target API deployment process allowed the team to derive The greatest benefit from this optimization comes from
milestones that needed to be achieved to improve the the frequent re-deployments that are necessary for the
initial process. The team of CI/CD CHAMP was responsible usual minor changes. The high level of automation not only

17 10
For a detailed explanation refer to “Glossary”, page 36.
Applied CI/CD CHAMP Services

Frameworks Automated DevOps


CI/CD Use Cases Highlights
Pipeline Coaching

API Banking Accelerating the API development process and


further improving agile working methods

Big Data & Onboarding of a Google-Cloud use case on CI/CD


Advanced Analytics CHAMP to enable an on-premise development
process for new products

CCB Security Enhancing release flexibility of new products as well


Frontend as enabling earlier testing and faster feedback

CI/CD CHAMP service completely applied CI/CD CHAMP service almost completely applied CI/CD CHAMP service partly applied No CI/CD CHAMP service applied

Figure 7: Overview Commerzbank CI/CD CHAMP Use Cases

provided scalability but also a high software quality since ment session. “The full-time training sessions gave us the
the reduction of manual steps makes the whole setting less opportunity to reflect and focus on specific areas of our
prone to errors. The new process also increased the usabi- daily work with an outside-in perspective. As a result, we
lity of the API mid-tier application deployment process sin- received valuable feedback about which additional agile
ce developers can automatically create new pipelines for software development practices could support us,” says
API applications via a self-service tool. The team around Akli Amar-Youcef (Commerzbank AG) since the first week
CI/CD CHAMP also took some key learnings for future pro- of the program aims at educating the team on different
jects from working together with the API Banking cluster. agile development practices and offering them the pos-
For instance, this experience contributed to the further sibility to challenge their agile working approach. At the
standardization of the CI/CD pipeline tools ensuring the end of the first week, the team identified two focus topics
flexibility to integrate additional tools if necessary. Addi- they wanted to intensify in the following full-time coaching:
tionally, the regular meetings with the API Banking team Value Stream Mapping (VSM) as a more holistic practice
during the project contributed to the practice of conduc- and Test-Driven Development (TDD) as an agile technical
ting weekly checkpoints and having a single overarching practice related to the coding process. Both selected agile
product backlog for the whole project. practices were then applied to an existing user story du-
ring and after the two weeks full-time coaching session.

CI/CD CHAMP DevOps-Coaching


The API Banking team aimed to further develop their agile The team applied VSM for analyzing existing processes
skills and challenge their current agile working practices to and thus set the basis to further increase its efficiency and
identify areas for further improvement. Hence, they par- productivity in delivering new ideas and solutions to custo-
ticipated in a three-months coaching program offered by mers. In applying TDD in the software coding process, the
CI/CD CHAMP DevOps Coaching to learn how they could team was also able to improve programming habits that
sustain efficiency and quality in their daily business and are valuable for agile software development. Hence, the
how to benefit from additional agile software development team benefitted mainly from two effects. First, since the
methodologies. team members worked together on the software artefact,
they got to know each other and their individual strengths
better. This contributed to a higher level of team spirit and
Prior to the coaching program, the cluster API Banking fa- a better understanding of how the team members could
ced two main challenges they wanted to address. First, the support each other in the best way having different skill
API team is dependent on the deliveries of a large set of sets and levels of experience. Second, the team practiced
different stakeholders. For instance, it is possible that the with an experienced coach providing them with individual
priority of user stories changes within an ongoing sprint guidance and an overview of possibilities. Following the
if a software artefact provided by a third party is delayed. two weeks full-time coaching sessions, the CI/CD CHAMP
Second, the team is very heterogenous in terms of skills DevOps coaches offered further support according to the
and software engineering background. pull principle based on the requests of the API team.

The CI/CD DevOps Coaching program started with an Undoubtedly, the two weeks full-time training and coa-
analysis workshop assessing the current agile team ma- ching sessions were an investment since it interrupted the
turity and the aspects the team wanted to improve. In a normal working process. But for the API team it has proven
next step, the team took part in full-time training sessions a fruitful investment enabling the team to perform even
covering relevant agile software development practices better than before or as Arnd Bischoff (Commerzbank AG)
based on the results of the prior analysis and assess- shared: “I really appreciated that the full-time training

18
session was tailored to our specific interests and needs. we needed,” says Michael Bellinger (Commerzbank AG).
The CI/CD CHAMP DevOps Coaching definitely further Usually, the onboarding and the maintenance specialists
improved our team motivation and collaboration – and of CI/CD CHAMP Automated Pipeline are working sepa-
it was a lot of fun as well.” The CI/CD CHAMP DevOps rately from the developers of the respective customer
Coaching contributed to the agile working practices and teams, however, the BDAA case required the expertise
collaboration of the API team. After the program the team from both teams. Thus, there was one colleague from CI/
applied new agile practices in their daily software develop- CD CHAMP Automated Pipeline as central contact person
ment, they recognized efficiency gains and productivity for BDAA. When further expertise was needed additional
improvements as well as growing closer as a team. Addi- experts were included. In general, the procedure of only
tionally, the CI/CD CHAMP DevOps Coaching team also using a reasonably small team and adding expertise on a
gained valuable learnings trough this experience working situational level has proven best practice to quickly identify
together with the team at API Banking. For instance, they challenges and find the required solutions. Together the
learned that tailoring the training and coaching sessions joint team onboarded the BDAA talkbot on the automated
to the individual team situation is crucial and thus the time CI/CD CHAMP pipeline and already set the preconditions
for preparation and assessment was extended for future necessary in the cloud environment to be able to deploy
internal customers. on demand. Further, the BDAA team used the PO-Cock-
pit provided by CI/CD CHAMP Frameworks to document
their software development process. The PO-Cockpit
CI/CD CHAMP Use Case 2: Big Data & Advanced Analytics guided the PO through the whole process and supported
the overall goal to build a process that does not demand
Within the cluster Big Data & Advanced Analytics (BDAA) special knowledge. “We are now able to easily develop
new products are developed applying analytics to internal and deploy new software products on demand thanks to
and external data. In 2020, the cluster started to develop the support of our CI/CD CHAMP colleagues,” says Peggy
a talkbot, which is in essence a chatbot utilizing voice-ca- Schuchert (Commerzbank AG). Additionally, the BDAA
pabilities to assist Commerzbank`s employees with their team built a holistic partnership with the team from CI/CD
inquiries via the Commerzbank internal service hotline. CHAMP DevOps Coaching to methodologically support the
The talkbot enables automated processing of our collea- whole technical process and further strengthen their agile
gues’ service requests that otherwise needed to be carried software development practices. This way, methodological
out manually by agents from the internal service center. and mindset support was provided to BDAA to establish
Although the BDAA talkbot case is still ongoing, it de- software craftsmanship and thereby facilitate the use of
monstrates the integration and tool flexibility of the CI/CD CI/CD.
CHAMP Automated Pipeline.

In summary, the process of offering the automated CI/


Although, the BDAA cluster has been using CI/CD CHAMP CD pipeline in a cloud environment is still ongoing but the
in their software development endeavors since 2018, BDAA joint team is sure that their journey will allow other cloud
was first built utilizing Google Cloud capabilities to take use cases to be onboarded easily and allow them to use
advantage of the cloud infrastructure. However, the Cluster the on-premise process for the development of new pro-
faced some challenges with the deployment process if the ducts. Applying a paradigm of shifting more use cases to
deployment of a new software artefact required a de- cloud environments the pioneering use case of the BDAA
viation from the standard deployment process. In these talkbot paved the way for other use cases to be developed
cases, the BDAA team faced a high complexity and manual using the on-premise automated CI/CD pipeline of Com-
rework that negatively impacted the time needed for de- merzbank. In addition, CI/CD CHAMP also took two other
ployment and always demanded specialist knowledge for learnings from this use case. First, new approaches such
finding solutions. Thus, they contacted the CI/CD CHAMP as shifting a Google Cloud use case on the automated CI/
pipeline team to support them in automating the deploy- CD pipeline requires an analysis of the initial situation as
ment process. well as understanding of the requirements that need to be
considered in the process. Second, in cases in which uncer-
tainty is high and the project outcome is highly dependent
At the beginning of the project, the CI/CD CHAMP team on external factors, a joint team working in short iteration
carefully assessed the BDAA team’s requirements and cycles can help to mitigate potential risks. Additionally,
current approach of the software deployment process. the BDAA team provided valuable feedback to the CI/CD
The BDAA talkbot case is the first Google Cloud native use CHAMP PO-Cockpit which laid the foundation for further
case for CI/CD CHAMP since CI/CD CHAMP offered only adaptions to increase the flexibility in its tailoring feature.
the possibility to automate the software deployment pro- We are looking forward to jointly work together with the
cess on-premise at this time. So, the mission was to enable team of BDAA to have them finally onboarded on the CI/
the first use case to be deployed in the Google Cloud using CD CHAMP pipeline.
CI/CD CHAMP and to establish a software deployment
process that was stable and easy to use. For instance, they
need to deploy on demand due to the complexity of the CI/CD CHAMP Use Case 3: CCB Securities Frontend
applications used.
The cluster CCB Securities Frontend (CCB stands for Cross
Channel Banking) develops online solutions for Commerz-
The team jointly decided to work together in a mixed bank customers to manage their portfolios and to carry
setting to ensure that the requirements and learnings out their stock exchange transactions within their online
made of BDAA team members in working in the cloud en- banking application. For instance, Commerzbank custo-
vironment are reflected in the future deployment process. mers can easily buy and sell stocks on major exchanges
“The colleagues from CI/CD CHAMP Automated Pipeline independently, anytime and anywhere via their smart-
always carefully considered requirements on the final phone or PC.
deployment process which really supported the solution

19
Prior to their application of CI/CD and their onboarding on After the team was onboarded on the CI/CD CHAMP Auto-
the automated CI/CD CHAMP pipeline, the team of CCB mated Pipeline, they are now able to autonomously release
Securities Frontend was highly dependent on the techno- new software solutions or features on demand due to the
logy stack of the CCB Online Banking cluster since their high level of automation the CI/CD CHAMP Automated
Securities Frontend application was one of many appli- Pipeline provides. Additionally, the team can detect, analy-
cations embedded in the CCB application. This led to a ze, and correct defaults faster. “We are very satisfied with
high complexity within the deployment process and CCB the tools and process provided by CI/CD CHAMP Auto-
Securities Frontend was only able to release once a quarter mated Pipeline. The combination of automation, early
since deployment and test cycles need to be coordinated testing and continuous support from the CI/CD CHAMP
among the whole CCB application. Thus, the CCB Securi- colleagues makes the difference,” says Klaus Donath
ties Frontend team aimed to have shorter deployment cyc- (Commerzbank AG).
les independently from the IT operation team of the CCB
main application leading to shorter feedback loops concer-
ning their new software features. CCB Securities Frontend In summary, the CCB Securities Frontend use case demon-
also wanted to use cloud-based technologies and decided strates the possibility of separating and onboarding an
to leverage a cloud-based Container Platform that was set existing software application from the technology stack
up in Commerzbank at that time. of Commerzbank to CI/CD CHAMP. Additionally, this use
case is also an example of how teams use CI/CD CHAMP
as a self-service tool for developing new software applica-
The cluster aimed to separate its deployment process from tions or features. Moreover, the CCB Securities Frontend
the CCB application to gain speed and flexibility and thus use case served as blueprint how CCB applications might
got in touch with the CI/CD CHAMP team. After the team be separated from the monolithic legacy technology stack,
of CCB Securities Frontend was onboarded on the CI/CD thus provided the impetus for the practice of breaking
CHAMP pipeline, they both worked together to provide down the technology stack into containerized applications.
the CCB Securities Frontend, with capabilities to deploy This case also provided important learnings for the CI/
software to the cloud-based container platform. As with CD CHAMP team. For instance, the onboarding process to
previous use cases, CI/CD CHAMP was extended to allow the pipeline was further improved and is now conducted
for this. The CCB Securities Frontend team also applied automatically enabling DevOps teams from the delivery or-
the PO-Cockpit provided by CI/CD CHAMP Frameworks. ganization to directly create new projects on the pipeline.
The PO-Cockpit allowed the team to easily structure the Moreover, this feedback of the DevOps teams concerning
documentation process and centrally coordinate the single the PO-Cockpit was used to further enhance the tailoring
process steps during the software development process. function of the tool as well as the possibility to build on
As Claus Weber (Commerzbank AG). notes “The PO- existing documentation in the case of a software change.
Cockpit supports the documentation process by offering
a simple tailoring to the required documentation. I really
appreciate that it guides product owners throughout the Commerzbank’s Best Practices – How to Master CI/CD
documentation process in one single application.” In the last four years, we learned that a one-size-fits all
approach to successfully apply and live CI/CD in the orga-
nization does not exist. Instead, success is highly depen-

20
dent on the interplay between different factors and how delivery is the ability to provide infrastructure and deploy
the technical and mindset components of this paradigm code dynamically.
are responsive to demanding business needs. Considering
our analogy of the Formula E, success on the racetrack
depends on the optimal adaption of the racing car to the 3. Focusing on measuring and visualizing
driving conditions. We as Commerzbank would like to The basis of any continuous improvement should be facts
share with you the four most important best practices from and figures. To obtain these, it is necessary to monitor the
our CI/CD experience that might also support you in your complete value stream during a CI/CD cycle. The automa-
next CI/CD endeavors. ted CI/CD CHAMP pipeline allows for easy measuring and
tracking relevant Key Performance Indicators (KPI) for
each step of the software development process. The gat-
1. Following a self-service approach as a guiding principle hered data is then visualized in a dashboard to guarantee
The first principle we followed when we designed and full transparency about the efficiency of the process and
developed the automated CI/CD CHAMP pipeline further is indicate areas for optimization. The dashboard contains
the principle of self-service that enhances scalability. The all the relevant DevOps KPI, such as deployment frequen-
pipeline should be easily accessible and usable for every cy, deployment speed or deployment failure. This allows
DevOps team member. For that reason, we created a por- DevOps teams to improve in the next iteration. Besides
tal as the gate to our pipeline where a new team member that, at Commerzbank, we continuously observe the usage
can order and get access to all the tools within one day. of the automated CI/CD CHAMP Pipeline so that we can
We further complement the self-service function by user inspect and improve certain building blocks or services of
friendly documentation for each of the tools, templates (of- CI/CD CHAMP.
ten provided as code in Git Repository) which can be used
for a quick start and different samples in a playground to
intensify learning and collaboration. “Building the sur- 4. Applying Shift Left
rounding fostering these components requires a strategy Our experience shows that DevOps teams that start early
and roadmap that articulates clear goals and measures of with using the automated CI/CD CHAMP pipeline gain
how employees can be encouraged to collaborate,” says more speed, efficiency, and confidence in their work. They
Dr. Lars Friedrich (Commerzbank AG). Another important can react early and quickly to quality issues and fix them to
aspect to consider is end-to-end responsibility to emp- finally improve their software artefacts without compromi-
ower DevOps teams and allow for team autonomy. DevOps sing an accelerated time-to-market. Therefore, the CI/CD
teams need flexibility to define their own workflows and CHAMP team consult, coach and support DevOps teams
to configure their own set of rules in certain tools without in the onboarding process on the pipeline in all environ-
risking noncompliance. The CI/CD CHAMP vision is to ac- ments so that they can start early realizing their ideas and
celerate the software delivery with a standard pipeline able delivering faster high-quality software. At Commerzbank,
to fulfill all requirements that teams could have while deve- we advise to shift left as much as possible, start to automa-
loping and delivering software. Although, diversity among te tests and think in iterations which will result in delivering
team members and autonomy within DevOps teams paired small pieces of software more frequently.
with a standard pipeline seems contradictory, it is import-
ant to find the right balance between flexibility and control.
Therefore, the automated CI/CD CHAMP pipeline incor- 5. Continuously improving through collaboration with
porates all necessary governance rules and guidelines of CHAMP users
Commerzbank to ensure compliance and at the same time CI/CD CHAMP is a technology foundation that provides its
provide teams the possibility to implement their own rules services to all DevOps teams in Commerzbank’s Delivery
and templates without losing speed and autonomy. Organization. From that perspective, CI/CD CHAMP users
are the ones who influence the further development of the
pipeline. Therefore, a potential extension of the pipeline
2. Aiming for an expandable, flexible, and integrated is always done in cooperation with the users. They know
pipeline best what they need to have a better user experience
A new feature in the hands of a customer always starts and decide which new features are essential. Besides the
with an idea. Therefore, we have designed CI/CD CHAMP importance of gaining valuable impulses from users, the
as a highly integrated pipeline to enable full automation CI/CD CHAMP team itself drives innovation to create new
in the software development process starting with a user technologies and possibilities which are subsequently stan-
story and completing the cycle with an automated de- dardized for the users. CI/CD CHAMP is open to incorpo-
ployment into production. All the tools in the pipeline are rate new innovative ideas coming from inside and outside
connected via APIs or have pre-configured plug-ins which Commerzbank. The generation of internal ideas is ensured
build direct interfaces between them. New components by internal cooperation, while external ideas are created
and tools can be easily added to the existing pipeline through interactions and exchanges with vendors, different
based on the needs of the respective DevOps team. At proof of concepts and conference participations.
Commerzbank, we believe an expandable pipeline is an ap-
propriate approach to reach different target environments
in a broader range from mainframe to cloud. A standard The three exemplary use cases highlight how Commerz-
pipeline should be technology stack agnostic and at the bank leverages CI/CD capabilities to be a strong partner
same time provide the possibility to treat infrastructure or for customers delivering valuable solutions that meet
code in the same way as application code. To successful- highest quality standards. We hope that these use cases
ly live DevOps it is crucial to provide the same tools and contributed to a better understanding of CI/CD CHAMP
practices for the functions of development and operations. and that our best practices might benefit you in your next
The automated CI/CD CHAMP pipeline is regarded as the CI/CD endeavor. In the next chapter we want to share our
common path application when software artefacts are expectations about the evolution of agile software de-
built, tested, quality checked and deployed. Beside scala- velopment.
bility another important leverage for accelerating software

21
Hypotheses About the
Future of Agile Software
Development until 2026

When we think of where the future of agile software due to the convergence of three interrelated factors [27]
development is heading to, we need to understand the [28]. First, an increasing number of tool vendors take an
dynamics that might shape its evolution. Now that DevOps EaC-first approach to tool configuration and deployment
practices have achieved widespread adoption in agile by offering organizations the possibility to manage everyt-
software development [18], future developments that go hing using code files [28]. Second, the ongoing standard-
beyond bringing Dev and Ops closer together do exist. ization of configuration formats also supported the EaC
So, the question is: in which direction will agile software trend since a common format makes it easier for develo-
development evolve within the next five years and how will pers to manage all of their tools with the same language
new technologies affect it? Together with 12 experts from and methodology (Tozzi 2020). Third, the possibility to
different industries we discussed trends and predictions to build and manage an entire tool chain on a CI/CD pipe-
weigh in on what to expect in the upcoming five years. As line, applying EaC drives the overall EaC approach also for
a result, we formulated five hypotheses about the future of other tools and processes [28]. Thus, EaC offers not only
agile software development until 2026 as depicted in figure a highly repeatable and scalable approach to tasks, but
8. These hypotheses center on a few key themes: as orga- also increases consistency across IT systems and processes
nizations accelerate on their agile transformation journey, [27].
focusing on developing agile capabilities and embracing
agile values will be the imperative; leveraging technologies
will continue to offer new possibilities for automation and The reason why EaC will become a priority is mainly based
ultimately provide the basis for improved collaboration; and on its various potential benefits. For instance, EaC will
organizations will further focus on an Everything as Code ensure transparency and a clear understanding of the
approach to create a common language for the whole infrastructure, configurations and policies without the
organization. need to rely on manually updated documentation as the
code does evolve to a single point of truth incorporating
all information. EaC will provide DevOps teams with the
01 Everything as Code (EaC) will become a priority lea- technical foundation for their daily collaboration, helping
ding to the practice that everything will be adjusted and them to organize their work more efficiently [28]. Moreo-
managed using code ver, Compliance as Code (CaC) and Security as Code (SaC)
An EaC approach treats all components of a system such ensure that compliance and security requirements and
as documentation, application code, configuration ma- checks are built as code into the software development
nagement, infrastructure, and compliance as code so that life cycle from the first stages of the process. This creates
everything adheres to the same software development a common standard for all DevOps teams and guarantees
practices [25]. As focusing on the concrete CI/CD use case, that compliance and security issues are automatically
the idea of EaC will make its entry across each layer of the detected and corrected, in near real time [29]. Additionally,
CI/CD pipeline so that infrastructure, schemas, pipelines, if compliance or security is written into software applicati-
and operations are all described and treated like applica- ons, they can also be continuously updated in the event of
tion code [26]. This means that automation via software changing regulations without the need to manually adapt
will be everywhere providing the technical foundation for applications to new regulations. EaC also offers the possi-
an array of other developments such as enhanced cross- bility to establish documentation as a continuous automa-
functional collaboration, or a shift left approach of security ted process output throughout the whole CI/CD process.
and compliance that can be expected to occur in the next In future, developers will be supported with documentation
five years. that is automatically generated via a set of templates that
collect and transform the required information from its
origin to the target documents for a simultaneous delivery
At its core, EaC is not at all a new trend. Some isolated of the new code and documentation [30]. This automated
forms of EaC have already gained popularity within the last shift left of documentation will largely guarantee the accu-
years, Infrastructure as Code (IaC), Pipeline as Code (PaC) racy and integrity of documentation and will significantly
and GitOps 11 being the most prominent examples and it increase the delivery speed of the documentation process
can be expected that this development will pick up steam [30]. Moreover, IaC offers organizations the possibility to

11
GitOps refers to the approach to use Git to manage IT Operations. 22
01 Everything as Code (EaC) will become a priority leading to the practice
that everything will be adjusted and managed using code

02 “BizDevSecOps” will be the evolution of DevOps ensuring a holistic


cross-functional team approach in delivering customer value

03 Organizations will leverage Machine Learning (ML) algorithms to


automate a wide range of processes in the DevOps lifecycle

04 End-to-End autonomous testing will be the standard approach for


software testing leveraging cloud-based services and anonymous test
data extracted from production
Consectetuer adipiscing elit

05 Organizations will focus more on developing agile capabilities and


following agile values to comprehensively “being agile” instead of just
“doing agile”

Figure 8: Hypotheses About the Future of Agile Software Development until 2026

deploy infrastructure whenever it’s required and to integra- need to ensure the appropriate working conditions so that
te it into a CI/CD pipeline to dynamically build and destroy experts from the business, security, development, and ope-
different environments as the pipeline executes. Finally, ration side should work together to improve processes and
EaC ensures consistency in migrations, deployments, and develop software that can hit the market early.
configurations. Deployments in a specific environment will
result in the same configuration as if the same deployment
was conducted in another environment. Additionally, if 02 “BizDevSecOps” will be the evolution of DevOps ensu-
organizations follow the practice of EaC, the efficiency and ring a holistic cross-functional team approach in deliver-
performance of an organization to react to customer needs ing customer value
will be measured and tracked by CI/CD value streams. Building on the foundation of DevOps, BizDevSecOps
incorporates business and security experts into the soft-
ware development process from the early beginning,
While this trend offers many potential benefits, it is im- ensuring the targeting of key business outcomes through
portant to consider that the pace of this development is the DevOps lifecycle. “Bridging the functions of develop-
highly dependent on the flexibility of vendors concerning ment and operations is not enough, organizations should
code and installation playbooks as well as the existence of rethink the traditional DevOps cross-functional team ap-
tools to verify the code files to avoid potential pitfalls. In proach a bit further and follow a holistic concept to deli-
addition, organizations will need to create the necessary ver customer value,” says Daniel Zwicker (Authada GmbH).
conditions to take full advantage of EaC. As Daniel Meixner The active involvement of business and security experts
(Microsoft AG) put it, “EaC will certainly become an im- in traditional DevOps teams builds the foundation for a
portant trend. However, organizations will need to meet collaborative culture which enables a new model of opera-
a number of prerequisites to be prepared to take full ting to provide superior customer experience by offering
advantage of EaC. For instance, appropriate tooling will customer-centric products and services through a CI/CD
be necessary and access rights sensitive areas will have pipeline while being compliant with security standards
to be installed and managed.” Additionally, EaC requires [1]. Although, many organizations have already started
a mindset shift so that all team members irrespective of to stronger integrate the business (BizDevOps) or secu-
their function create a deep understanding for each other’s rity functions (DevSecOps) into their DevOps teams, this
activities and tasks. For example, team members of opera- development will further accelerate since organizations will
tions should adopt a development mentality and automate increasingly adopt a product-thinking approach to embark
whatever and wherever possible. Moreover, organizations on the path of accelerating their customer-driven digital

23
transformation [31]. Hence, cross-functional teams will be vironment to operations, operations can use observability
the decisive factor for organizations to develop software tools on the CI/CD platform to determine impacts across
more quickly, more secure and to release features that are different software releases. Subsequently, they can auto-
built specifically to service business objectives, and to be matically share this information with the developers who
more responsive to user demand [31]. can modify their builds accordingly and with the security
specialist who endure that the change complies with secu-
rity standards.
The ”Sec” component of BizDevSecOps teams will ensure
that security is built from the beginning into the software
development lifecycle ranging from prioritizing security re- 03 Organizations will leverage Machine Learning (ML)
quirements as part of the product backlog over continuous algorithms to automate a wide range of processes in the
testing software code for vulnerabilities to comprehensive DevOps lifecycle
monitoring in production [32][33]. This extends the origi- ML will potentially impact current approaches and process
nal DevOps objective of delivering software features with steps in the DevOps lifecycle. Considering the develop-
high velocity by early detection and mitigation of vulnera- ment part of the DevOps lifecycle, ML provides potential to
bilities [34]. “Security will be an integral part in all phases operate and optimize CI/CD pipelines. First, organizations
of the software development by rethinking DevOps to Biz- will apply a combination of EaC and ML to establish self-
DevSecOps to ensure a security shift left approach,” says optimizing CI/CD pipelines since ML algorithms enable an
Doron Reuter (ING Groep N.V.). From a technical perspec- automated observation of the pipeline operation, continu-
tive, organizations will establish security as a mandatory ously collecting data. “Leveraging ML algorithms in CI/CD
design requirement for their CI/CD pipelines applying the pipelines requires an EaC approach, thus organizations
EaC practice to reflect security best practices in their IaC need to follow this approach first to take full advantage
declarative scripts. of the integration of ML algorithms,” says Mirco Leimgru-
ber (Swisscom AG). Organizations integrating ML in their
CI/CD pipelines will be directly notified if anomalies exist
The “Biz” component of BizDevSecOps teams will ensure in the process or tools included. Moreover, ML algorithms
the active involvement of business experts in traditional can predict potential defects in code generated CI/CD
DevOps teams. This builds the foundation for a collabo- pipelines and subsequently react by triggering scripts for
rative culture which enables a new model of operating to destroying and provisioning certain parts of the pipeline.
provide superior customer experience by offering valuable Second, ML algorithms contribute to a stable, resilient
products and services through a CI/CD pipeline [1]. “Inclu- operation of CI/CD pipelines by analyzing monitored data
ding the business perspective in software development is that are produced from different events to predict or avoid
nothing new, it is how the agile movement started. Orga- outages.
nizations need to integrate the customer perspective from
the early beginning and deliver their software features
fast to collect customer feedback,” states Rahul Verma Additionally, within the CI/CD process, the application
(Trendig Technology Services GmbH). Integrating business of ML offers various advantages from decision-making
stakeholders into the value stream allows organizations to process improvements to automated operations and code
ensure that the DevOps cycle is aligned with business ob- quality enhancements. Although, this trend is in its ad-
jectives and in turn allows them to execute their transfor- aption still in its infancy due to challenges in preparing
mation objectives faster and more efficiently [31]. Whereas large data sets and a challenging regulatory environment
business professionals will enable a stronger integration especially in the banking industry, there exist four promi-
of the voice of the customer and allow organizations to sing developments to watch within the next five years. The
see the impact of new products on business value more first important trend is the application of ML for automated
directly [1][31], IT professionals ensure further automation code reviews. The result will be the automated and early
and reliable software. Additionally, security experts make detection of code flaws, security issues, and code-related
sure the software development process and new feature defects which will lead to an enforcement of code and
comply with security policies [1]. Neither is more important security standards [35]. “The integration of ML into the
than the other, but they all need to work together in order DevOps lifecycle offers the possibility of carrying out
to provide software that meets customer requirements, automated application security vulnerability checks that
is compliant with security standards and is delivered with will lead to both real-time security risk assessments and
high quality and speed. an increased efficiency due to the reduction of time nee-
ded to conduct those tests,” says Raz Raviv (ING Groep
N.V.). Another promising ML development comprises the
At the core of BizDevSecOps is the common language of prediction of potential issues based on data. ML models
code that enables collaboration and co-innovation among have the ability to detect patterns and predict signs of
the different functions. This common language across failure in cases in which humans hardly perceive them [36].
the whole software development lifecycle ensures that Such early predictions support DevOps teams to identity
everyone is working towards the same outcomes. Mo- and fix issues before they have a negative impact on their
reover, it aligns teams into a coordinated BizDevSecOps software artefact. The next trend to monitor in leveraging
practice, where all functions have better insight into how the possibilities of ML within the DevOps lifecycle is the
each activity contributes to the overall objective. In this possibility of a faster root cause analysis. ML can recognize
context, EaC provides the technical conditions to integra- patterns between implication and activity to determine the
te all IT systems into one common CI/CD platform, such root cause behind a defect [36]. Organizations can esta-
that BizDevOps teams can use the same tools in their daily blish an automated process to fix defects permanently by
business. In this context, organizations will have one source conducting ML-based root cause analysis that can reduce
of truth for data and tracking points that can be establis- the risk of human error while providing the development
hed to measure the speed or waste in certain phases of the team with recommendations for writing more efficient and
software development lifecycle. For instance, if business performant code [37]. The last important ML trend in the
experts give feedback on an incident in the production en- DevOps environment is the adaption of Robotic Process

24
Automation (RPA) for test automation [36]. RPA can be If organizations follow an IaC approach enabling a quick
utilized to transform manual, time-consuming and error- reproduction, destruction, and provision of infrastructu-
prone tests into automated and streamlined ones. General- re, end-to-end autonomous testing can be dynamically
ly, if ML tools are integrated in the DevOps tool stack, they conducted to save resources and time. Additionally, the
offer the opportunity to free up DevOps teams from low- utilization of cloud-based services will make testing faster,
value tasks that potentially constrain productivity while at cheaper, and more efficient since it will allow organizations
the same time improving software quality and application to leverage an almost infinite resource pool and flexible
security [38]. However, implementing the above-men- infrastructure capacities [42].
tioned ML trends and realizing its benefits substantially
depends on the regulatory environment and fulfillment of
the data set conditions by organizations. Thus, it remains In the upcoming years, the application possibilities of
to be seen whether a wide adoption of ML in the DevOps cloud-based and autonomous end-to-end-testing solutions
lifecycle will become standard in five years. will be complemented by production-related test data. This
will be enabled by the anonymous extraction of test data
from production to ensure test cases reflect real world
Considering the operations part of the DevOps lifecycle, scenarios. This development will lift the quality of the soft-
ML will also have a key role in ensuring operations continui- ware delivered on an even higher level since the probability
ty after the product or feature is released into production. of a case that has not been tested before will decrease.
The pattern-based functionality of ML algorithms provi- Nevertheless, it is important that organizations ensure that
des the possibility to anonymously analyze user metrics anonymous extracted test data is always compliant with
and the functioning of applications in production. In the the law in force (e.g., General Data Protection Regulation).
case of any anomalies, ML tools will alert DevOps teams Hence, using real-life data without the knowledge and
in the case of any issue including possible solutions about assent of the customer is forbidden and does complica-
how to solve this issue. Although, these predictions sound te generating test data. Concerning the financial service
very promising, organizations such as banks operating in industry, the realization of this hypothesis depends on the
a highly regulated environment might face obstacles in development of future regulatory conditions since the cur-
applying ML algorithms to analyze and interpret customer rent regulation forbids tests based on real-life data [43].
data from production.

Currently, end-to-end autonomous testing leveraging


04 End-to-End autonomous testing will be the standard cloud-based services and anonymous test data extracted
approach for software testing leveraging cloud-based from production is still in its infancy and hence organizati-
services and anonymous test data extracted from produc- ons need to establish the preconditions to take advantage
tion of this functionality. As René Gressly (PostFinance AG)
In the last decade, testing has progressed by small incre- stated, “in the financial services industry a special focus
mental steps. By now shift left is the standard approach in must be put on anonymizing the data that is used for test
the DevOps and CI/CD community as it helps to mitigate cases since inference on the real-life persons cannot only
the risk for identifying issues only at a late stage of the be drawn by having the clear name but also by combining
software development lifecycle [39]. In future, organiza- a set of other data – a case that needs to be precluded.”
tions will continue to further automate test- and develop- Embracing a high or even total autonomy in testing requi-
ment environments that provide all necessary resources, res the combination of humans and machines to ensure a
such as interfaces to systems and applications, data and scalable, stable, and secure test application [41]. Hence,
tooling on demand based on cloud services [17]. Or as organizations need to invest in hybrid tools that allow the
Nicholas Mills (CircleCi) puts it differently, “in the upco- combination of human and machine capabilities as well as
ming years, we will see an increasing level of automation train their software engineers to understand and interpret
in testing and validating change in software. If a task is the test results produced by the machine. Moreover, orga-
repeated multiple times, it will get automated. This means nizations should focus on the human-machine interface by
organizations will realize material cost savings (since the focusing on how algorithms should be trained based on
per-minute cost of computing is orders of magnitude the emergence of new findings and how the interaction of
cheaper than the per-minute cost of software engineers), humans and machines can be designed in an optimal way
and drive significant productivity gains from software for the organization itself.
development teams (by increasing the efficiency and
effectiveness of validating changes to code).” Autono-
mous testing allows test cases to be created and executed 05 Organizations will focus more on developing agile
without much human intervention, thus decreasing the capabilities and following agile values to comprehensively
time needed for testing while increasing available time for “being agile” instead of just “doing agile”
DevOps teams to focus on more complex issues [40]. Or- As organizations are facing the need to constantly trans-
ganizations will apply end-to-end autonomous testing that form themselves to stay successful in a dynamic environ-
will innovate test creation, maintenance, and execution by ment, organizations will increasingly adapt agile working
applying experience from failed tests and making decisions methods at the enterprise level to improve delivery, increa-
on how new tests should be created and executed even se speed, and enhance customer experience. However,
when conditions such as testing environments or test data in order to holistically transform to an agile organization,
change [41]. Further, using autonomous testing will allow following a “doing agile” approach that includes the sole
for applying tests automatically in parallel to building a implementation of agile practices, methods, and tools is
change [42]. not enough. Organizations will need to develop agile capa-
bilities and embrace agile values to becoming a truly agile
organization that follows a “being agile” approach [44].
The combination of EaC and autonomous testing tools The shift to “being agile” will substantially increase enter-
provides the possibility for dynamic and flexible testing. prise-wide agility and ensure that change is sustained. Do-

25
26
minique Mühlbauer (Trendig Technology Services GmbH) agile” approach. A value stream analysis tool might sup-
said, “this can be seen as the crucial hypothesis underly- port this monitoring and evaluation process by measuring
ing the other hypotheses since, having motivated indivi- different tracking points on the value stream of a CI/CD
duals, taking pride in their craftsmanship, working toge- pipeline. This allows DevOps teams to have an overview of
ther in cross-functional teams, will enable delivering high all data in one integrated dashboard that indicates areas
quality software on high speed.” Therefore, organizations for improving efficiency in the overall software develop-
will have to focus on the development of agile capabilities ment lifecycle.
such as sensing, learning, adaptability, resilience, speed,
innovation, collaboration, and efficiency on different or- These hypotheses about the future of agile software
ganizational dimensions in future [5]. This approach will development illustrate in the direction CI/CD and DevOps
lead to the holistic alignment of organizational strategy, might be heading to. “At Commerzbank, we believe that
structure, processes, technology, and leadership to these the underlying trends and dynamics of these concepts
agile capabilities. For instance, as agile transformations are should be considered by organizations to further drive
grounded in cross-functional teams empowered to operate their agile transformation,” says Michael Varona (Com-
in rapid decision-making and learning cycles, organizations merzbank AG).
must ensure that the organizational structure and proces-
ses are designed around this purpose [45]. Additionally, Commerzbank’s Vision for CI/CD CHAMP 2.0
organizations will need to invest disproportionally in the
upskilling or reskilling of their employees to enable them to
work in new roles, collaboration setups, and new fields of As in Formula E, teams are constantly searching for impro-
responsibility. vement, and at Commerzbank we are working with high
intensity to bring CI/CD CHAMP to the next level. Thus, we
questioned ourselves: how should we anticipate the expec-
Besides focusing on the development of agile capabilities, ted developments to support the fast delivery of high-qua-
organizations will also follow agile values that are reflected lity software that meets Commerzbank-customers’ expec-
in behaviors as well as mindsets, and practices drive the tations? And which tools, methods or processes support
required cultural shift sustainably. Therefore, organizations the projects in the best way to deliver value to Commerz-
will increasingly focus on empowering their employees in bank’s customers?
their way of thinking comprising attitudes, orientations,
and mentality patterns towards living agile values and prin-
ciples [46]. One crucial component of this is an empowe- Driven by these questions we envisioned CI/CD CHAMP 2.0
ring leadership approach that embraces an open communi- that is based on three interrelated developments. First, the
cation, a culture of constructive criticism and collaboration. CI/CD CHAMP Automated Pipeline 2.0 will contain a mix of
This requires leaders to cultivate an environment of psy- self-developed and commercially available tool solutions
chological safety in which employees are encouraged to that follow an IaC approach. Second, CI/CD CHAMP 2.0
experiment and learn without fear of negative consequen- will be cloud-native to provide maximum scalability and
ces to self-image, status, or career [47]. “The enterprise flexibility for its users. This will be complemented by reu-
level of agility can often be defined based on the existen- sable templates for all the CI/CD CHAMP 2.0 tools that can
ce of psychological safety in the teams since “being agile” be tailored for the software project‘s specific needs and
requires a high degree of psychological safety. Thus, es- that would enable DevOps teams to build and deploy new
tablishing an environment in which every employee feels software application versions within minutes. Third, CI/CD
safe to contribute and speak up is a decisive factor for the CHAMP 2.0 aims to be a pioneer in leveraging EaC-
overall success of the company,” says Schlomo Schapiro approach and implementing it in context of a CI/CD pipe-
(DB Systel GmbH). This psychological empowerment will line.
shape employees‘ consciousness, mindset, and behavior
that go beyond simply following agile methodologies or
frameworks. Another important component of establis-
hing an agile value system are coaching activities that can
support the required mindset shift of employees. Coaches
could stimulate thought within agile teams, help them to
reflect on their agile way of working, collaboration and
attitudes to find further areas for improvement [48].

As organizations embark on their journey to developing


agile capabilities and following agile values, it is important
to focus on the main objective of agile software develop-
ment – cross-functional collaboration to enable quick
delivery of high-quality and customer-oriented software.
Or as Markus Eisele (RedHat) put it differently, “agility
from the textbook is doomed to fail. The idea of cross-
functional teams working together is much more im-
portant since “being agile” needs the appropriate agile
capabilities and mindset.” In this context, continuous
learning and improvement is a core principle of “being
agile”. Successful agile transformations have shown that
monitoring of the transformation process, evaluating the
incremental product development flow and its impact on
performance as well as running regular retrospectives to
learn from success and failures are key to sustain a “being

27
Summary and Conclusion
Software changes the world. Going beyond agile software business within traditional DevOps teams. BizDevSecOps
development, CI/CD and DevOps have pioneered the way teams will reinforce the collaborative approach of software
software development is thought, conducted, and lived development enabled by CI/CD. We as Commerzbank truly
and thus have become essential approaches for many believe BizDevOps is here to stay, representing a roadmap
organizations in their software endeavors. In this context, for companies to stay ahead of their agile transformation
the concept of CI/CD supports organizations in accelera- curve and realize customer-driven business success. Howe-
ting delivery speed, quality, and customer centricity. Most ver, organizations that want to leverage the full potential of
importantly, the interrelation of the CI/CD cornerstones these developments need to fulfill specific prerequisites to
that comprise people, processes and tools ultimately drive be ready to anticipate the respective changes. For instan-
automation, co-innovation, and collaboration within the ce, organizations need to establish a roadmap that com-
whole organization to ultimately create customer value. municates the long-term vision while providing flexibility
for their operating teams to quickly re-prioritize activities
when relevant trends emerge that contribute to this vision.
In this white paper we discussed -besides how CI/CD af- Being able to anticipate trends also requires the presence
fects value creation in general and in the financial services of a highly automated and well-functioning CI/CD pipeline
industry in particular- also how this concept could be that provides consistency and reliability. Additionally, orga-
brought to life by providing insights on Commerzbank’s nizations with a monolithic IT legacy need to be open for a
CI/CD approach. At Commerzbank, we believe that there certain degree to architectural change to establish flexible
is no one-size-fits-all approach when it comes to imple- and adaptive systems.
menting CI/CD since every organization has its individual
requirements and faces different situational characteristics.
However, the implementation of a CI/CD pipeline should The discussed conclusions of this white paper are centered
always follow a set of technical principles comprising the around five key take-aways that are illustrated in figure 9.
setting up of end-to-end responsibility for teams, reducing
risk throughout the software development lifecycle, and
integrating short feedback loops. We are convinced that acceleration will be the new norm
for creating customer value in agile software development
and we are excited to contribute with our experience and
The true value of CI/CD comes from the enablement of best practices to this development. We invite you to also
teams to own the whole software development lifecyc- share with us insights you gained on your CI/CD journey
le from the initial product idea to the release of the final so far and to discuss where the future of agile software
software feature. Additionally, CI/CD as a cross-functio- development is heading to. We are looking forward to
nal concept leverages on the joint strength of different hearing from you and exchanging our ideas. What do you
functions and thus ensures the strong integration of crucial think about beyond banking in CI/CD and the next wave of
aspects such as security, compliance, and the voice of innovation in agile software development?
the customer within the software development process.
“With even more customer interaction becoming digital
the concept of CI/CD is a key differentiator for us in the
market. It offers unprecedented opportunities to develop,
test and deploy software fast and with high quality for our
customers,” says Dr. Carsten Bittner (Commerzbank AG).
In this context, the Commerzbank CI/CD CHAMP journey
shows that CI/CD as an organization-wide approach to
agile software development enables teams to release soft-
ware features faster and more independently. Employing
the metaphor of Formula E, teams being equipped with the
necessary safety equipment (CI/CD CHAMP Frameworks),
provided with the appropriate racing car (CI/CD CHAMP
Automated Pipeline) and having an experienced pit crew
(CI/CD CHAMP DevOps Coaching) can autonomously drive
innovation and value creation within the organization.

Although many organizations still refine their CI/CD ap-


proaches, the next wave of innovation in agile software
development is just around the corner and will significantly
impact the direction in which the future of CI/CD and De-
vOps will be heading for in the next five years. We discus-
sed that the evolvement of EaC as the new norm provides
the foundation for an array of other developments such as
the integration of additional functions such as security and Figure 9: Key Take-Aways

28
29
Appendix

Methodological Approach

This white paper was developed in a collaborative research


project including business and technical experts from
Commerzbank as well as researchers from the Business
Engineering Institute St. Gallen. We based our insights
on three main sources. First, we conducted an extensi-
ve desk research between May and July 2021. The desk
research comprised literature from academic discussion,
practical studies, other white papers, blogs, and news
articles. Second, we included both our technical as well as
our business experience into this white paper, for exam-
ple from deep dives with different Commerzbank CI/CD
CHAMP business experts. Third, we conducted interviews
with internal Commerzbank teams and with representati-
ves from external organizations as well to gain additional
perspectives on our research. From July to August 2021,
we interviewed three different Commerzbank product
teams on their experience in applying CI/CD CHAMP to
share their perspectives on CI/CD CHAMP. During August
and October 2021, we conducted 9 interviews with 12
representatives from technology corporations, financial
institutions, consulting companies, and FinTechs to valida-
te our findings and to further develop our ideas about the
future of agile software development. We documented our
interviews and collected general insights such as recurring
statements or mentioned trends. Specific quotations have
also been collected and in accordance with our interview
partners integrated into this white paper. The results of our
research and our interviews has been extensively discus-
sed and refined during several meetings during September
and October. This approach helped us to develop a more
nuanced picture of the CI/CD case studies presented in
chapter 3 (integrated results of the interviews with Com-
merzbank product teams) and the hypotheses about the
future of agile software development in chapter 4 (integ-
rated results of interviews with representatives of external
companies).

30
Our External Interview Partners

We conducted 9 interviews with the following representatives from a diverse set of companies. We hereby would like to
express our gratitude towards our interview partners for spending their valuable time as well as sharing their insightful
ideas with us. We appreciate the open conversations and the challenging discussions.

Organization Interview Partner Function

Authada GmbH Daniel Zwicker Chief Technology Officer

Circle Internet Services Inc. (“CircleCI”) Nicholas Mills General Manager EMEA

DB Systel GmbH Schlomo Schapiro Chief Architect Cloud

ING Groep N.V. Raz Raviv Cyber Innovation Tech Lead

Doron Reuter Venture Lead

Microsoft Corporation Daniel Meixner DevOps Architect

PostFinance AG René Gressly IT Architect

Redhat Markus Eisele Developer Strategist EMEA

Swisscom AG Mirco Leimgruber DevOps Engineer

Trendig Technology Services GmbH Dominique Mühlbauer Quality Advocate & Agile Evangelist

Rahul Verma Head of Test Automation Department

31
Our Commerzbank Sponsors and
Use Case Partners

We would also like to warmly gratitude our internal sponsors for their support during the preparation phase of this white
paper. Moreover, we also want to thank our colleagues from the API Banking Cluster, Big Data & Advanced Analytics and
Securities Frontend for spending time and sharing their valuable feedback on how CI/CD CHAMP supports them in their
daily business.

Our Commerzbank Sponsors

Sponsor Function

Dr. Jörg Oliveri del Castillo-Schulz Chief Operating Officer

Dr. Carsten Bittner Divisional Board Member, Group Technology Foundations

Sabine Vigelius
Key Area Lead of Customer Process & Data Management

Dr. Alena Kretzberg Head of comdirect, Marketing & Digital Banking

Dr. Lars Friedrich Head of Group Securities & Brokerage

Michael Varona Head of Group Digital Transformation

Our Use Case Partners

Use Case Use Case Partner Function

API Banking Akli Amar-Youcef Scrum Master

Arnd Bischoff Product Owner

Christian Pfaff Solution Architect

Thorsten Roth Expert Project Engineer & Scrum Master

Volker Sulzbach Subject Matter Expert API & EAI Technologies

Big Data & Advanced Analytics (BDAA)

Michael Bellinger Business Expert

Peggy Schuchert Product Owner

Securities Frontend

Klaus Donath Software Engineer

Claus Weber Senior Product Owner

32
About the Authors
Commerzbank Cluster CI/CD CHAMP

The cluster CI/CD CHAMP provides a reliable end-to-end solution consisting of a fully automated self-service pipeline,
modern software development and test methods, corresponding tools and DevOps coaching. The objective is to enable all
clusters of Commerzbank delivery organisation to deliver regulatory compliant software fast and efficient to customers in
high quality and with short release cycles.

Together with our research partner the Business Engineering Institute St. Gallen, we published this White Paper.

Florian Meiser
Cluster Lead CI/CD CHAMP
Commerzbank AG
Florian.Meiser@commerzbank.com

Damian Waszak
Product Manager CI/CD CHAMP
Commerzbank AG
Damian.Waszak@commerzbank.com

Fonlinda Frasheri
System Architect CI/CD CHAMP
Commerzbank AG
Fonlinda.Frasheri@commerzbank.com

Jan Willemsen
Senior Product Owner CI/CD CHAMP
Commerzbank AG
Jan.Willemsen@commerzbank.com

Stephanie Radet
Product Owner CI/CD CHAMP
Commerzbank AG
Stephanie.Radet@commerzbank.com

Rostam Rahnema
Senior Product Owner CI/CD CHAMP
Commerzbank AG
Rostam.Rahnema@commerzbank.com

33
Business Engineering Institute St. Gallen

The Business Engineering Institute St. Gallen is a Swiss institute that conducts research in the areas of ecosystems, digital
transformation, and disruptive technologies to develop academic insights with practical impact. We collaborate with insti-
tutions such as the University of St. Gallen, the University of Göttingen and partner companies from the financial services
sector to develop answers to strategic questions related to technology and information-based value creation in the digital
age.

Katharina Schache
Research Associate & PhD Candidate
Business Engineering
Institute St. Gallen AG
Katharina.Schache@bei-sg.ch

Oliver Kutsch
Head of Consulting
Business Engineering Institute St. Gallen AG
Oliver.Kutsch@bei-sg.ch

Dennis Vetterling
Research Associate & PhD Candidate
Business Engineering Institute St. Gallen AG
Dennis.Vetterling@bei-sg.ch

Acknowledgements

While working on this white paper, we have had numerous discussions and received lots of valuable feedback from within
and outside of Commerzbank. We would like to express our sincere gratitude towards all those who supported us on this
journey. We would like to thank Christian Betz and Marc Burkhalter constructive feedback and their valuable ideas. We
would also like to thank Nigel Champion for his linguistic revision.

34
Glossary

Application Programming Interfaces (API) Application Programming Interfaces are a technology


which allows software applications to talk to each other in
a machine-readable way.

BizDevSecOps BizDevSecOps is a portmanteau word that expresses


the combination of bridging the functions of business,
development, security and operations in one single team.

Cluster Organizational structure of Commerzbank, a Cluster


describes a group of teams working in the same business
area.

Cross Channel Banking (CCB) CCB is the main Commerzbank portal, comprising of core
portal solution and multiple financial products across diffe-
rent channels.

Continuous Integration and Continuous Deployment CI/CD forms the operational backbone of DevOps practi-
(CI/CD) ces and essentially allows organizations to provide users
with new software products or features at any time in a
sustainable way.

Commerzbank Delivery Guide CHAMP (CDG CHAMP) Commerzbank Delivery Guide CHAMP helps developers to
be guided in the development process containing all rele-
vant documentation requirements.

Commerzbank Hyper Acceleration Master Pipeline Commerzbank Hyper Acceleration Master Pipeline descri-
(CHAMP) bes the CI/CD Pipeline of Commerzbank and incorporated
services.

Delivery Organisation Organizational structure of Commerzbank. All software


development happens in Clusters, all Clusters form Delive-
ry Organization.

DevOps DevOps describes the paradigm of bringing Development


and Operations closer together in the software develop-
ment process.

Everything as Code (EaC) EaC is the idea that all functionalities of a software de-
velopment pipeline can be managed in using code.

Failing Forward Failing Forward describes an approach based on the idea


that mistakes are not evil, but necessary for learning and
continuous improvement. Employees should leave their
comfort zone to drive development.

Integrated Development Environment (IDE) IDEs are applications that support developers in develo-
ping software in offering a compilation of different tools
that are needed during programming.

35
Kanban Framework for establishing Agile and DevOps in software
development. The different tasks are visualized on a Kan-
ban Board to enable the team members to gain an over-
view anytime [49]

Mainframe A mainframe is a complex and powerful computer system

Product Owner (PO) The Product Owner (PO) is a member of the agile team
who is responsible for defining user stories and prioritizing
the backlog. POs streamline the execution of project priori-
ties and ensures the representation of customer interests in
the software development process [50]

Product Owner Cockpit CHAMP (PO-Cockpit CHAMP) The PO-Cockpit CHAMP is a tool that guides the PO`s
through the development workflow

Scrum Scrum is a framework that supports people, teams and or-


ganizations to create value through adaptive solutions [51]

Shift Left Shift left is the practice of shifting performing unit or inte-
gration tests and code quality checks early in the software
development process and can result in significant cost
reductions as well as in increased software quality since
potential pitfalls in the software code are identified early
during the development process [12]

Technology Stack Describes the combination of frameworks and tools used


for software development.

Test-Driven Development (TDD) Test-Driven Development is a procedure in which the


programmer consistently creates software tests before
the components to be tested, also called test-driven pro-
gramming

User Story A user story is an informal, general explanation of a soft-


ware feature written from the perspective of the end user.
Its purpose is to show what value a software feature has
for a customer [52]

Value Stream Mapping (VSM) Value Stream Mapping is a methodology for analyzing the
steps and their sequence of a current process easily quan-
tifying the time and working effort needed at each process
step. This allows for designing a target process that mini-
mizes the process time while maximizing the efficient use
of resources and the process quality [53]

36
37
Literature Overview
[1] McKinsey & Company, „The five Core IT shifts of scaled agile organizations,“ 2021. [Online]. Available at: https://www.mckinsey.com/business-functions/people-and-
organizational-performance/our-insights/the-five-core-it-shifts-of-scaled-agile-organizations [Accessed: 20.10.2021]
[2] The Boston Consulting Group, „Leaner, Faster, And Better With DevOps,“ 2017
[3] Oliver Wyman, „When Vision and Value Collide - The State Of The Financial Services Industry 2020,“ 2020
[4] K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham, M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, J. Kern, B. Marick, R.C. Martin, S. Mellor,
K. Schwaber, J. Sutherland, D. Thomas, „Manifesto for Agile Software Development,“ 2001. [Online]. Available at: https://agilemanifesto.org/. [Accessed: 05.11.2021]
[5] D. J. Proba, „Technikkompositionen für das situative Methoden-Engineering agiler Methoden,“ 2021
[6] J. Halstenberg, B. Pfitzinger, T. Jestädt, „DevOps - Ein Überblick,“ 2020
[7] S. Blank, „Why Continuous Deployment May Mean Continuous Customer Disappointment,“ 2014. [Online]. Available at: https://hbr.org/2014/01/why-continuous-de-
ployment-may-mean-continuous-customer-disappointment. [Accessed: 03.11.2021]
[8] C. Riley, „Waterfall to Agile to DevOps: The State of Stagnant Evolution,“ 2014. [Online]. Available at: https://devops.com/waterfall-agile-devops-state-stagnant-evo-
lution/. [Accessed: 05.11.2021]
[9] BCG, „Going All In With DevOps,“ 2018
[10] D.K. Rigby, J. Sutherland, A. Noble, „Agile at Scale,“ 2018. [Online]. Available at: https://hbr.org/2018/05/agile-at-scale. [Accessed: 05.11.2021]
[11] M. Virimani, „Understanding DevOps & Bridging the gap from Continuous Integration to Continuous Delivery,“ Firth international conference on Innovative Computing
Technology, 2015, doi: 10.1109/INTECH.2015.7173368
[12] J. Benjamin, J. Mathew, „Enhancing the efficiency of continuous integration environment in DevOps,“ IOP Conf. Series: Materials Science and Engineering, 1085, 2022,
doi: 10.1088/1757-899X/1085/1/012025
[13] E. Laukkanen, J. Itkonen, C. Lassenius, „Problems, causes and solutions when adopting contunuous delivery - A systematic literature review,“ Information and Soft-
ware Tecnology, 82, pp. 55-79, 2017, doi: 10.1016/j.infsof.2016.10.001
[14] P. Rodriguez, A. Haghighatkhah, L.E. Lwakatare, S. Teppola, „Continuous deployment of software intensive products and services: A systematic mapping study,“
2016, doi: 10.1016/j.jss.2015.12.015
[15] N. Forsgren, J. Humble, „The Role of Continuous Delivery IT and Organizational Performance,“ 2016, Proceedings of the Western Decision Sciences Institute (WDSI),
doi: 10.2139/ssrn.2681909
[16] A. Mishra, Z. Otaiwi, „DevOps and software quality: A systematic mapping,“ Computer Science Review, 38, 2020, doi: 10.1016/j.cosrev.2020.100308
[17] DevOps Institute, „2021 Upskilling Enterprise DevOps Skills Report,“ 2021
[18] Puppet, „2021 State of DevOps Report,“ 2021
[19] K. Conboy, S. Coyle, X. Wang, M. Pikkarainen, „People Over Process: Key People Challenges in Agile Development,“ IEEE Software, 2010, doi: 10.1109/MS.2010.132
[20] Atlassian and CITE Research, „2020 DevOps Trends Survey,“ 2020
[21] G. Mayank, „DevOps phases across Software Development Lifecycle,“ TechRxiv, 2020, doi: 10.36227/techrxiv.13207796.v2
[22] DORA, „Accelerate: State of DevOps 2019,“ 2019
[23] McKinsey & Company, „Managing security as code enables companies to create value in the cloud securely,“ 2021. [Online]. Available at: https://www.mckinsey.com/
business-functions/mckinsey-digital/our-insights/security-as-code-the-best-and-maybe-only-path-to-securing-cloud-applications-and-systems. [Accessed: 05.11.2021]
[24] Commerzbank AG, „Open Banking White Paper - The Future of Collaboration in Corporate Banking,“ 2020
[25] J. Kanjilal, „6 lessons from the everything-as-code-shift,“ TechBeacon, 2021. [Online]. Available at: https://techbeacon.com/enterprise-it/6-lessons-everything-code-
shift. [Accessed: 2010.2021]
[26] E. Batraski, „The beginning of „Everything as Code,“ 2020. [Online]. Available at: https://medium.com/swlh/the-beginning-of-everything-as-code-a25c4e9a75e9.
[Accessed: 29.10.2021]
[27] A.A. Zeeshan, „Automating Everything as Code,“ DevSecOps for .NET Core. 2020. doi: 10.1007/978-1-4842-5850-7_4
[28] C. Tozzi, „Everything as Code: What It Is and Why It`s Gaining Traction,“ 2020. [Online]. Available at: https://www.itprotoday.com/development-techniques-and-ma-
nagement/everything-code-what-it-and-why-its-gaining-traction [Accessed: 03.11.2021]
[29] Capgemini, „DevSecOps in Regulated Industries - Accelerating Software Reliability & Compliance,“ 2021
[30] G. Rong, J. Zefeng, H. Zhang, Y. Zhang, W. Ye, D. Shao, „DevDocOps: Enabling continuous documentation in alignment with DevOps,“ 2019, doi: 10.1002/spe.2770
[31] Capgemini, „BizDevOps to enable a product-thinking approach,“ 2020. [Online]. Available: https://www.capgemini.com/2020/10/bizdevops-to-enable-the-product-
thinking-approach/. [Accessed: 27-Oct-2021]
[32] Puppet, CircleCI, Splunk, „State of DevOps Report 2019,“ 2019
[33] Puppet and CircleCI, „State of DevOps Report 2020,“ 2020
[34] Accenture, „From NoOps to DevSecOps and beyond,“ 2020. [Online]. Available: https://www.accenture.com/us-en/blogs/software-engineering-blog/emerson-de-
vops-devsecops. [Accessed: 15.10.2021]
[35] E. Kinsbruner, „How artificial intelligence (AI) and machine learning are changing DevOps,“ 2020. [Online]. Available at: https://enterprisersproject.com/artic-
le/2020/9/artificial-intelligence-ai-and-machine-learning-change-devops. [Accessed: 05.11.2021]
[36] M. Trivendi, „12 Ways AI Is Transforming DevOps,“ n.n.[Online] Available at: https://www.tadigital.com/insights/perspectives/12-ways-ai-transforming-devops. [Ac-
cessed: 05.11.2021]
[37] Open Data Science, „How AI and ML are the Next Evolutionary Step for DevOps,“ 2020. [Online]. Available at: https://medium.com/@ODSC/how-ai-and-ml-are-the-
next-evolutionary-step-for-devops-61565448a468. [Accessed: 03.11.2021]
[38] M. Vizard, „GitLab Acquires UnReview to Further AI Ambitions,“ 2020. [Online]. Available at: https://devops.com/gitlab-acquires-unreview-to-further-ai-ambiti-
ons/#:~:text=GitLab%20announced%20this%20week%20it,previous%20efforts%20and%20current%20workloads. [Accessed: 29.10.2021]
[39] BCG , „Going All In With DevOps,“ 2018
[40] E. Kinsbruner, „How artificial intelligence (AI) and machine learning are changing DevOps,“ 2020. [Online]. Available at: https://enterprisersproject.com/artic-
le/2020/9/artificial-intelligence-ai-and-machine-learning-change-devops. [Accessed: 05.11.2021]
[41] SmartBear Software, „Six Stages from Manual to Autonomous Testing,“ n.n. [Online] Available at: https://smartbear.com/resources/ebooks/six-stages-from-manu-
al-to-autonomous-testing/. [Accessed: 03.11.2021
[42] P. Clauson, „Autonomous Testing`s Role in Agile DevOps,“ 2019. [Online]. Available at: https://devops.com/autonomous-testings-role-in-agile-devops/. [Accessed:
05.11.2021]
[43] Proton Technologies AG, „Complete guide to GDPR compliance,“ 2021. [Online]. Available at: https://gdpr.eu/. [Accessed: 05.11.2021]
[44] McKinsey & Company, „Doing vs being: Practical lessons on building an agile culture,“ 2020. [Online]. Available at: https://www.mckinsey.com/business-functions/
people-and-organizational-performance/our-insights/doing-vs-being-practical-lessons-on-building-an-agile-culture. [Accessed: 05.11.2021]
[45] McKinsey & Company, „Building agile capabilities: The fuel to power your agile „body“,“ 2019. [Online] Available at: https://www.mckinsey.com/business-functions/
people-and-organizational-performance/our-insights/building-agile-capabilities-the-fuel-to-power-your-agile-body. [Accessed: 05.11.2021]
[46] K. Eilers, B. Simmert, C. Peters, „Doing Agile vs. Being Agile - Understanding Their Effects to Improve Agile Work,“ 2020, ICIS 2020 Proceedings.6.
[47] Scaled Agile Inc., „Lean-Agile Leadership,“ 2021. [Online]. Available at: https://www.scaledagileframework.com/lean-agile-leadership/. [Accessed: 14.10.2021]
[48] V. Stray, B. Memon, L. Paruch, „A Systematic Literature Review on Agile Coaching and the Role of the Agile Coach,“ 2020, in M. Morison, M. Torchiano, A. Jedlitschka,
„Product-Focused Software Process Improvement,“ PROFES 2020, Lecture Notes in Computer Science, 12562. doi: 10.1007/9878-3-030-34148-1_1
[49] Atlassian , „Was ist Kanban?,“ n.n. [Online]. Available at: https://www.atlassian.com/de/agile/kanban. [Accessed: 05.11.2021]
[50] Scaled Agile Inc., „Product Owner,“ 2021. [Online]. Available at: https://www.scaledagileframework.com/product-owner/. [Accessed: 05.11.2021]
[51] Scrum.org, „Scrum Glossary,“ 2021. [Online] Available at: https://www.scrum.org/resources/scrum-glossary. [Accessed: 05.11.2021]
[52] Atlassian, „User Storys mit Beispielen und Vorlagen,“ n.n. [Online]. Available at: https://www.atlassian.com/de/agile/project-management/user-stories. [Accessed:
05.11.2021]
[53] I.S. Lasa, C.I. Laburu, R. de Castro Vila, „An evalutaion of the value stream mapping tool,“ Business Process Management Journal, Vol. 14, No. 1, 2008, pp. 39 - 52

38
Commerzbank AG
Head Office
Kaiserplatz
Frankfurt am Main
www.commerzbank.de

Postal Address
60261 Frankfurt am Main
Tel.: +49 69 136-20
info@commerzbank.com

You might also like