Front cover

IBM Fibre Channel Endpoint Security

for IBM DS8900F and IBM Z

Roger Hathorn
Matthew Houzenga
Jacob Sheppard
Robert Tondini
Alexander Warmuth
Bert Dufrasne

Redbooks
IBM Redbooks

IBM Fibre Channel Endpoint Security

for IBM DS8900F and IBM Z

January 2020

SG24-8455-00
 Note: Before using this information and the product it supports, read the information in “Notices” on page v.

First Edition (January 2020)

This edition applies to IBM DS8000 with Licensed Machine Code (LMC) 7.9.0 (bundle version 89.0), referred
to as Release 9.0, along with the new model IBM DS8900F, IBM z15, and SKLM 3.0.1.3 or later

© Copyright International Business Machines Corporation 2020. All rights reserved.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Now you can become a published author, too . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Chapter 1. Introducing IBM Fibre Channel Endpoint Security . . . . . . . . . . . . . . . . . . . 11

1.1 The need for data protection and encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2 IBM Fibre Channel Endpoint Security overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.2.1 IFCES design overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Chapter 2. IBM Fibre Channel Endpoint Security solution design. . . . . . . . . . . . . . . . 17

2.1 Required Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.1 External key manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.2 Endpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 IFCES settings and policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 Establishing IFCES between an initiator - target port pair . . . . . . . . . . . . . . . . . . . . . . 22
2.4 Key renewal policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Chapter 3. Endpoint Security Requirements and Planning . . . . . . . . . . . . . . . . . . . . . 27

3.1 Planning and implementation process flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2 Digital certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.1 DS8900F digital certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.2 z15 HMC digital certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2.3 Security Key LIfecycle Manager digital certificate. . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3 DS8900 ordering and configuring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.4 IBM z15 ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.5 Security Key Lifecycle Manager ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.5.1 SKLM Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.5.2 IBM Storage Appliance Model AP1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.5.3 IBM Lab services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.6 Requirements for IBM Fibre Channel Endpoint Security. . . . . . . . . . . . . . . . . . . . . . . . 32
3.6.1 Key expiry policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6.2 Digital certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6.3 Encryption key manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6.4 IBM z15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6.5 FICON Directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6.6 Host Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Chapter 4. Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1 IBM Z CPC configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1.1 Define External Key Servers and export certificate to SKLM . . . . . . . . . . . . . . . . 36
4.1.2 Configure IBM Z CPC policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.2 DS8900F configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2.1 Export SKLM server SSL/KMIP certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2.2 Enable Endpoint Security on the DS8900F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

© Copyright IBM Corp. 2020. All rights reserved. iii

 4.2.3 Enable Endpoint Security on DS8900F host ports . . . . . . . . . . . . . . . . . . . . . . . . 56

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment . . . . . . . 67

5.1 Managing and Monitoring Endpoint Security on the DS8900 . . . . . . . . . . . . . . . . . . . . 68
5.1.1 DS8900F GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.1.2 DS8900 command-line interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.2 Monitoring and maintaining endpoint encryption on SKLM . . . . . . . . . . . . . . . . . . . . . . 75
5.2.1 SKLM backup and restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.2.2 SKLM key manager device group verification. . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.3 Monitoring Endpoint Encryption on the z15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.3.1 Modify Fibre Channel Endpoint Security Policy from the z15 HMC . . . . . . . . . . . 79
5.3.2 z/OS commands to display Fibre Channel Endpoint Security status . . . . . . . . . . 81
5.3.3 z/OS Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.3.4 z/OS SMF Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Chapter 6. Managing Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

6.1 Introduction to the certificates used with IFCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.1.1 What are digital certificates used for . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.1.2 Special considerations for the DS8900F encryption certificate. . . . . . . . . . . . . . . 88
6.2 Modify customer defined certificate for use with IFCES . . . . . . . . . . . . . . . . . . . . . . . . 88
6.3 Changing digital certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.3.1 Changing the SSL/KMIP digital certificate for SKLM . . . . . . . . . . . . . . . . . . . . . . 92
6.3.2 Changing the digital certificate on the IBM Z HMC . . . . . . . . . . . . . . . . . . . . . . . . 97
6.3.3 Changing the Digital Certificate for the DS8900F . . . . . . . . . . . . . . . . . . . . . . . . 103

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

iv IBM Fibre Channel Endpoint Security

Notices

This information was developed for products and services offered in the US. This material might be available
from IBM in other languages. However, you may be required to own a copy of the product or product version in
that language in order to access it.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user’s responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not grant you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive, MD-NC119, Armonk, NY 10504-1785, US

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION “AS IS”

WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A
PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties in
certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.

Any references in this information to non-IBM websites are provided for convenience only and do not in any
manner serve as an endorsement of those websites. The materials at those websites are not part of the
materials for this IBM product and use of those websites is at your own risk.

IBM may use or distribute any of the information you provide in any way it believes appropriate without
incurring any obligation to you.

The performance data and client examples cited are presented for illustrative purposes only. Actual
performance results may vary depending on specific configurations and operating conditions.

Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.

Statements regarding IBM’s future direction or intent are subject to change or withdrawal without notice, and
represent goals and objectives only.

This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to actual people or business enterprises is entirely
coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs are
provided “AS IS”, without warranty of any kind. IBM shall not be liable for any damages arising out of your use
of the sample programs.

© Copyright IBM Corp. 2020. All rights reserved. v

Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines
Corporation, registered in many jurisdictions worldwide. Other product and service names might be
trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright
and trademark information” at http://www.ibm.com/legal/copytrade.shtml

The following terms are trademarks or registered trademarks of International Business Machines Corporation,
and might also be trademarks or registered trademarks in other countries.
DS8000® Passport Advantage® z Systems®
FICON® Redbooks® z/OS®
IBM® Redbooks (logo) ® z/VM®
IBM Z® Resource Link® z/VSE®
IBM z Systems® System Storage™ z15™
IBM z15™ XIV®

The following terms are trademarks of other companies:

The registered trademark Linux® is used pursuant to a sublicense from the Linux Foundation, the exclusive
licensee of Linus Torvalds, owner of the mark on a worldwide basis.

Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other
countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Other company, product, or service names may be trademarks or service marks of others.

vi IBM Fibre Channel Endpoint Security

Preface

This IBM® Redbooks® publication will help you install, configure, and use the new IBM
Fibre Channel Endpoint Security function.

The focus of this publication is about securing the connection between an IBM DS8900F and
the IBM z15™.

The solution is delivered with two levels of link security supported: support for
link authentication on Fibre Channel links and support for link encryption of data in flight
(which also includes link authentication).

This solution is targeted for clients needing to adhere to Payment Card Industry (PCI) or other
emerging data security standards, and those who are seeking to reduce or eliminate insider
threats regarding unauthorized access to data.

Authors
This book was produced by a team of specialists from around the world.

Roger Hathorn is a Senior Technical Staff Member and Master

Inventor at IBM Systems Storage in Tucson, AZ. He is
responsible for IBM DS8000® I/O architecture and
development with expertise in the IBM FICON® and Fibre
Channel I/O Architecture and standards, and has over 31 years
of experience on the enterprise storage platform. Roger
represents IBM at INCITS Fibre Channel T11 standards,
including the responsibility of vice-chair of the T11.3 technical
committee, chairman of the FC-SB-6 (FICON) working group,
and secretary of the FC-NVMe working group. Roger is the
lead architect of the IBM Fibre Channel Endpoint Security
solution on the new DS8900F storage controllers.

Matthew Houzenga is an Executive IT Specialist and Client

Technical Specialist for IBM Storage in Cleveland, OH. He
joined IBM Storage in 2000, and worked for the support and
development organizations in San Jose and Tucson before
moving to the field in 2006. He holds a degree in Computer
Science from Northern Illinois University.

© Copyright IBM Corp. 2020. All rights reserved. vii

 Jacob Sheppard joined the IBM DS8000 device adapter
development team in 2003, where he worked on problems of
multi-node management of metadata, logical volume
expansion, and solid state disk control. He began working on
encryption of data at rest using self encrypting spinning disks in
2008. In 2012, he helped implement the encryption of data at
rest solution for IBM XIV®. In 2014, he returned to IBM
DS8000 development to tackle problems of KMIP
standardization for data at rest encryption, customer defined
certificates, Transparent Cloud Tiering encryption, and
Encryption of Data in Flight.

Robert Tondini is an IBM Consulting IT Specialist in IBM

Australia and New Zealand. He has 24 years of experience in
IBM enterprise storage for mainframe and open systems. He
joined IBM in 2000, and since then he has been providing
presales and implementation support for high-end disk, tape,
and SAN fabric systems with high availability and disaster
recovery solutions. He co-authored several IBM Redbooks
publications and workshops for IBM DS8000 systems.

Alexander Warmuth is a Consulting IT Specialist in IBM’s

European Storage Competence Center. Working in technical
sales support, he designs and promotes new and complex
storage solutions, drives the introduction of new products, and
provides advice to customers, IBM Business Partners, and
sales. His main areas of expertise are: high-end storage
solutions and business resilience for IBM z Systems® and
Linux. He joined IBM in 1993. Alexander holds a diploma in
Electrical Engineering from the University of Erlangen,
Germany.

Bert Dufrasne is an IBM Certified Consulting IT Specialist and

Project Leader for IBM System Storage™ disk and flash
products at the ITSO, San Jose Center. He has worked at IBM
in various IT areas. He has written many IBM Redbooks
publications, and has developed and taught technical
workshops. Before joining the ITSO, he worked for IBM Global
Services as an Application Architect. He holds a Master’s
degree in Electrical Engineering.

Thanks to the following people for their contributions to this project:

Larry Brocious
Pasquale Catalano
Rashmi Chandra
Justin Crips
Andrew Crimmins
Patty Driever
Donna Freck
Igor Popov
IBM

viii IBM Fibre Channel Endpoint Security

Now you can become a published author, too
Here’s an opportunity to spotlight your skills, grow your career, and become a published
author—all at the same time. Join an IBM Redbooks residency project and help write a book
in your area of expertise, while honing your experience using leading-edge technologies. Your
efforts will help to increase product acceptance and customer satisfaction, as you expand
your network of technical contacts and relationships. Residencies run from two to six weeks
in length, and you can participate either in person or as a remote resident working from your
home base.

Find out more about the residency program, browse the residency index, and apply online at:
ibm.com/redbooks/residencies.html

Comments welcome
Your comments are important to us.

We want our books to be as helpful as possible. Send us your comments about this book or
other IBM Redbooks publications in one of the following ways:
򐂰 Use the online Contact us review Redbooks form:
ibm.com/redbooks
򐂰 Send your comments in an email:
redbooks@us.ibm.com
򐂰 Mail your comments:
IBM Corporation, IBM Redbooks
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400

Stay connected to IBM Redbooks

򐂰 Find us on Facebook:
http://www.facebook.com/IBMRedbooks
򐂰 Follow us on Twitter:
http://twitter.com/ibmredbooks
򐂰 Look for us on LinkedIn:
http://www.linkedin.com/groups?home=&gid=2130806
򐂰 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks
weekly newsletter:
https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm
򐂰 Stay current on recent Redbooks publications with RSS Feeds:
http://www.redbooks.ibm.com/rss.html

Preface ix
x IBM Fibre Channel Endpoint Security
 1

Chapter 1. Introducing IBM Fibre Channel

Endpoint Security
IBM Fibre Channel Endpoint Security (IFCES) is designed to protect data that is transferred
over Fibre Channel storage area networks (SANs). It consists of two components:
򐂰 Link authentication
򐂰 Encryption of data in flight (EDIF)

In this chapter, we first discuss the general need for data protection and encryption. Then we
explain the benefits of IFCES and how it fits into the IBM Z® Pervasive Encryption strategy.

We complete the introduction with a high-level functional overview of the solution.

© Copyright IBM Corp. 2020. All rights reserved. 11

1.1 The need for data protection and encryption
Today, data stored and processed in IT systems is one of the core assets of most enterprises
or organizations. Losing or exposing data often results in high cost or even irreparable
damage. In addition, more and more regulatory requirements are introduced, forcing
organizations to protect the data they store and process, inducing severe penalties if
requirements are not met or sensitive data is lost or exposed. Thus, organizations are
experiencing increased pressure from internal and external sources to protect and govern
data.

The two main aspects of data protection are the following goals:
򐂰 Protection against loss: losing access to data can have a severe impact on an
organization’s ability to function, but will generally have limited impact on third parties. In
the past, most of the efforts of data protection focused on this aspect. Hardware
redundancy, backup and restore processes, or disaster recovery solutions are examples
of methods used here.
򐂰 Protection against unauthorized access and abuse: losing control of data not only impacts
the storing and processing organization itself, but also other organizations or persons it is
interacting with. This aspect is gaining significance in recent years as data breaches and
abuse are reported frequently. Furthermore, data is increasingly stored in cloud
environments outside of an organization’s own data center where it is much harder to
control. Here, the most effective methods of protection are access control and encryption.

With the IBM Z Systems z14, IBM introduced the concept of Pervasive Encryption. IBM Z
clients are no longer required to put excessive effort into planning, implementing, and
maintaining effective access control and encryption of their data. Pervasive encryption
provides the means to encrypt all data at all levels and in all components of the IT
infrastructure, without impacting the operation and without requiring changes to existing
applications.

Figure 1-1 shows a graphical representation of the layers where encryption can take place.

Figure 1-1 The IBM Pervasive Encryption Pyramid

The width of each layer represents the coverage that can be achieved related to overall
protection. The vertical position of the layers represent the granularity of control, but also the
complexity of implementation and management.

12 IBM Fibre Channel Endpoint Security

 Starting from the top, we briefly explain each layer. You can also refer to Getting Started with
z/OS Data Set Encryption, SG24-8410, for more details about Pervasive Encryption.
򐂰 Application encryption provides encryption and data protection by each individual
application. It is highly granular and specific, but also requires the highest efforts, because
each application needs the necessary encryption capabilities and has to be managed
individually. It can provide protection of highly sensitive application data, which is not
covered by any of the lower levels or if their protection is not sufficient.
򐂰 Database encryption provides the capability to protect key database files and database
backup images from inappropriate access. It is less granular and therefore easier to
manage, but covers only a certain subset of data.
򐂰 File or data set encryption provides broad coverage for sensitive data by using encryption
that is tightly integrated with the operating system and managed by policies. It is not
apparent to applications and allows for separation of duties within an organization.
Security administration can be performed independently of application, database, or
storage administration.
򐂰 Disk and tape encryption provide coverage for data-at-rest at the storage infrastructure
level. It is an “all or nothing” solution and encrypts data at rest within a storage controller
without differentiating type, sensitivity, or importance of data. Therefore it requires the
least organizational effort of all layers with the broadest coverage. It protects against
intrusion, tampering, or removal of physical infrastructure.

For the upper three levels shown in the pyramid, data is encrypted on the host side. Therefore
it is protected at rest on external storage media, as well as in flight, while being read or
written. With conventional disk encryption, data is unprotected as long as it is outside of the
respective storage system.

IBM Fibre Channel Endpoint Security (IFCES) adds the protection of data in flight between
the IBM Z and the IBM DS8900F storage system, controlling access and encrypting data that
is transferred over a SAN.

Note: Only data transferred between an IBM Z CPC and IBM DS8900F storage systems
can be protected with IFCES. Data in flight is not protected in the following situations:
򐂰 On PPRC replication links between DS8000 storage systems
򐂰 Between an IBM Z CPC and virtual or physical tape devices

In addition, IFCES can also be used to protect data flowing between hosts using Channel to
Channel (CTC) connections.

1.2 IBM Fibre Channel Endpoint Security overview

While data set encryption is supported through most IBM z/OS® access methods, there are
some that do not yet support it. Utilizing datasets encryption also requires a set of z/OS
components to be functional in order to perform the encryption and key management, so
operations on the link prior to those components being initialized and functional are not
protected by encryption. IBM Z also supports 5 operating systems (z/OS, Linux, z/TPF, IBM
z/VSE®, and IBM z/VM®), and not all of them provide file or datasets encryption capability for
all file systems or access methods.

DS8000 Data at Rest (DAR) encryption provides protection for all data stored within the
storage system, regardless of application, access method, or operating system. However, it

Chapter 1. Introducing IBM Fibre Channel Endpoint Security 13

 does not protect the data on its way (in flight) between the IBM Z CPC and the storage
system.

Figure 1-2 on page 14 illustrates how IBM Fibre Channel Endpoint Security complements
DAR encryption and extends the protection to data in flight.

Figure 1-2 The protection scope of IFCES

The blue logical volumes in the storage systems and the datasets or files stored in them
represent the level of protection from conventional DAR encryption. The orange parts,
consisting of the FICON or FCP HBAs in the IBM Z CPC, all SAN components, and the host
adapters in the DS8000 storage system show the extent of additional protection provided by
IFCES.

Note: IFCES also supports Fibre Channel-to-Channel (CTC) connections.

1.2.1 IFCES design overview

In this section, we provide a high-level overview of the IFCES solution. For a detailed
explanation see Chapter 2, “IBM Fibre Channel Endpoint Security solution design” on
page 17.

Solution components
IFCES is a SAN end-to-end solution, and requires support from both endpoints, the IBM Z
CPC (initiator) and the DS8900F storage system (target). Both sides provide encryption
capability and support for the creation, reception, interpretation, and transmission of
messages that are exchanged to establish endpoint security. Fibre Channel fabric
components, like switches, also have to support the solution.

Note: IFCES can be used with switched links and direct Fibre Channel point-to-point
connections.

For secure key management, the solution also requires an external key manager. It maintains
the shared secrets (keys) that associate the IBM Z CPCs and the DS8000 storage systems
with each other as trusted partners.

14 IBM Fibre Channel Endpoint Security

 Note: IBM Security Key Lifecycle Manager Version 3.0.1.3 or higher is the required
External Key Manager. For IFCES, we recommend SKLM version 4.0.0.2 or later.

Figure 1-3 on page 15 shows the major components involved in IFCES.

Figure 1-3 Main IFCES components

The endpoints communicate with the key manager through the Hardware Management
Consoles (HMCs) of the IBM Z CPC and the DS8000 storage system, using Transport Layer
Security (TLS) to establish secure connections. The Fibre Channel endpoint ports use inband
Fibre Channel link services to set up the trusted and encrypted connections for IFCES,
governed by the IBM Security Key Exchange (SKE) protocol, which IBM developed on the
basis of the industry standard Fibre-Channel Security Protocols 2 (FC-SP 2).

Note: See Chapter 3, “Endpoint Security Requirements and Planning” on page 27 for
detailed hardware, firmware, and key manager requirements.

Solution design
IBM Fibre Channel Endpoint Security is initiated in two phases:
1. Link authentication: ensure that only ports of trusted endpoints are communicating with
each other (a common symmetric key is used for authentication)
2. Link encryption: establish encryption of all payload traffic between the endpoints using a
pair of derived symmetric keys.

When you set up your environment for IFCES, a Device Authentication Key (DAK) is created
in the External Key Manager. It acts as a shared secret that associates a pair of endpoint
devices as trusted partners. We refer to these devices as host or initiator and storage or
target.

Secure communication between a pair of Fibre Channel ports (initiator port and target port)
from a trusted endpoint device pair is always initiated by the initiator port. It happens
according to the following high-level sequence:
1. The initiator retrieves the DAK from the External Key Manager.
2. The initiator notifies the target that it wants to setup a protected connection.

Chapter 1. Introducing IBM Fibre Channel Endpoint Security 15

 3. The target on its part requests the DAK from the External Key Manager
4. Initiator and target each create a symmetric encryption key independently, using a
common algorithm and the DAK.
5. Initiator and target use their common encryption key to authenticate with each other and
establish encryption if possible.

16 IBM Fibre Channel Endpoint Security

 2

Chapter 2. IBM Fibre Channel Endpoint

Security solution design
In this chapter, we cover the IBM Fibre Channel Endpoint Security solution design.

© Copyright IBM Corp. 2020. All rights reserved. 17

2.1 Required Components
We describe the solution components and how they securely communicate with each other.
Then we go through the available policies and settings for IFCES and their effects. Next, we
explain how security is established between two endpoints. Finally, we give an overview of
what happens when you change settings or policies.

IBM Fibre Channel Endpoint Security (IFCES) involves three major components:
򐂰 Two endpoint devices:
– A z15, IBM Z Central Processing Complex (CPC), also referred to as the initiator.
– An IBM DS8900F storage system, also referred to as the target.
IFCES is set up between individual FICON or Fibre Channel port pairs of these devices.
For FICON, the ports are logically connected through the definitions in the Input Output
Control Data Set (IOCDS) of the host system. IFCES requires no changes to the IOCDS.
򐂰 An external key manager that maintains the shared secrets that identify the trusted
relationships between endpoint devices.

See Figure 1-3 on page 15 for an illustration of the IFCES infrastructure.

2.1.1 External key manager

The External Key Manager maintains the shared secrets that associate pairs of host and
storage system, or initiators and targets. Such a shared secret is called a Device
Authentication Key (DAK). It is stored securely in the External Key Manager key store. To
make sure that a DAK is always available when needed, the External Key Manager must be
redundant and configured for continuous availability.

Important: If IFCES is in place and enforced, the External Key Manager is a crucial
component during startup of either an IBM Z CPC or any connected DS8900F storage
system. Should it be unavailable, Fibre Channel connections between host and storage will
fail to initialize and data cannot be accessed.

The endpoint devices retrieve the DAK from the External Key Manager when needed:
򐂰 When endpoint security is set up for the first time (for example at power up)
򐂰 When the DAK is renewed according to the specified IFCES policies.

During normal operation, each endpoint device maintains a copy of the DAK in its local key
manager (LKM) to avoid excessive External Key Manager traffic. Note that the LKM function
is implemented n the Z firmware.

External Key Manager and endpoint devices use the industry standard Key Management
Interoperability Protocol (KMIP) for their communication. IBM developed an extension to the
KMIP protocol that adds support for peer-to-peer device groups, which enables you to store
the trusted association of two devices (peers). The peers are the IFCES endpoints, with the
IBM Z system being the owner of the group, and the DS8900F storage system the partner.
The Fibre Channel World Wide Node Names (WWNNs) of the endpoint devices are used to
provide unique identification. One such device group is created and maintained by the
External Key Manager for each IBM Z CPC and DS8900F pair.

18 IBM Fibre Channel Endpoint Security

 Figure 2-1 shows an example with two device groups, associating an IBM Z CPC with two
DS8900F systems.

Figure 2-1 Peer to peer device group example

The External Key Manager composes the names of the peer-to-peer device groups from the
WWNNs of the peers (endpoint devices) in the group. The device group contains the security
credentials (certificates) that the External Key Manager needs to communicate with each of
the peers and their common DAK. The peer to peer device group and initial DAK are created
on the External Key Manager automatically at the request of the IBM Z CPC.

When endpoints need to retrieve the DAK from the External Key Manager, they provide the
WWNNs of both peers and identify themselves with their certificate. The External Key
Manager then pulls the DAK from the matching device group and passes it to the requesting
peer. Note that the device certificates are presented to the External Key Manager by the
HMCs (IBM Z and DS8900F) on behalf of the initiator and target, respectively. This is
out-of-band communication.

Note: IBM Security Key Lifecycle Manager (SKLM) configured in multi master mode is the
only External Key Manager solution supported for IFCES. See Chapter 3, “Endpoint
Security Requirements and Planning” on page 27 for more detail.

2.1.2 Endpoints
In this section, we describe the endpoint functionality in more detail and explain the high-level
steps that are required to configure IFCES. See Chapter 4, “Implementation” on page 35 for
detailed instructions and examples.

IBM Z CPC

Note: An IBM z15 with at least one encryption-capable FICON adapter

(FICON Express 16SA feature) is required for IFCES. See Chapter 3, “Endpoint Security
Requirements and Planning” on page 27 for more detail.

Chapter 2. IBM Fibre Channel Endpoint Security solution design 19

 From a Fibre Channel or FICON view, the IBM Z CPC is the initiator of all I/O operations. It is
also the initiator of all IFCES configuration activities. You can configure and enable IFCES on
an IBM Z CPC when certain requirements are met:
򐂰 The IBM Fibre Channel Endpoint Security feature is activated.
򐂰 The CP Assist Cryptographic Facility (CPACF) feature is activated (note however that the
Fibre Channel Security Endpoint Security feature cannot be ordered without CPACF
enablement anyway).
򐂰 At least one encryption capable Fibre Channel adapter is installed.

To enable IFCES for an IBM Z CPC, you only have to define the External Key Manager (IBM
SKLM servers
IP addresses or host names, and port numbers) in the corresponding configuration panel in
the IBM z Hardware Management Console (HMC). The IFCES firmware running on the IBM Z
CPC performs all the necessary steps to set up secure communication to the External Key
Manager, using the HMC as the communication gateway and user interface.

The IFCES firmware in the IBM Z CPC retrieves the DAK from the External Key Manager
when needed and stores the DAK in the LKM. The local copy of the DAK is used for normal
operations to avoid excessive External Key Manager traffic. The LKM utilizes the DAK in the
authentication sequence with target peers and initiates a key renewal of the DAK when it is
due.

This part of the IBM Z firmware runs in an encapsulated container to make sure that the
secrets it keeps cannot be compromised, for example when a dump or trace is generated. It
doesn’t consume client memory or processing power, because it runs with system internal
resources.

IBM DS8900F storage system

Note: Only IBM DS8900F storage systems support IFCES. See Chapter 3, “Endpoint
Security Requirements and Planning” on page 27 for more detail.

Storage systems are targets for I/O operations, and also targets for IFCES configuration
requests. A DS8900F receives an IFCES configuration request from an IBM Z CPC and acts
on it. The DS8900F does not initiate endpoint protection on its own.

To support IFCES, a DS8900F must be able to access the same External Key Manager (IBM
SKLM servers) as the IBM Z CPC.

Important: Only one set of key servers from the same multi-master cluster is configurable
on any Z CPC and on the DS8900F storage system.

You can define the SKLM servers to the DS8900F through its HMC, using either the GUI or
the DSCLI. In addition to providing the SKLM servers’ IP addresses and port numbers, you
also have to make sure that credentials for secure communication between DS8900F and
SKLM are in place:
򐂰 Export communication certificates from the SKLM servers and import them to the
DS8900F.
򐂰 The factory-installed default communication certificates of a DS8900F or Z CPC is known
and trusted by the SKLM servers. You don’t have to take any further action if you intend to
use those certificates.
򐂰 If you intend to use another certificate, you have to install it on the DS8900F and import it
to the SKLM servers.

20 IBM Fibre Channel Endpoint Security

 Because the WWNNs of the endpoints are used to associate them with each other, you also
have to make sure that any certificate you provide for an endpoint contains its WWNN in the
Subject Alternative Name field. See “Modify customer defined certificate for use with IFCES”
on page 88 for detailed instructions. The factory-installed certificates already meet this
requirement.

Note: If a DS8900F is already configured for another type of encryption (data at rest or
Transparent Cloud Tiering), the existing SKLM servers can also be used as External Key
Manager for IFCES, as long as they meet the IFCES requirements.

Similar to the IBM Z CPC, the HMC is acting mainly as a user interface and communication
gateway. Communication to the External Key Manager is initiated by the DS8900F server
nodes. Both of them also have their own LKM to keep the DAK during normal operations.

After you complete the steps to configure a DS8900F for IFCES, it will contact the External
Key Manager and test whether it can perform all necessary KMIP operations.

Note: For these tests, the DS8900F initiates the creation of a special peer-to-peer device
group with the External Key Manager. In the SKLM list of device groups, you can identify it
by its name consisting of the letter “D” followed by the DS8900F's WWNN, repeated twice,
indicating it is both owner and partner of the group.

Fibre Channel endpoints

Until now, we only looked at the endpoint devices (host and storage) at a system level, the
entire z15 or DS8900F. However, IFCES is a function that protects data flowing between
individual Fibre Channel port pairs:
򐂰 Host or initiator ports
򐂰 Storage or target ports

The port pairs perform the authentication and encryption setup individually, as we explain in
2.3, “Establishing IFCES between an initiator - target port pair” on page 22. They
communicate inband over Fibre Channel and use the IBM Secure Key Exchange (SKE)
protocol, developed by IBM on the basis of the industry standard Fibre-Channel Security
Protocols 2 (FC-SP 2).

2.2 IFCES settings and policies

If all hardware and software requirements are met and you successfully defined the External
Key Manager to both endpoint devices (initiator and target), you can switch on endpoint
security on a storage system host adapter (target) port level. A target port can be configured
to one or more initiator ports by the IOCDS. The policy setting affects all possible connections
to this target port from any configured initiator ports. You can set each port individually to one
of three IFCES policies:
Disabled The target port does not signal IFCES capability to the initiators.
Therefore, all attached initiator ports do not try to set up endpoint
security. The Fibre Channel endpoint pairs act as if IFCES did not
exist.
Enabled The target port signals IFCES capability to the initiators. If an attached
initiator port is IFCES capable, it tries to set up endpoint security.
However, the port pair can set up a connection, regardless of whether
the authorization succeeds. If a host port is not endpoint security

Chapter 2. IBM Fibre Channel Endpoint Security solution design 21

 capable, or if the necessary connection to SKLM has not been setup
yet, it cannot start the IFCES authorization sequence, but can also
connect to the target without it. This policy is also called Audit Mode
because you can use it to verify the IFCES configuration without
impacting access to data.
Enforced The target port signals IFCES capability to the initiators. If an attached
initiator port is IFCES capable, it tries to set up endpoint security. If the
authorization succeeds, the port pair can connect. If the authorization
fails, the port pair cannot set up a connection. If a host port is not
endpoint security capable or if the necessary connection to SKLM has
not been setup yet, the connection will also fail.

Whenever you change the IFCES policy of a target port, this port goes offline (drop light) and
the IFCES negotiation starts from the beginning. This way, you can switch IFCES on and off
while the systems are in operation.

There will be a short period of time where the affected port pairs cannot transfer data, until
the connections are established again. This interruption is handled by the z/OS Input Output
Supervisor (IOS) multipathing, and is transparent to the running applications.

Important Notes:
򐂰 It is advised to vary off paths to the target port before changing security settings.
Change only one port at a time, and verify paths are back online before proceeding to
the next port.
򐂰 To avoid unexpected data access issues, it is good practice to verify the endpoint
security capability and configuration of all affected endpoints using the Enabled policy
before switching to Enforced. See 5.1, “Managing and Monitoring Endpoint Security on
the DS8900” on page 68 for methods to do so.

2.3 Establishing IFCES between an initiator - target port pair

When the IBM Z firmware first attempts a connection from a FICON or Fibre Channel Protocol
(FCP) initiator port to a target port for which such a connection is defined (e.g, in the IOCDS
for FICON) and allowed (e.g. in Fibre Channel SAN zoning), if the initiator port is IFCES
capable it will determine the IFCES capability of the target port based on the login parameters
set by the target. The following reactions are possible:
򐂰 A target port is not IFCES capable, because it belongs to an older or non-IBM storage
system. In this case, it does not indicate support during login. The initiator port does not
make attempts to set up IFCES and handles the connections conventionally.
򐂰 A target port is IFCES capable, but its policy is Disabled. In this case it does not indicate
support during login Again, the initiator port does not make attempts to set up IFCES and
handles the connections conventionally.
򐂰 A target port is IFCES capable and its policy is either Enabled or Enforced. In this case, it
does indicate support during login and the initiator starts the authorization sequence.
Whether a connection can be established depends on the results of the authorization
sequence and on the policy setting (see “IFCES settings and policies” on page 21).

The initiator starts the authorization sequence. It is performed inband over the Fibre Channel
link to the target using the IBM SKE protocol. If the DAK is not yet available in the endpoint
device LKM, it will involve KMIP requests to the External Key Manager.

22 IBM Fibre Channel Endpoint Security

In the following section, we describe the IFCES authorization sequence including the External
Key Manager requests that might be required. Figure 2-2 illustrates the description. We cover
the high-level messaging going on between the affected components. We do not include the
low-level handshaking to avoid unnecessary complexity.

Figure 2-2 IFCES authorization sequence

Steps A, B, C, D, and E provide the initiator and target device with the shared secret (DAK)
that they need to authenticate with each other. These steps occur only once for each endpoint
device (host - storage) pair. After they have been processed, the initiator and target device
both have the DAK in their respective LKM as long as they remain powered on or until the
DAK is renewed. Steps 1 through 6 are performed for each Fibre Channel endpoint port pair
that is trying to set up IFCES.
Step A If the initiator does not have the DAK for the affected initiator - target
device pair in its LKM, it sends a KMIP request to the External Key
Manager to create this key. If the required peer-to-peer device group
does not exist yet, the External Key Manager automatically creates it.
Step B The initiator still does not have the DAK. It sends another KMIP
request to the External Key Manager to retrieve the DAK, identifying
itself and the target as a trusted pair with their WWNNs.
Step C The External Key Manager returns the requested DAK to the initiator.
Step 1 The initiator port sends an SKE authentication notification to the target
port.
Step D If the target device does not have the DAK for the initiator - target
device pair in its LKM, it sends a KMIP request to the External Key
Manager to retrieve this key, also identifying itself and the initiator as a
trusted pair with their WWNNs.
Step E The External Key Manager returns the requested key to the target.
Step 2 The target port confirms to the initiator port that is has the DAK for
their pair.
Step 3 Both, initiator and target port create a unique symmetric one-time port
level authentication key. They use the same algorithm, based on the

Chapter 2. IBM Fibre Channel Endpoint Security solution design 23

 common DAK and a pair of random tokens. These tokens ensure that
the key cannot be recreated elsewhere.
Step 4 The initiator port sends the authentication request to the target port,
encrypted with the previously created one-time key. It identifies itself
with its Fibre Channel World Wide Port Name (WWPN). The target
can decrypt the request, because it has the same key.
Step 5 The target port acknowledges the authentication request, also
identifying itself with its WWPN and also encrypting this message
using the common one-time key.
Step 6 The target port notifies the initiator port that the authentication
completed successfully.

With step 6 completed, the ports have successfully authenticated with each other and are
allowed to set up a Fibre Channel port to port connection. If both ports are also capable of
encrypting data (EDIF), they amend the sequence with two more steps to set up encryption,
as illustrated in Figure 2-3.

Figure 2-3 Setting up encryption of data in flight (EDIF) after authentication

Step 7 Both ports create two more unique symmetric keys for the port pair,
one for each direction, called data encryption keys (DEKs), using the
same algorithm as in step 3.
Step 8 The ports use the new DEK pair in all subsequent Fibre Channel
exchanges to encrypt the frame payload data.

The DEKs created in step 7 are used for encryption until an encryption key renewal is
scheduled or until one of the two ports is reset, for example due to a change in the IFCES
policy or a link recovery or initialization. Hardware supported symmetric AES256 encryption
is used for EDIF to allow for effective encryption at Fibre Channel line rates.

2.4 Key renewal policies

Both the DAK and DEK do not expire implicitly. They can theoretically be in use forever.
However, the IBM Z firmware has a key expiration policy built in. It triggers renewal of both
keys at defined intervals. Both intervals have default settings:

24 IBM Fibre Channel Endpoint Security

򐂰 The DAK default renewal period is 7 days.
򐂰 The DEK default renewal period is 8 hours.

You can change both values according to your security requirements in the IBM z HMC
endpoint security setup panels. Refer to 5.3, “Monitoring Endpoint Encryption on the z15” on
page 79 for details.

To renew the DAK, the endpoint devices have to be able to access the External Key Manager.
If the External Key Manager should not be accessible for any reason, the DAK cannot be
renewed. The old key stays active until access to the External Key Manager is restored.
Access to data is not compromised by key expiration. However, your security requirements
might be violated if the External Key Manager is not available for extended periods of time.

If key renewal fails for any reason, the IBM z IFCES firmware reports hardware messages in
the IBM z HMC and also notifies the operating systems running in the affected Z systems for
auditing purposes. See 5.3, “Monitoring Endpoint Encryption on the z15” on page 79 for
details.

Note: Unlike the encryption for data-at-rest (DAR), where the keys are also needed to
decrypt when reading the data, the keys used for IFCES (DAK and DEK) are no longer
required when the data has reached the target storage. The system can create new keys
as needed, without compromising data access and without interruption to active
workloads.

Chapter 2. IBM Fibre Channel Endpoint Security solution design 25

26 IBM Fibre Channel Endpoint Security
 3

Chapter 3. Endpoint Security Requirements

and Planning
This chapter describes the planning process and requirements for implementing IBM Fibre
Channel Endpoint Security. It covers:
򐂰 Planning and implementation process flow
򐂰 Digital certificates
򐂰 DS8900 ordering and configuring
򐂰 IBM z15 ordering
򐂰 Security Key Lifecycle Manager ordering
򐂰 Requirements for IBM Fibre Channel Endpoint Security

© Copyright IBM Corp. 2020. All rights reserved. 27

3.1 Planning and implementation process flow
A high-level overview of the planning and implementation process for Fibre Channel Endpoint
Security is as follows:
1. Plan for digital certificates
– Decide on IBM-provided or customer-provided digital certificates, with an
understanding of possible restrictions.
For Ds8900F certificates, decide between Gen 2 or Gen 3, IBM provided digital
certificates.
2. Order DS8900 with required features
3. Order z15 with required features
4. Plan for Security Key Lifecycle Manager
– Decide on server platform and Operating System (OS)
5. Order Security Key Lifecycle Manager Software
6. Install SKLM software in Multi-Master configuration on at least two servers
7. Connect z15 HMC to SKLM
8. Connect DS8900 to SKLM
9. Enable port security on DS8900 ports

3.2 Digital certificates

This section describes the different digital certificates and options.

3.2.1 DS8900F digital certificate

There are two different certificates that are pre-installed on DS8900 Release 9.0, called
Gen-2 and Gen-3. With the DS8900F, both Gen-2 and Gen-3 certificates are IFCES enabled,
containing the WWNN of the DS8900F in the Subject Alternative Name field. Note that Gen-1
certificates are no longer supported.
򐂰 Gen-2 certificates were first introduced for the DS8000 family in Release 7.2 in the
7.7.20.xx LMC levels. They feature an SHA 256 digital signature and a 2048-bit key. The
Gen-2 certificate is active by default on a DS8900F. The Gen-2 certificates meet the
requirements of the NIST Special Publication 800-131a: Transitions Recommendation for
Transitioning the Use of Cryptographic Algorithms and Key Lengths. In addition, Gen-2
certificates in DS8000 Release 8.1 have a UID added to the Client Certificate
Authentication, which enables the most secure way to connect a Key Management
Interoperability Protocol (KMIP) capable key server, such as SKLM V4.0.0.2 and Gemalto
SafeNet KeySecure, to the DS8000 by SSL session and a specific user name.
򐂰 Gen-3 certificates were introduced for the DS8000 family in Release 9.0. They feature a
SHA512 digital signature and a 4096 bit key. The Gen 3 certificate is dormant by default
on the DS8900.

Note: The Gen-3 certificate requires KMIP and is not supported with IPP. Be aware of
this restriction if you had set up data at rest encryption with IPP using the Gen-2
certificate.

28 IBM Fibre Channel Endpoint Security

 Careful planning is required when selecting which certificate is used, especially when
migrating an existing encryption environment.

DS8000 Release 7.2 and later also supports Transport Layer Security (TLS) 1.2 for network
communication. All components in the storage environment must support TLS 1.2 before
implementing the Gen-2 certificates and TLS 1.2.

Tip: For Endpoint Security, SKLM V3.0.1.3 is required, however we recommend SKLM
4.0.0.2 or later.

3.2.2 z15 HMC digital certificate

The IBM z15 HMC includes a self-signed certificate. It can be replaced with a
customer-supplied digital certificate. The instructions on how to change the z15 HMC digital
certificate are given in Chapter 6, “Managing Certificates” on page 85.

3.2.3 Security Key LIfecycle Manager digital certificate

The IBM Security Key Lifecycle Manager software includes a self-signed certificate. It can be
replaced with a customer-supplied digital certificate. The instructions on how to change the
digital certificate are given in Chapter 6, “Managing Certificates” on page 85.

3.3 DS8900 ordering and configuring

The DS8900 Fibre Channel Endpoint Security solution requires the following components:
򐂰 DS8900F as either:
– DS8910F (Model 533x-993 or 533x-994)
– DS8950F (Model 533x-996)
Former DS8000 models do not support Fibre Channel Endpoint Security
򐂰 32 GFC host adapters
– Feature code 3355 (SW)
– Feature code 3455 (LW)
Any of the 32 GFC adapters support both link authentication and encryption when
connected to a FICON Express 16SA adapter in the z15.
򐂰 16 GFC host adapters
– Feature code 3353 (SW)
– Feature code 3453 (LW)
The 16 GFC adapters only support link authentication when connected to either a FICON
Express 16S+ or FICON Express 16SA adapter in the z15.

Table 3-1shows what functions, link authentication or encryption, are supported with the
different host adapters used in the DS8900 or in the z 15 system.

Table 3-1 Adapters combinations and functions supported

Adapters z15 16S+ adapter z15 FICON Express 16SA adapter

DS8900 16 GFC Authentication Authentication

Chapter 3. Endpoint Security Requirements and Planning 29

 Adapters z15 16S+ adapter z15 FICON Express 16SA adapter

DS8900 32 GFC Authentication Authentication and Encryption

The DS8900F does not require any other feature codes to enable Fibre Channel Endpoint
Security.

You must still accomplish the specific tasks to enable encryption, as described in 4.2,
“DS8900F configuration” on page 46.

3.4 IBM z15 ordering

The IBM z15 requires the following features be ordered in order to support Fibre Channel
Endpoint Security:
򐂰 Central Processor Assist for Cryptographic Function (CPACF), (Feature Code #3863)
The CPACF delivers cryptographic support for Data Encryption Standard (DES), Triple
DES (TDES), and Advanced Encryption Standard (AES) data encryption/decryption, as
well as Secure Hash Algorithm (SHA)
򐂰 Endpoint Security Feature (Feature Code #1146)
򐂰 FICON Express 16SAadapters
– Feature code 0436 (LX)
– Feature code 0437 (SX)

Note: To support link authentication and encryption, at least one z15 FICON Express
16SA adapter, in combination with a 32 GFC adapter in the DS8900 is required.

Refer to Table 3-1 for details

3.5 Security Key Lifecycle Manager ordering

DS8900F Fibre Channel Endpoint Security requires IBM Security Key Lifecycle Manager
(SKLM) software. The SKLM software must be installed on at least two servers, and up to four
servers are supported by the DS8900F.

For Fibre Channel Endpoint Security, SKLM version 3.0.1.3 is required , but we recommend
SKLM version 4.0.0.2 or later , and it must be set up in a Multi-Master configuration. This type
of configuration requires one SKLM license per server.

The following SKLM product identification numbers are available:

򐂰 5641-SKL
– IBM Security Key LIfecycle Manager License with 1 year SW S & S
򐂰 5641-SKM
– IBM Security Key Lifecycle Manager LIcense with 3 year SW S & S

30 IBM Fibre Channel Endpoint Security

3.5.1 SKLM Licensing
When ordering the IBM Security Key Lifecycle Manager, a few things must be considered:
򐂰 The quantity of IBM Security Key Lifecycle Manager servers per data center
򐂰 The amount of data to be encrypted (if planning to use data at rest encryption).
򐂰 The number of IBM z systems.

Note: Earlier SKLM was licensed based on quantity of drives to be encrypted.

The following needs to be ordered:

򐂰 For DAR (Data at rest) encryption:
– SKLM Basic Edition.
For Master-Clone (one per instance of SKLM server and Clone)
For Multi-Master (one per instance of SKLM server)
– SKLM for RAW Decimal Terabyte Storage Resource Value Units (RVUs). Quantity = X
where X is the RVU calculation. For details refer to the License usage metrics at:
https://www.ibm.com/support/knowledgecenter/SSWPVP_4.0.0/com.ibm.sklm.doc/ov
erview/cpt/cpt_coredef_license_usage_metrics.htm
򐂰 For Endpoint Security additionally:
– SKLM z-platform for KMIP client (one for each IBM Z CPC)
򐂰 For Transparent Cloud Tiering (TCT)
– Same as for DAR, based on the raw capacity of the DS8900F

SKLM server software entitlements must be purchased for production as well as

high-availability and disaster recovery environments (HA/DR). There is no special licensing
for HA/DR. Simply, if the data is encrypted and SKLM serves the encryption key(s), then the
server must be licensed.

After you license the software, it is available for download through IBM Passport Advantage®

After the SKLM software is licensed, it is available for download through IBM Passport
Advantage. For information on IBM Passport advantage, refer to the following link:
https://www.ibm.com/software/passportadvantage/?mhsrc=ibmsearch_a&mhq=IBM%20Pas
sport%20Advantage

3.5.2 IBM Storage Appliance Model AP1

The IBM Storage Appliance 2421 Model AP1 is a hardware offering that supports the ability to
run IBM Security Key Lifecycle Manager for encryption for the DS8900F, previous models of
the DS8000 family, and other storage products. These appliances can be ordered with the
DS8900F.

3.5.3 IBM Lab services

Customers can leverage offerings from the IBM Lab-based software services team to assist in
installing the IBM Security Key LIfecycle Management software.

Chapter 3. Endpoint Security Requirements and Planning 31

3.6 Requirements for IBM Fibre Channel Endpoint Security
The following sections describe the requirements for IBM Fibre Channel Endpoint Security.

3.6.1 Key expiry policies

Policy input should be gathered from the appropriate parties about the frequency at which the
security keys should be re-keyed.
򐂰 Key Expiry Times
– Device Encryption key - defaults to 8 hours
– Device Authentication key- defaults to 7 days (168 hours)

3.6.2 Digital certificates

A decision needs to be made about which digital certificates to use. The DS8900 includes
both Gen2 and Gen3 certificates, which are signed by a certificate authority. SKLM and the
z15 provide self-signed certificates. All three allow the option of importing your own
certificate. See Chapter 6, “Managing Certificates” on page 85 for information on how the
digital certificates can be changed.

3.6.3 Encryption key manager

The IBM Fibre Channel Endpoint Security solution requires at least two encryption key
manager servers. At this time, only IBM Security Key LIfecycle Manager is supported and we
recommend using SKLM version 4.0.0.2 or later.

3.6.4 IBM z15

The IBM z15 server is required to support IBM Fibre Channel Endpoint Security.

Note also that the Z CPC must be in PR/SM mode, DPM is not supported.

3.6.5 FICON Directors

Take note of the following:
򐂰 FICON emulation and other features, such as the accelerator feature used for Extended
Remote Copy (XRC) are not be supported.
򐂰 ISL encryption however is supported. The switch will recognize data that has been
encrypted by Fibre Channel Endpoint Security and will not re-encrypt this data.

If FICON directors will be used for connectivity, they must be at the appropriate maintenance
level:
򐂰 Fibre Channel Endpoint Security is supported on the IBM C type (Cisco) IBM Z qualified
products SAN384C-6 (MDS 9710), SAN192C-6 (MDS 9706) and SAN50C-R (9250i)
running qualified versions of NxOS 8.4(2). Data Center Network Manager 11.3 or later
qualified version is also recommended.
򐂰 Fibre Channel Endpoint Security is supported on the IBM B type (Brocade) IBM Z
qualified Gen 5 products SAN768B-2 (8510-8), SAN384B-2 (8510-4), SAN 48B-5 (6510),

32 IBM Fibre Channel Endpoint Security

 and SAN42B-R (7840) and Gen 6 products SAN512B-6 (X6-8), SAN256B-6 (X6-4), and
SAN64B-6 (G620) running FOS levels 8.2.2 and later qualified versions. Brocade Network
Advisor 14.4.4 or later qualified version is also recommended.

To ensure that the planned products to be implemented are qualified, registered users can
visit the IBM Resource Link® library for current information about IBM Z qualified switch
products and restrictions of use at:
https://www-01.ibm.com/servers/resourcelink/lib03020.nsf/pages/switchesAndDirec
torsQualifiedForIbmSystemZRFiconRAndFcpChannels

To get access to Resource link, you first need to create an IBMiD. After an iD has been
created you can request access at this link:
https://www-01.ibm.com/servers/resourcelink/hom03010.nsf?OpenDatabase&login

3.6.6 Host Software

Although IFCES process is transparent to the operating systems running on the IBM Z CPC
(as it is implemented in firmware), OS support is required for monitoring and audit of the
solution. Functionality has been added to z/OS and z/VM to allow for host monitoring and
recording.

The following is required:

򐂰 z/OS 2.2 and up with APAR OA56924 applied
򐂰 z/OS 2.2 and up with PTFs
򐂰 z/VM 6.4 or 7.1 with PTFs

Chapter 3. Endpoint Security Requirements and Planning 33

34 IBM Fibre Channel Endpoint Security
 4

Chapter 4. Implementation
This chapter describes the implementation tasks required to configure and enable IBM Fibre
Channel Endpoint Security (IFCES) between IBM DS8900F and IBM z15 CPCs, using IBM
Security Key Lifecycle Manager (SKLM) as external encryption key managers.

Important: Two SKLM servers should be already installed, operational, and connected to
the same LAN/WAN network as the IBM z15 CPC and the DS8900F systems.

See IBM Knowledge Center for SKLM for information about SKLM software download,
installation, and implementation:
https://www.ibm.com/support/knowledgecenter/SSWPVP_4.0.0

The implementation is easiest if you perform the steps in a specific order. Start with the IBM Z
CPC configuration, followed by the DS8900F, and conclude with the verification on the SKLM
key servers.

Accordingly, we describe the required steps to enable IFCES in the following order:
1. IBM Z (z15) CPC IFCES configuration:
a. Define SKLM servers
b. Export certificate to SKLM servers
c. Configure IBM Z policies (key renewal periods)
2. DS8900F configuration (GUI or DSCLI):
a. Configure SKLM servers and endpoint security encryption group
b. Enable security on DS8900F host ports
3. SKLM key manager device group verification

© Copyright IBM Corp. 2020. All rights reserved. 35

4.1 IBM Z CPC configuration
This section provides instructions for tasks run on IBM Z Hardware Management Console
(HMC), required to enable IBM Fibre Channel Endpoint Security (IFCES).

We assume you have already installed SKLM servers, set up Multi Master HA-DR
configuration between them, and accordingly created the SSL/KMIP certificate.

The first step is to define Security Key Lifecycle Manager (SKLM) key servers and establish
connections between the IBM Z system HMC and the SKLM servers. This step is followed by
exporting the certificate from the HMC to each SKLM key server. Finally, conclude the Z
Server configuration by defining the key lifecycle policies to meet your organization’s security
requirements.

Although the minimum requirement to enable IFCES is at least one SKLM key server, it is
strongly advised to have at least two SKLM key servers configured and defined to the IBM Z
HMC for high availability and redundancy.

The IBM Z CPC configuration steps are completed using the Z HMC GUI interface.

4.1.1 Define External Key Servers and export certificate to SKLM

Important: Before you start, make sure that your Z HMC user has ACSADMIN authority to
be able to access the required menu for adding SKLM servers.

The following steps are required to define SKLM key servers:

1. Log on to the IBM Z HMC using the user with ACSADMIN authority. Figure 4-1 shows the
IBM Z HMC Welcome window.

Figure 4-1 The HMC welcome window

36 IBM Fibre Channel Endpoint Security

2. Select your IBM Z system under the System Management section. A new window
provides more information for the selected host, along with Task section, including
Configuration tasks, as shown in Figure 4-2.

Figure 4-2 Systems Management Welcome window

3. Expand the Configuration task, and select Manage Key Manager Connections
(Figure 4-3).

Figure 4-3 HMC Configuration - Manage Key Manager Connections

Note: If the Manage Key Manager Connections selection does not appear, either the user
does not have the required permissions or the system prerequisites as described in
Chapter 3, “Endpoint Security Requirements and Planning” on page 27 have not been met.

Chapter 4. Implementation 37
 4. A new window shows the high-level diagram with some connections already established
between IBM Z CPCs defined to this HMC and encryption key managers (as shown in
Figure 4-4). The exclamation point (!) on the M304 system is due to a single SKLM key
server connection (there is no redundancy). In our example, the Joshua system is not yet
connected to any encryption key server.

Figure 4-4 Display of connect systems and key managers

5. To create new connections from your IBM Z host to SKLM key managers, click Connect
systems to key managers under the Actions section (Figure 4-5).

Figure 4-5 Connect systems to key managers

6. As shown in Figure 4-6 on page 39, there are four IBM Z CPCs listed in our lab
environment. More than one host can be selected in this window, even if the hosts are
already connected to some key managers. If multiple hosts are selected, all these hosts
will be configured to the same key managers.
After you make a selection, click Next to continue.

38 IBM Fibre Channel Endpoint Security

 Figure 4-6 Select z System

7. In the new Choose Key Managers window you have an option to select from the already
configured key servers listed in the table in Figure 4-7, or alternatively, press Add Key
Manager to add a new one. In our example, we selected two encryption key servers that
are already used for other hosts.
When you select key servers, you can verify if the connections to key servers are available
by clicking TEST CONNECTIVITY. Click Next only if the connectivity test results are
successful and go to step 9.
In case you do not have any key servers already defined, click the Add Key Manager to
invoke the new wizard and add a new server (step 8).

Figure 4-7 Choose Key Managers window

Chapter 4. Implementation 39
 Important: IBM Fibre Channel Endpoint Security requires that both the storage system
and IBM Z CPC must be configured to communicate with the same set of key servers,
all belonging to the same multi-master cluster. Only one set of key servers is
configurable on the host and on the DS8900F storage system.

If you configure two IBM Z CPCs (each with their own separate set of key servers) to
run on any security enabled ports on the same storage system, one of the hosts will be
prevented from authenticating. This configuration may cause paths to be logged out if
security is enforced on the storage ports. When this situation occurs, the storage
system will continue to issue serviceable events, and concurrent code load (CCL) will
fail when there are open serviceable events. If you notice any of the above symptoms,
ensure that every host that is configured to use Fibre Channel Endpoint Security with
the storage system is configured with the same set of key servers as the storage
system or ensure that security is disabled on ports that have paths from hosts that do
not share key servers with the storage system.

If you want to change the set of key servers after Fibre Channel Endpoint Security is
enabled, you cannot delete the old set of key servers and add new ones. Use one of
the following procedures:
򐂰 Completely disable Fibre Channel Endpoint Security on the storage system and
then reconfigure Fibre Channel Endpoint Security with the new key servers.
or
򐂰 Prior to changing the key server configuration, use a key server migration procedure
by backing up the keys from the old set of key servers and then restoring the keys to
the new set of key servers

8. In the Add key manager window (Figure 4-8), provide the following information:
– Name is a required field and it must be a unique name per Z HMC/SE.
– Description is optional but it is recommended to identify the SKLM server. Include the
location of the server, which Z System and associated LPARs it is used for, and so on.
– Hostname or IP address is required.
– Port number is required. The default port is 5696.

Figure 4-8 Add key manager window

40 IBM Fibre Channel Endpoint Security

In case you would like to verify your network settings, the hyperlink to the HMC Network
Settings is available under the Guidance section of the window.
a. Click Connect to continue.
b. The progress bar window opens and, if the connection is successful, the Trust and
import key manager certificate window appears. This is the certificate from the
SKLM server to which we connected this HMC. In our example in Figure 4-9 on
page 41, this is a self-signed certificate from our SKLM server. Click See additional
certificate details to get more information about it, such as version and certificate
expiration date.
When it is reviewed, click Trust and import certificate to continue.

Figure 4-9 Trust and import key manager certificate window

c. After the SKLM certificate is imported to the Z HMC, you must export the Z HMC
certificate to the SKLM server. As shown in Figure 4-10, there are four options to
export the certificate. Considering that the communication between Z HMC and SKLM
servers is working (from previous steps), the fastest way is to directly export the
certificate to SKLM as in our example. In this case, you need the SKLM server
credentials to complete the export.
Other options can be used in case the communication between the Z HMC and SKLM
servers is not ready yet. You can export it to a USB key, email, or even an FTP server.
Click Export to continue.

Chapter 4. Implementation 41
 Figure 4-10 Export Certificates to Key Managers

d. Because in our example we chose the option to export the certificate directly to the
SKLM server, we are prompted to provide SKLM server credentials, as shown in
Figure 4-11. The default port is 443. Click Connect and export.

Figure 4-11 Providing SKLM server credentials

e. When the certificate is successfully exported to the SKLM server, you receive a
confirmation message that the certificate export to your SKLM server has completed
successfully.

42 IBM Fibre Channel Endpoint Security

9. The final message shown in Figure 4-12 displays.

Figure 4-12 Key managers configured

The high-level diagram in the Manage Key Manager window (Figure 4-13 on page 43) is
updated, now showing connections we defined from host Joshua to two added SKLM servers.
By selecting this system, the new pop-up window displays with the connection status and a
hyperlink to view certificate details. The line color coding also indicates the status:
򐂰 Blue: normal
򐂰 Orange: warning
򐂰 Red: critical

Individual connection status can be obtained by clicking a connection. Key manager

information can be obtained by clicking a Key Manager.

Figure 4-13 Manage Key Manager diagram

Chapter 4. Implementation 43
 In case only one SKLM server is defined at the initial setup, or if the connection to SKLM
servers fails, the warning messages display, as shown in Figure 4-14.

Figure 4-14 Warning messages due to communication failure and not enough key managers

4.1.2 Configure IBM Z CPC policies

After the SKLM servers are defined and the connection is established between the IBM Z
CPC and the SKLM key servers, you have to review systems policies for authentication key
lifecycle, in addition to how often you want to refresh the device and authentication encryption
keys on the IBM Z CPC and DS8900F.

The following steps are required:

1. Log on to IBM Z HMC, select your host and, from the Configuration task menu, select
Manage Key Connections (as shown in Figure 4-3 on page 37).
2. Click Configure system policies (Figure 4-15) to continue.

Figure 4-15 Configure system policies

44 IBM Fibre Channel Endpoint Security

3. Choose your system and click Next, as shown in Figure 4-16.

Figure 4-16 Select z System

4. The default authentication key expiration is set to seven days (168 hours), and the default
device encryption key is set to 8 hours. Change these values according to your security
requirements and click Save, as shown in Figure 4-17.

Figure 4-17 Choose security policy

5. The confirmation window opens. Click Continue to save the new policy (Figure 4-18).

Figure 4-18 Save new policy

With this, you completed the IBM Fibre Channel Endpoint Security configuration on Z HMC.
You can proceed with DS8900F configuration described in the next section.

Chapter 4. Implementation 45
4.2 DS8900F configuration
In this section, we explain how to complete the following tasks:
1. Export the SKLM key server certificate
2. Define SKLM key servers to the DS8900F
3. Import the SKLM certificate into the DS8900F
4. Create the encryption group for Endpoint Security
5. Enable Endpoint Security on eligible DS8900F host ports

We describe each step in detail, both using the DS8000 GUI and the DSCLI.

4.2.1 Export SKLM server SSL/KMIP certificate

We assume you have already installed SKLM servers, set up Multi Master HA-DR
configuration between them, and accordingly created the SSL/KMIP certificate.

To establish trusted communication between DS8900F and each SKLM key server, you
must export the SSL/KMIP certificate from each SKLM key server and import it in the
DS8900F, as described in “Enable Endpoint Security on the DS8900F” on page 49, DS8000
GUI, step 7.

You can export the SKLM certificate from the SKLM GUI, or alternatively by using an
OpenSSL command line application used for various cryptography tasks, such as managing
certificates. OpenSSL is supported on any UNIX OS, Mac OS, or Windows system. We
describe the exporting SKLM certificate process for both methods.

Export the SKLM SSL/KMIP certificate using the SKLM GUI

Complete the following steps to export the SSL/KMIP Server certificate:
1. Log in to any SKLM server as SKLMAdmin and navigate to Advanced Configuration and
select Server Certificates (Figure 4-19).

Figure 4-19 SKLM Advanced Configuration - Server Certificates

2. In case you have many certificates listed, click the arrow next to No filter applied to open
a filter window. Using specific search criteria, you can find your certificate. Highlight the
certificate, then click Export, as shown in Figure 4-20.

46 IBM Fibre Channel Endpoint Security

 Figure 4-20 Select SSL/KMIP certificate

3. Provide a unique filename for the certificate and browse for the directory to export it. The
default directory for Linux is /opt/IBM/WebSphere/AppServer/products/sklm/data.
Click Export Certificate as shown in Figure 4-21.

Figure 4-21 Export SSL/KMIP certificate

4. A confirmation window displays, with certificate file name and location (Figure 4-22). The
SSL/KMIP certificate is now exported.

Figure 4-22 Exported SSL/KMIP certificate information

Chapter 4. Implementation 47
 5. You must provide the certificate later while adding the SKLM servers to the DS8900F. Log
on to the SKLM server operating system with a user ID that allows access to the directory
that the certificate was exported to, and transfer it to a location that is accessible when
activating endpoint security encryption in the DS8900F.

Repeat the same process to get the certificate from the other SKLM key server (although,
being in Multi Master HA DR mode, both SKLM servers have identical certificates).

Export the SKLM SSL/KMIP certificate using OpenSSL

An alternative method of obtaining the SSL/KMIP certificate is to use OpenSSL. With this
method, there is no need to log into your SKLM servers. You just need to provide the TCP/IP
address or qualified DNS name of your SKLM servers.

Make sure that you have OpenSSL installed on the system that you are using for this
operation. Some operating systems, such as Mac OS or Linux, have it pre-installed. For
others, such as Windows, you might have to install it.

Use the following steps to get this certificate from each SKLM server:
1. From your OS command-line interface, type the command as shown in Example 4-1.
Make sure to change the port number if you use one different than the default port.

Example 4-1 OpenSSL command to obtain SSL/KMIP certificate from SKLM server
openssl s_client -connect 9.9.10.107:5696

2. The partial output of the command is shown in Example 4-2.

Copy the bolded text between the BEGIN CERTIFICATE and END CERTIFICATE statements,
including those two lines, and paste it in a <SKLM servername>.pem file on your workstation.

Example 4-2 SSL/KMIP certificate included in the OpenSSL command output

CONNECTED(00000003)
depth=0 C = US, O = IBM, OU = STG, CN = vinzclortho.tuc.stglabs.ibm.com
verify error:num=18:self signed certificate
verify return:1
depth=0 C = US, O = IBM, OU = STG, CN = vinzclortho.tuc.stglabs.ibm.com
verify return:1
---
Certificate chain
0 s:/C=US/O=IBM/OU=STG/CN=vinzclortho.tuc.stglabs.ibm.com
i:/C=US/O=IBM/OU=STG/CN=vinzclortho.tuc.stglabs.ibm.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDRzCCAi+gAwIBAgIGA2nnG47OMA0GCSqGSIb3DQEBCwUAMFMxCzAJBgNVBAYT
AlVTMQwwCgYDVQQKEwNJQk0xDDAKBgNVBAsTA1NURzEoMCYGA1UEAxMfdmluemNs
b3J0aG8udHVjLnN0Z2xhYnMuaWJtLmNvbTAeFw0xOTAxMDkxNjUzNTBaFw0yOTAx
MDYxNjUzNTBaMFMxCzAJBgNVBAYTAlVTMQwwCgYDVQQKEwNJQk0xDDAKBgNVBAsT
A1NURzEoMCYGA1UEAxMfdmluemNsb3J0aG8udHVjLnN0Z2xhYnMuaWJtLmNvbTCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMTL+6jOxEN20oQsBQG2Q4N4
4zUYQKbASIleHw5E20Vui2bZFDK/tLJlcxtE0K4I1AvcFaQ37L1eZ/e4+UO2dqaM
CKlFAi+H1YqC7Jt5XDxLQnnNFUycmcDbXgKZChWYO7J/p5QLh41A8AsjyTQ+djnp
HoD6W4ZYOqMXPm8E/bEufaopzKb9xra7lZi8Cz+ZBb/JEm9I+sGWfBK3FIJk1+0p
vzEKg6s4nw1hTH55xHuaDqL/y/Lz+BGfa4AbGFvugT2AlId0eO26hsIP0IRJiPHu
nH+gHme0y4EBHvEweB68iggU/0YUeo/026tCousWRSl0Jjci+TSo0e/ShxVXzNkC
AwEAAaMhMB8wHQYDVR0OBBYEFJ24U1W8vgoDBLtdmxaFMhL7QdVmMA0GCSqGSIb3

48 IBM Fibre Channel Endpoint Security

 DQEBCwUAA4IBAQBDnVAqrDj3wKx4qg7TDsoGNqMADbjRblwO3C/UOP7sbDkxYN1i
kmGM69/CIZQViv0g4IyK2fLSf1YovX8eNNkqD8RbPm27oSxwodcHQi/a0WzlJ6A6
HljwShKxExH8BabEvr0kWbHvR1jaivxHx7Lndd066/PzkpX2Tx81pE++tRJLiFJZ
3EBhQGoVcmTpDbfTiyZ/8dnTgpAKnJod2Nas506t/Glk4UKklMcIXQ09zUbfOTlD
NR0MXKV1KuOQiclh7ZtxcZ3cHPj6K9GRbKcz3rm9iB6f7fEOo+eBZEvwNQ5ot+Ff
jLhq4kt3skYNhTcx3c93/IFTUnZoURyW3WxA
-----END CERTIFICATE-----
subject=/C=US/O=IBM/OU=STG/CN=vinzclortho.tuc.stglabs.ibm.com
issuer=/C=US/O=IBM/OU=STG/CN=vinzclortho.tuc.stglabs.ibm.com
...
...
...
closed

If you have the standard Linux or UNIX tools available on your work station, you can run a
string of connected commands to extract the certificate directly, as shown in Example 4-3.

Example 4-3 Chain of commands to export the SKLM server certificate

openssl s_client -connect 9.9.10.107:5696 2>/dev/null </dev/null | \
sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'

3. Repeat the same process to get the certificate from the other SKLM server.

Now that you have exported the SKLM SSL/KMIP certificate to your workstation, you can
continue with the next steps to enable Endpoint Security by using either the DS8000 GUI or
the DS CLI.

4.2.2 Enable Endpoint Security on the DS8900F

When you have SKLM certificates ready, you can continue with enabling Endpoint Security on
the DS8900F. In this section, we describe the steps required to configure and enable
Endpoint Security using DS8900F GUI or DSCLI commands:
1. Define the SKLM servers
2. Create the encryption key group for Endpoint Security
3. Verification

Chapter 4. Implementation 49
 DS8900F GUI
The following steps are required to enable Endpoint Security configuration:
1. Log into the DS8900F GUI. From the menu on the left, select Settings → Security, as
shown in Figure 4-23.

Figure 4-23 DS8000 GUI: Settings > Security

2. The new window appears as in Figure 4-24. Select the Endpoint Security tab and click
the Configure Endpoint Security button to start the configuration wizard.

Figure 4-24 Configure Endpoint Security

50 IBM Fibre Channel Endpoint Security

3. The Welcome window displays basic information. You can expand the Prerequisites
sections for more detail. Click Next to continue (Figure 4-25).

Figure 4-25 Configure Endpoint Security - Welcome window

4. In this step, you need to provide TCP/IP address or qualified DNS name, along with the
communication port for each SKLM server. In the example in Figure 4-26, we use the
default port. After you provided this information for one SKLM server, click the Plus sign
(+) on the right (next to the port number) to add the second SKLM server. Click Next to
continue.

Note: As previously mentioned, for Fibre Channel Endpoint Security, the storage
system and IBM Z CPC must be configured to communicate with the same set of key
servers. Only one set of key servers, all from the same multi-master cluster is
configurable on the host and on the DS8900F storage system.

Figure 4-26 Adding SKLM servers

Chapter 4. Implementation 51
 5. The DS8900F tests the connectivity to each SKLM server, and you can follow the progress
of this activity. In case the connection is successful, you get a notification as shown in
Figure 4-27.

Figure 4-27 Testing connections with SKLM servers

6. In Figure 4-28 is the final confirmation that the connection to each SKLM server was
successfully tested.

Figure 4-28 SKLM connection confirmation

7. Now you have to import the SSL/KMIP certificate from each SKLM server that you
exported before according to “Export SKLM server SSL/KMIP certificate” on page 46. In
the panel shown in Figure 4-29 on page 53, click the folder icon in the empty box next to
each SKLM server. Provide the previously exported certificate files in the upcoming file
selection dialog. You cannot complete the configuration without providing these
certificates.
Click Next to proceed.

52 IBM Fibre Channel Endpoint Security

 Figure 4-29 Specify SSL/KMIP certificate for each SKLM server

8. Complete the Endpoint Security configuration by clicking the Finish button, as shown in
Figure 4-30. This creates one encryption group for Endpoint Security, including previously
added SKLM servers.

Figure 4-30 Complete the Endpoint Security configuration

Chapter 4. Implementation 53
 9. When the previous step completes, a new window displays with Endpoint Security
information. As shown in Figure 4-31, the state is Enabled. By expanding the Key
Servers section, you can see that two SKLM servers are Online. From the Encryption
Communication Certificate section, you can view the certificates on the DS8900F and
on each SKLM server.

Figure 4-31 Endpoint Security enabled

Note: There is an Update Certificate button, which is only active if data-at-rest

encryption is also enabled. If it is active, you can use it to replace an expired certificate,
upgrade the DS8900F certificate from Gen 2 to Gen 3, or implement your own
certificate. The process for upgrading the DS8000 certificate is described in Chapter 6,
“Managing Certificates” on page 85.

There is also an option to Export Certificate for your convenience and if required.

With this, you completed the Endpoint Security enablement using the DS8900F GUI. The
next step required to enable end-to-end security between the IBM Z CPC and DS8900F is to
enable security on eligible DS8900F I/O ports. Proceed to “Enable Endpoint Security on
DS8900F host ports” on page 56 to complete Endpoint Security enablement.

DS8900F DSCLI

Important: Some of the DSCLI commands required to enable Endpoint Security are not
supported on the embedded DSCLI. Use the standalone DS CLI to complete the following
steps. Remember to always use the corresponding DSCLI version included in your
DS8000 code bundle.

54 IBM Fibre Channel Endpoint Security

The following steps are required to enable Fibre Channel Endpoint Security using the
DS8000 DSCLI:
1. Before you start adding SKLM servers and creating an encryption group, check if there are
already defined key servers (such as for Data at Rest, or Transparent Cloud Tiering
(TCT)), as well as if there is any other encryption group already defined. Use the lskeymgr
and lskeygrp commands as shown in Example 4-4. In our example, there are not any
other key servers or encryption groups present.

Example 4-4 Check key managers and encryption key group status
dscli> lskeymgr -l
CMUC00234I lskeymgr: No Key Manager found.
dscli> lskeygrp -l
No Key Group found.

2. Add the SKLM servers using the mkkeymgr command, as shown in Example 4-5. We add
two SKLM servers.
In this example, we provide the SKLM servers’ fully qualified DNS name for the addr
parameter. We use the cert parameter to identify the location of the SKLM server’s
SSL/KMIP certificates. The keygrp parameter specifies the encryption group that the new
key servers will be associated with. In the last parameter, you specify the key server ID.

Important: Encryption group ID 1 cannot be used for Endpoint Security. It is reserved

for data-at-rest encryption. It is a good practice to always use the next available ID. For
example, if you have no encryption yet, or data-at-rest encryption only, use ID 2. If you
have TCT encryption already using ID 2, then use ID 3 for Endpoint Security.

Example 4-5 Add SKLM key managers

dscli> mkkeymgr -addr vinzclortho.tuc.stglabs.ibm.com -port 5696 -keyprotocol
kmip -cert /home/hscroot/certs/vinz.pem -type endpoint -keygrp 2 1
CMUC00354I mkkeymgr: The key server 1 has been created.
dscli> mkkeymgr -addr zuul.tuc.stglabs.ibm.com -port 5696 -keyprotocol kmip
-cert /home/hscroot/certs/vinz.pem -type endpoint -keygrp 2 2
CMUC00354I mkkeymgr: The key server 2 has been created.

3. Create the encryption key group for Endpoint Security using the command shown in
Example 4-6. The key group ID must match the keygrp parameter value used in the
mkkeymgr command in step 2.

Example 4-6 Create Endpoint Security encryption group

dscli> mkkeygrp -keyprotocol kmip -type endpoint 2
CMUC00358I mkkeygrp: The key server key group 2 has been created.

4. For verification, use the lskeymgr and lskeygrp commands, as shown in Example 4-7 on
page 56.
The SKLM key manager state should be active and status normal. The encryption key
group state should be accessible and mgrstatus normal. Normal mgrstatus means path
access status for all key servers that are associated with the specified key group are
operational.
At this stage (soon after you created an encryption key group), the lskeygrp command
output does not display any value for the grpstatus. The grpstatus indicates if the key
group has access to the Endpoint Security authentication key on each SKLM server.

Chapter 4. Implementation 55
 This parameter is refreshed by either a DS8900F background process that runs every 8
hours or a DSCLI command (see Example 4-7).

Example 4-7 Endpoint Security configuration verification

dscli> lskeymgr -l
ID state status keyprotocol addr port type keygrp
===============================================================================
1 active normal KMIP vinzclortho.tuc.stglabs.ibm.com 5696 ENDPOINT 2
2 active normal KMIP zuul.tuc.stglabs.ibm.com 5696 ENDPOINT 2
dscli> lskeygrp -l
ID state reckeystate reckeydate datakeydate grpstatus mgrstatus
=======================================================================
2 accessible disabled - 08/30/2019 normal normal

label label2 keyprotocol type name

============================================
- - KMIP ENDPOINT ENDPOINT_2

5. In order to test that the encryption key group can access the Endpoint Security
authentication key on each SKLM server, use the managekeygrp command with the
testaccess option, as shown in Example 4-8.
You need to confirm that you want to initiate a key retrieval task. This task can be run at
any time. Assuming that the SKLM servers are operational and accessible, it takes a few
seconds for this task to complete in the background.
You can issue the lskeygrp command soon after, and the grpstatus value is updated
accordingly. A grpstatus of normal indicates that the authentication key can be retrieved
from each SKLM server.

Example 4-8 Test key group authentication key access

dscli> managekeygrp -action testaccess 2
CMUC00480W managekeygrp: Are you sure that you want to initiate a key retrieval
for key group 2? [Y/N]: Y
CMUC00481I managekeygrp: The Test Access action is submitted for key server key
group 2.
dscli> lskeygrp -l
ID state reckeystate reckeydate datakeydate grpstatus mgrstatus
=======================================================================
2 accessible disabled - 08/30/2019 normal normal

label label2 keyprotocol type name

============================================
- - KMIP ENDPOINT ENDPOINT_2

With this, you completed the Endpoint Security enablement using the DS8900F DSCLI. The
next step required to enable end-to-end security between the IBM Z CPC and DS8900 is to
enable security on eligible DS8900 I/O ports. Proceed to the next section “Enable Endpoint
Security on DS8900F host ports” to complete Endpoint Security enablement.

4.2.3 Enable Endpoint Security on DS8900F host ports

The final step to configure Endpoint Security between the IBM Z System and DS8900F is to
enable security on eligible DS8900F host ports. We describe the process for both the
DS8900F GUI and DSCLI.

56 IBM Fibre Channel Endpoint Security

 Note: In order to prevent an unauthorized user from changing these port features, control
for these settings is limited to the Administrator user role.

Using the DS8900F DSGUI

The following steps are required to enable Endpoint Security on DS8900F eligible host ports:
1. Log into the GUI and from the menu on the left, select Settings, then the Network option,
as shown in Figure 4-32.

Figure 4-32 Settings > Network

2. The Fibre Channel Ports table displays all available ports on the DS8900F system. If you
have many ports, you can click Filter to identify specific ports for which you would like to
enable Endpoint Security (Figure 4-33).

Figure 4-33 Filter IO ports

3. To display a range of ports, click the Advanced Filter icon. The Advanced Filter provides
a variety of search criteria options, such as ID, protocol, and so on, providing a faster way
to identify required I/O ports. In Figure 4-34 on page 58 we searched based on ID, starting
I031.
Click Apply to continue.

Chapter 4. Implementation 57
 Figure 4-34 Advanced Filter

4. Now, the Fibre Channel Ports table displays only I/O host ports based on your search
criteria. You can select one port, either with a right mouse click or from the Action
drop-down menu, select Modify Endpoint Security, as shown in Figure 4-35.

Figure 4-35 Select port and choose Modify Endpoint Security option

Note that you cannot select multiple ports when trying to modify the Endpoint Security. If
multiple ports are selected, you get a warning message, as shown in Figure 4-36.

Figure 4-36 Multiple ports selected warning message

58 IBM Fibre Channel Endpoint Security

5. The Modify Fibre Channel Port Endpoint Security panel displays, as shown in
Figure 4-37 on page 60. Apart from the option to disable Endpoint Security (default), you
can select one of the following options:
– Enabled
The port will allow security protocols to negotiate the use of link authentication and
encryption if all prerequisites are met. However, if you choose this option, and some of
the prerequisites are missing (ports not supporting Endpoint Security), it will not
enforce link authentication and encryption, and will allow I/O traffic to flow regardless of
whether or not security was successfully negotiated.
– Enforced
This policy is strict and enforces that the port performs a secure login and successfully
performs link authentication (and negotiated use of encryption, where supporting
hardware is configured) before allowing I/O traffic to flow. Host ports that do not
support security or do not complete authentication will be logged out or denied login.

Important: Do not use the Enforced policy for any replication ports (Metro Mirror or
Global Mirror). They do not support Endpoint Security and the PPRC path creation will
fail. This includes shared ports that are used for both, PPRC and host traffic.

It is good practice to use the Endpoint Security policy of Enabled with the initial security
enablement. This way, you can verify that everything is configured as per requirements.
After you verify that all required host channels are fully secured and operational, you
can change the Endpoint Security policy from Enabled to Enforced, if this is your
security standard requirement.

In addition to the previously mentioned options, you can see the current port statistics,
showing the number of ports logged in and their current security settings:
– Current logins is the total number of host ports logged in to the DS8900F port.
– Security capable logins is the number of host ports that have any kind of IFCES
capability (either authentication only or authentication and encryption).
– Authenticated logins is the number of host ports that are logged in and authenticated
only. Ports that are also encrypted are not counted here.
– Encrypted logins is the number of host ports that are logged in, authenticated, and
encrypted.

Chapter 4. Implementation 59
 Figure 4-37 Modify Fibre Channel Port Endpoint Security

In our example, we selected the Enabled option. Click Modify to continue.

If you select to change the security level of a port to Enforced, and there are host ports
accessing this port that are not IFCES capable, a warning panel displays with the following
message:
There are host paths on Port that are not secured by either authentication or
encryption. Setting security to Enforced on this port will cause those host
paths to go offline which may cause a loss of access to data.
Do you want to continue?
You have the option to click No to back out without affecting host access. Otherwise, click
Yes.
6. If you changed the security level in the previous step, a confirmation panel comes up, as
shown in Figure 4-38, with the following message:
A port security change causes a port to be taken offline for a few seconds.
Take all operational paths to the port offline before changing its settings.
Change only one port at a time; then, verify that the paths are online before
proceeding to another port.
Do you want to continue?

Figure 4-38 Confirm endpoint security change

Click Yes to continue.

60 IBM Fibre Channel Endpoint Security

 Any port setting change will cause the affected port to reinitialize. This results in a few
seconds of I/O interruption if paths are in use at the time of changing the setting.
Therefore, plan accordingly when enabling or changing port settings.

Important: It is recommended to stop using a port before changing its security setting
by taking or varying paths offline to that port. You must verify that all paths are
successfully online to all devices after the change. DS8900F inserts one-minute delays
after a port security setting is changed to allow time for path recovery to take place
before another port setting can be changed.

7. The task progress bar window displays activities for the port selected, as shown in
Figure 4-39.

Figure 4-39 Modify Endpoint Security task progress

At the end, you get notification that the port has been successfully updated with security
enabled, as shown in Figure 4-40. You can also see the one-minute delay from the task
startup to completion.
8. Click Close to complete this activity.

Figure 4-40 Task completion notification

Chapter 4. Implementation 61
 Note: In order to prevent an unauthorized user from changing these port features,
control for these settings are limited to the Administrator user role.

9. You are now back to the Fibre Channel Ports window, still displaying the previously
selected port.
You can verify whether the Security status is according to your settings, as shown in
Figure 4-41.

Tip: If the Security status does not reflect your change, you might have to log out and
log back in into the DSGUI, or refresh and reload the page using your browser window.

You can customize your table to display wanted columns in one window (including
Security). Alternatively, scroll the bar to the end to check the Security status.

Figure 4-41 Check the Security status of selected I/O port

10.For FCP connections, confirmation of connected authenticated and encrypted hosts can
be determined by selecting any of these ports and, either with a right mouse click or from
the Action menu, select Logged in WWPNs, as shown in Figure 4-42.

Figure 4-42 FCP ports - Logged in WWPNs

A new panel displays, showing the WWPN of logged in FCP ports (for FICON ports the
WWPN list is not shown).

62 IBM Fibre Channel Endpoint Security

 The right side of the panel shows current login counts, as shown in Figure 4-43. It shows
the total number of logins, and then distinguishes between security capable,
authenticated, and encrypted logins.

Figure 4-43 Fibre Channel Port statistics for 32 GFC port

Using the DS8000 DS CLI

Using either the standalone or imbedded DS8900F DSCLI, the following steps are required to
enable Endpoint Security on eligible host ports. We assume you are logged on to the DSCLI.
1. Use the lsioport command to identify the ports you want to configure for endpoint
security. In Example 4-9, we also show the lsioport command with the metrics
parameter to display the current security related port statistics.
The lsioport -metrics command output is large and includes extensive statistics for
each port. Therefore, we truncated the output and included only the section relevant to
Endpoint Security statistics and limited to only two ports.

Example 4-9 Display host ports with the lsioport command

dscli> lsioport -l
ID WWPN State Type topo
============================================================================
I0210 5005076306111339 Communication established Fibre Channel-LW FICON
...
I0310 5005076306191339 Communication established Fibre Channel-SW FICON

dscli> lsioport -metrics

...
==Security Login Counts per FC IO Port==
ID Logins SecCapableLogins AuthLogins EncryptedLogins
========================================================
I0210 1 0 0 0
...
I0310 2 0 0 0

The four counts provided are the current port statistics and correspond to what is shown in
the GUI in Figure 4-37 on page 60.
– Logins is the total number of host ports logged in to the DS8900F port.
– SecCapableLogins is the number of host ports that have any kind of IFCES capability
(either authentication only authentication and encryption)
– AuthLogins is the number of host ports that are logged in and authenticated only. Ports
that are also encrypted are not counted here.
– EncryptedLogins is the number of host ports that are logged in, authenticated, and
encrypted.

Chapter 4. Implementation 63
 2. After you identify the ports that you want to configure for Endpoint Security, use the
setioport command.
Note that for changing ports security settings, you cannot specify a range of ports with the
setioport command. If you do, you will get the error message shown in Example 4-10.

Example 4-10 Error using setioport - security on a range of ports

dscli> setioport -security enabled I0001-I0003
CMUC00595E setioport: The security setting cannot be changed for multiple ports
simultaneously. Specify one port at a time or use the -force parameter.

However, you can specify multiple ports, but they need to be confirmed and processed
one at a time, as shown in Example 4-11.
The DS8900F inserts one minute delays after a port security setting is changed to allow
time for path recovery to take place before another port setting can be changed.
In the example, we set two ports to enabled. The setioport command’s security
parameter also has the enforced option. See “Using the DS8900F DSGUI” on page 57 for
important notes related to the enforced security parameter usage.

Example 4-11 Enabling port security with setioport command

dscli> setioport -security enabled I0001 I0002
CMUC00590W setioport: A port security change causes a port to be taken offline
for a few seconds. Take all operational paths to the port offline before
changing its settings. Change only one port at a time; then, verify that the
paths are online before proceeding to another port. Are you sure that you want
to modify the security of I/O Port I0001? [Y/N]: y
CMUC00011I setioport: I/O Port I0001 successfully configured.
CMUC00590W setioport: A port security change causes a port to be taken offline
for a few seconds. Take all operational paths to the port offline before
changing its settings. Change only one port at a time; then, verify that the
paths are online before proceeding to another port. Are you sure that you want
to modify the security of I/O Port I0002? [Y/N]: y
CMUC00011I setioport: I/O Port I0002 successfully configured.

Tip: Use of the -quiet option suppresses the messages and confirmations (see
Example 4-12). Note that there will still be a one-minute delay between each port
setting. Therefore, and out of caution, we advise you not to use the -quiet option so
that you can verify and confirm each port individually.

Example 4-12 Using the -quiet option for setioport

dscli> setioport -quiet -security disabled I0001 I0002 I0003
CMUC00011I setioport: I/O Port I0001 successfully configured.
CMUC00011I setioport: I/O Port I0002 successfully configured.
CMUC00011I setioport: I/O Port I0003 successfully configured

3. Verify the security status and statistics with the lsioport -metrics command, as shown
in Example 4-13 on page 65.
The Security status changed to Enabled, but also the port security statistics changed (in
comparison to the statistics in step 1, Example 4-9 on page 63). Port I0210 is 16 GFC and
supports only authentication (AuthLogins = 1, and EncryptedLogins = 0). Port I0310 is
32 GFC and supports both authentication and encryption.

64 IBM Fibre Channel Endpoint Security

Example 4-13 Verification using lsioport command
dscli> lsioport -l
ID WWPN State Type topo
============================================================================
I0210 5005076306111339 Communication established Fibre Channel-LW FICON
...
I0310 5005076306191339 Communication established Fibre Channel-SW FICON

portgrp Security Speed Frame I/O Enclosure HA Card

====================================================
0 Enabled 16 Gb/s 1 1 1
...
0 Enabled 32 Gb/s 1 4 2

dscli> lsioport -metrics

...
==Security Login Counts per FC IO Port==
ID Logins SecCapableLogins AuthLogins EncryptedLogins
========================================================
I0210 1 1 1 0
...
I0310 2 2 0 2

Furthermore, the showiport command is available to provide all details required for a
specific port. In Example 4-14, we provide the endpoint security related output for a
16 GFC port (I0210) and 32 GFC port (I0310) for comparison.

Example 4-14 Display individual port information with showioport command

dscli> showioport -metrics I0210
ID I0210
Date 09/02/2019 17:19:57 MST
byteread (FICON/ESCON) 0
bytewrit (FICON/ESCON) 0
...
SecCapableLogins (FC) 1
AuthLogins (FC) 1
EncryptedLogins (FC) 0

dscli> showioport -metrics I0310

ID I0310
Date 09/02/2019 17:24:22 MST
byteread (FICON/ESCON) 4418
bytewrit (FICON/ESCON) 2167883493
...
SecCapableLogins (FC) 2
AuthLogins (FC) 0
EncryptedLogins (FC) 2

Chapter 4. Implementation 65
66 IBM Fibre Channel Endpoint Security
 5

Chapter 5. Monitoring and Maintaining the

Endpoint Security Environment
This chapter provides information about monitoring and maintaining a Fibre Channel
Endpoint Security environment. It includes the following sections:
򐂰 Monitoring and maintaining Fibre Channel Endpoint Security on the DS8900F
򐂰 Monitoring and maintaining endpoint encryption on SKLM
򐂰 Monitoring and maintaining Fibre Channel Endpoint Security on z/OS:
– z15 HMC
– z/OS commands
– z/OS messages
– SMF records

© Copyright IBM Corp. 2020. All rights reserved. 67

5.1 Managing and Monitoring Endpoint Security on the DS8900
The DS8900F has added functions to both the GUI and DSCLI for monitoring and maintaining
Fibre Channel Endpoint Security.

5.1.1 DS8900F GUI

The Endpoint Security Settings can be accessed by clicking the Settings icon → Security →
Endpoint Security. The Endpoint Security panel is shown in Figure 5-1.

Figure 5-1 Accessing DS8900 Endpoint Security functions in the DS8900 GUI

Changing the Endpoint Security State

The Endpoint Security State can be changed from Enabled to Disabled.

Note: Changing the state to Disabled deletes all key manager definitions for key managers
associated with this group and then deletes the Endpoint encryption group. Access to
shared authentication keys will no longer be available, if needed, to authenticate host links.

68 IBM Fibre Channel Endpoint Security

Adding Encryption Key Management Servers
Additional encryption key management servers can be added by selecting Add Key Server,
as shown in Figure 5-2.

Figure 5-2 DS8900F GUI - Add Key Server

The Add Key Server pop-up prompts for the host name, port number, and SSL certificate for
the new encryption key management server.

Removing an Encryption Key Management Server

To remove an encryption key management server, highlight the server that should be
removed, click Actions, and select Remove, as shown in Figure 5-3.

Figure 5-3 DS8900F GUI - Removing an encryption key management server

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment 69

 Modifying Fibre Channel Port Settings
The ioport settings can be changed in the DS8900F GUI. From the home screen, select
Actions and then Modify Fibre Channel Port Protocols, as shown in Figure 5-4.

Figure 5-4 DS8900F GUI - Modify FIbre Channel Port Settings

Figure 5-5 shows the Fibre Channel Port Settings. The Security column shows the Fibre
Channel endpoint security setting for each ioport.

Figure 5-5 DS8900F GUI - Fibre Channel Port Settings

70 IBM Fibre Channel Endpoint Security

The Fibre Channel endpoint security setting can be changed by highlighting the ioport,
selecting Actions, and then clicking Modify Endpoint Security, as shown in Figure 5-6.

Figure 5-6 DS8900F GUI - Modify Endpoint Security

The ioport endpoint security settings can then be changed between Disabled, Enabled, or
Enforced, as shown in Figure 5-7.

Figure 5-7 DS8900F GUI - Fibre Channel Port Endpoint Security Options

You can use the Current login counts to validate the expected state of host links connected to
the port. The number of authenticated logins plus the number of Encrypted logins should be
equal to the number of Security Capable logins.

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment 71

 If this is not the case, then there may be a configuration issue or key manager access issue at
the host or target causing authentication to not be successful on one or more links. Mismatch
in the number of Current logins versus Security Capable logins, when security is enabled on
this port, indicates that there are host logged into this port that do not support IFCES.

5.1.2 DS8900 command-line interface

The DS8900F command-line interface provides several commands for monitoring and
managing Fibre Channel Endpoint Security.

lskeygrp
The lskeygrp command displays information about the key group entries (Example 5-1).

Example 5-1 The .lskeygrp output

dscli> lskeygrp
ID state reckeystate reckeydate datakeydate keyprotocol type name
===================================================================================
2 accessible disabled - 08/26/2019 KMIP ENDPOINT ENDPOINT_2

lsioport
The lsioport command lists ioports that are installed in a storage image. The -security
parameter with enabled or enforced can be used to show ioports that have security set to
enabled or enforced.

Example 5-2 The lsioport output

dscli> lsioport -security enabled
ID WWPN State Type topo portgrp Security
===========================================================================================
I0210 5005076306111339 Communication established Fibre Channel-LW SCSI-FCP 0 Enabled
I0211 5005076306115339 No light detected Fibre Channel-LW SCSI-FCP 0 Enabled
I0212 5005076306119339 Communication established Fibre Channel-LW SCSI-FCP 0 Enabled
I0213 500507630611D339 Communication established Fibre Channel-LW SCSI-FCP 0 Enabled

In the example above, -security enabled is used to display all ioports where endpoint
security is enabled.

showioport
The showioport command displays properties for an ioport, including a label that indicates
the endpoint security status (Security).

Example 5-3 The showioport output

dscli> showioport i0210
ID I0210
WWPN 5005076306111339
State Communication established
loc U1500.1B3.3333333-P1-C2-T0
Type Fibre Channel-LW
Speed 16 Gb/s
topo SCSI-FCP
portgrp 0
Security Enabled
unkSCSIlog -
physloc R1-I3-C2-T0

72 IBM Fibre Channel Endpoint Security

showioport -metrics
The showioport command with the -metrics option shows performance metrics for an ioport.
This includes the number of logins (logins), the number of security-capable logins
(SecCapableLogins), and the number of encrypted logins (EncryptedLogins).

Example 5-4 showioport -metrics output

dscli> showioport -metrics i0210
ID I0210
Date 08/28/2019 10:46:30 MST
byteread (FICON/ESCON) 0
bytewrit (FICON/ESCON) 0
Reads (FICON/ESCON) 0
Writes (FICON/ESCON) 0
timeread (FICON/ESCON) 0
timewrite (FICON/ESCON) 0
CmdRetries (FICON) 0
TransferReady (FICON) 0
Logins (FC) 1
SecCapableLogins (FC) 1
AuthLogins (FC) 0
EncryptedLogins (FC) 0
bytewrit (PPRC) 0
byteread (PPRC) 0
Writes (PPRC) 0
Reads (PPRC) 0
timewrite (PPRC) 0
timeread (PPRC) 0
byteread (SCSI) 0
bytewrit (SCSI) 0
Reads (SCSI) 0
Writes (SCSI) 0
timeread (SCSI) 0
timewrite (SCSI) 0
LinkFailErr (FC) 0
LossSyncErr (FC) 0
LossSigErr (FC) 0
PrimSeqErr (FC) 0
InvTxWordErr (FC) 0
CRCErr (FC) 0
LRSent (FC) 0
LRRec (FC) 0
IllegalFrame (FC) 0
OutOrdData (FC) 0
OutOrdACK (FC) 0
DupFrame (FC) 0
InvRelOffset (FC) 0
SeqTimeout (FC) 0
BitErrRate (FC) 0
RcvBufZero (FC) 0
SndBufZero (FC) 0
RetQFullBusy (FC) 0
ExchOverrun (FC) 0
ExchCntHigh (FC) 0
ExchRemAbort (FC) 0
CurrentSpeed (FC) 16 Gb/s
%UtilizeCPU (FC) 0 Dedicated
TxPower(RDP) 0.7 dBm(1185.7 uW)
RxPower(RDP) 0.4 dBm(1085.5 uW)
TransceiverTemp(RDP) 49 C

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment 73

 SupplyVolt(RDP) 3330.6 mV
TxBiasCurrent(RDP) 44.554 mA
ConnectorType(RDP) SFP+
TxType(RDP) Laser LC 1310-LW
FECStatus(RDP) Active
UncorrectedBlks(RDP) 0
CorrectedBlks(RDP) 0

setioport
The setioport command can be used to change the endpoint security status for an ioport.

74 IBM Fibre Channel Endpoint Security

5.2 Monitoring and maintaining endpoint encryption on SKLM
This section describes how to monitor and maintain Fire Channel Endpoint Security
encryption in SKLM.

5.2.1 SKLM backup and restore

SKLM provides options to backup or restore a copy in the user interface. These options can
be found by selecting the Administration tab and selecting Backup and Restore. As shown in
Figure 5-8, there are options to Create Backup or Restore from Backup. All available
backup copies are displayed.

Figure 5-8 SKLM Backup and Restore

5.2.2 SKLM key manager device group verification

In this section, we describe SKLM device groups created for IBM Fibre Channel Endpoint
Security (IFCES) enablement.

At this stage, IFCES is already enabled and there is no further action required on the SKLM
key server. However, you can view the newly created device groups for your reference and
awareness. These device groups have associated keys and certificates. In case your
certificate (either SKLM or DS8900F) expires or you would like to upgrade the DS8900F
certificate from the default factory Gen 2 to Gen 3 or even a customer-defined certificate (as
described in Chapter 6, “Managing Certificates” on page 85), the existing associated keys
and certificates need to be updated or replaced.

During the process of adding SKLM servers and creating encryption key group for Endpoint
Security in DS8900F (as explained in 4.2.2, “Enable Endpoint Security on the DS8900F” on
page 49), the DS8900F tests the connection to each SKLM key server, identifying itself to
SKLM key servers and performs a loop test.

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment 75

 Accordingly, the SKLM key server automatically creates so-called diagnostic device group
with Peer-to-Peer device family type. The name of this device group starts with D, followed by
the DS8000 WWNN (twice). An example of the name of this device group created by the
SKLM key server is shown in Figure 5-9.

Figure 5-9 Diagnostic Device Group example in SKLM server

As soon as you enable security on DS8900F ports, the DS8900F authenticates to the SKLM
server, asking for the authentication key associated with another Peer-to-Peer device group in
the SKLM key server, which actually contains in its name the IBM Z CPC WWNN and the
DS8900F WWNN (see Figure 5-10).

Figure 5-10 z System: DS8000 Device Group example in SKLM server

To easily find these devices groups in the SKLM server, you need to determine the DS8000
WWNN. From the DS8000 GUI Dashboard, select Properties from the Action drop-down
menu, as shown in Figure 5-11.

Figure 5-11 DS8000 Dashboard -Action -Properties selection

76 IBM Fibre Channel Endpoint Security

The Properties window displays WWNN (Figure 5-12).

Figure 5-12 DS8000 Properties window with WWNN

Alternatively, you can use the DS CLI showsi command as shown in Example 5-5.

Example 5-5 DS8000 DS CLI showsi command

dscli> showsi
Name -
desc -
...
WWNN 5005076306FFD339
...

After you have obtained the DS8000 WWNN, log in to the SKLM key server, and from the top
menu, select Advanced Configuration and Device Group, as shown in Figure 5-13.

Figure 5-13 SKLM key server - Advanced Configuration - Device Group

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment 77

 Use a filter to identify the device group associated to your DS8000 and attached Z CPC, as
shown in Figure 5-14. In the Filter window, paste the DS8000 WWNN.

Figure 5-14 Find device group associated to your DS8000 and attached z System host

In our example in Figure 5-15, there are two Peer-to-Peer device groups associated with the
DS8900F. The first one is the IBM Z CPC to DS8900F device group. The second one (starting
with D) is the DS8900F diagnostic device group.

Figure 5-15 SKLM device groups for DS8000 and attached z System

You can have many IBM Z CPCs attached to the same DS8900F system. Each IBM Z CPC
has a dedicated device group, so if you have 5 Z CPCs attached to the same DS8900F, you
should see 6 device groups (5 device groups for each CPC to DS8900F pairing, plus the
diagnostic device group).

This information is only for reference, and there is no action required at this stage. You can log
off from the SKLM server.

78 IBM Fibre Channel Endpoint Security

5.3 Monitoring Endpoint Encryption on the z15
Some monitoring can be done from the Z HMC or through specific z/OS commands,
messages or, SMF records.

5.3.1 Modify Fibre Channel Endpoint Security Policy from the z15 HMC
The Fibre Channel Endpoint Security Policy can be modified with the Z HMC. First, select
Manage Key Manager Connections for the system that you wish to modify, as shown in
Figure 5-16.

Figure 5-16 z HMC -

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment 79

 Next, select Configure System Policies, as shown in Figure 5-17.

Figure 5-17 z HMC - Configure System Policies

Then select the system to modify the Fibre Channel Endpoint Security policy.

Figure 5-18 z HMC - Modify Fibre Channel Endpoint Security Policy

The Device Authentication key and Device Encryption key expiration time can then be
modified.

80 IBM Fibre Channel Endpoint Security

5.3.2 z/OS commands to display Fibre Channel Endpoint Security status
The following z/OS commands have been modified to display information relevant to Fibre
Channel Endpoint Security.

D M=DEV Command
The z/OS Display command with M=DEV has been updated to display the Fibre Channel
Endpoint Security status for each channel path. A sample is shown in Example 5-6. The
CONNECTION SECURITY output line shows the capability of the individual channel paths, as
Authentication capable (Auth) or Authentication and Encryption capable (Encr).

Example 5-6 D M=DEV Command

D M=DEV(6800)
IEE174I 00.15.50 DISPLAY M 736
DEVICE 06800 STATUS=ONLINE
CHP 70 71 72 75 73 74 76 77
ENTRY LINK ADDRESS .. .. .. .. 04 05 06 07
DEST LINK ADDRESS 0D 0D 0D 0D 08 09 0A 0B
PATH ONLINE Y Y Y Y Y Y Y Y
CHP PHYSICALLY ONLINE Y Y Y Y Y Y Y Y
PATH OPERATIONAL Y Y Y Y Y Y Y Y
MANAGED N N N N N N N N
CU NUMBER 6800 6800 6800 6800 6800 6800 6800 6800
INTERFACE ID 0210 0211 0212 0213 0310 0311 0312 0313
CONNECTION SECURITY Auth Auth Auth Auth Encr Encr Encr Encr
MAXIMUM MANAGED CHPID(S) ALLOWED: 0
DESTINATION CU LOGICAL ADDRESS = 00
SCP CU ND = 002107.996.IBM.75.0000000DMC01.0210
SCP TOKEN NED = 002107.900.IBM.75.0000000DMC01.0000
SCP DEVICE NED = 002107.900.IBM.75.0000000DMC01.0000
WWNN = 5005076306FFD339
HYPERPAV ALIASES CONFIGURED = 0
FUNCTIONS ENABLED = MIDAW, ZHPF

D M=CHP Command
The z/OS Display command with M=CHP has been updated to display the Fibre Channel
Endpoint Security status for a given channel path. Illustrations are shown in Example 5-7 and
Example 5-8 on page 82 respectively, for an Authentication capable (CSEC (Auth)) and
Authentication and Encryption capable (CSEC (Encr)) channel path.

Example 5-7 Channel path with Authentication

D M=CHP(70)
IEE174I 06.40.44 DISPLAY M 179
CHPID 70: TYPE=1A, DESC=FICON POINT TO POINT, ONLINE
DEVICE STATUS FOR CHANNEL PATH 70
0 1 2 3 4 5 6 7 8 9 A B C D E F
0680 + +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@
...
069F UL UL UL UL UL UL UL UL UL UL UL UL UL UL UL UL
SWITCH DEVICE NUMBER = NONE

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment 81

 ATTACHED ND = 002107.996.IBM.75.0000000DMC01
PHYSICAL CHANNEL ID = 019C
OPERATING SPEED = 16 Gbs, GENERATION = 22
FACILITIES SUPPORTED = ZHPF, CSEC(Auth)
************************ SYMBOL EXPLANATIONS ************************
+ ONLINE @ PATH NOT VALIDATED - OFFLINE . DOES NOT EXIST
* PHYSICALLY ONLINE $ PATH NOT OPERATIONAL
BX DEVICE IS BOXED SN SUBCHANNEL NOT AVAILABLE
DN DEVICE NOT AVAILABLE PE SUBCHANNEL IN PERMANENT ERROR
AL DEVICE IS AN ALIAS UL DEVICE IS AN UNBOUND ALIAS

Example 5-8 Channel path with Authentication and Encryption

D M=CHP(74)
IEE174I 06.37.09 DISPLAY M 177
CHPID 74: TYPE=1B, DESC=FICON SWITCHED, ONLINE
DEVICE STATUS FOR CHANNEL PATH 74
0 1 2 3 4 5 6 7 8 9 A B C D E F
0680 + +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@
0681 +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@ +@
...
069F UL UL UL UL UL UL UL UL UL UL UL UL UL UL UL UL
SWITCH DEVICE NUMBER = NONE
ATTACHED ND = SLKWRM.X68.BRD.CA.2DZH3010M01L
PHYSICAL CHANNEL ID = 0114
OPERATING SPEED = 16 Gbs, GENERATION = 27
FACILITIES SUPPORTED = ZHPF, CSEC(ENCR)
************************ SYMBOL EXPLANATIONS ************************
+ ONLINE @ PATH NOT VALIDATED - OFFLINE . DOES NOT EXIST
* PHYSICALLY ONLINE $ PATH NOT OPERATIONAL
BX DEVICE IS BOXED SN SUBCHANNEL NOT AVAILABLE
DN DEVICE NOT AVAILABLE PE SUBCHANNEL IN PERMANENT ERROR
AL DEVICE IS AN ALIAS UL DEVICE IS AN UNBOUND ALIAS

5.3.3 z/OS Messages

IOS2001I and IOS2002I

These messages have been updated with new reason codes when the status indicates
LOGICAL PATH IS REMOVED OR NOT ESTABLISHED:
򐂰 CONNECTED PORTS DO NOT SUPPORT THE REQUIRED LEVEL OF ENDPOINT SECURITY
򐂰 THE ENCRYPTION-KEY MANAGER IS NOT AVAILABLE

5.3.4 z/OS SMF Records

SMF 124-2 Record – Encryption Status

This record shows the encryption status of each channel path (CHPID to control unit port).
This record is created at the following times:
򐂰 At IPL time for every channel path associated with each FICON CHPID that is configured
online to the current Z system, even if authentication/encryption is not being used,
provided that the endpoint security feature is installed.

82 IBM Fibre Channel Endpoint Security

򐂰 When a FICON CHPID is configured online for the first time for a Z system.
򐂰 When a path becomes operational for the first time. For example, the FICON CHPID was
online but the link to the control unit was not operational (cable pulled or port blocked on
the switch); or there were no control units defined for a particular destination port on a
switch and a dynamic ACTIVATE was done to define them.
򐂰 When a path becomes operational and the endpoint security status has changed (e.g.,
none to authentication).
򐂰 When the endpoint security feature is enabled dynamically. This causes the encryption
state of any capable paths to be changed after the channel and control unit exchange
information. SMF records are created for all channel paths connected to online FICON
Cupids, even ones that don’t support endpoint security.

The following information is included in the SMF 124-2 record:

򐂰 CHPID, PCHID, and WWPN for the channel
򐂰 Destination link address, control unit interface id, and WWPN for the CU port
򐂰 Encryption capability of the channel
򐂰 Current encryption mode of the link
򐂰 Source of the record
򐂰 Initial status at IPL or when path first comes online or encryption status change

SMF 124-3 Record – Peer Node Encryption Key Update

This record is created when the encryption key changes between CEC and peer node
(storage system or other CEC for CTC connections):
򐂰 This key is used for the authentication process between the channel and its peer
򐂰 The encryption key change occurs periodically based on the policy set on the HMC

The following information is included in the SMF 124-3 record:

򐂰 WWNN of the peer node

SMF 124-4 Record - Session Encryption Key Update

This record is created when the encryption key changes between a channel and its peer
(control unit port or other channel for CTC connections):
򐂰 This key is used to program the channel or host adapter to encrypt the data in flight
򐂰 This occurs periodically based on the policy set on the HMC

The following information is included in the SMF 124-4 record:

򐂰 CHPID, PCHID and WWPN for the channel
򐂰 Destination link address, control unit interface id, and WWPN for the CU port

SMF 124-5 Record – External Key Manager Information

This record is created when the availability of an external key manager changes. The External
Key Manager is queried by the CEC and storage controller to get shared secret used during
the authentication process.

The following information is included in the SMF 124-5 record:

򐂰 External key manager identifier (16 bytes)
򐂰 Type of external key manager:

Chapter 5. Monitoring and Maintaining the Endpoint Security Environment 83

 – Unknown
– Bytes 0 - 3 of external key manager id contain an IPV4 address
– Bytes 0 - 15 of external key manager id contain an IPV6 address
򐂰 Indication of whether external key manager is available (1) or unavailable (2)

84 IBM Fibre Channel Endpoint Security

 6

Chapter 6. Managing Certificates

Digital certificates are a fundamental component of trusted and secure computer network
communication.

In this chapter, we briefly introduce the digital certificate authorization and signing. We
explain, where and how certificates are used with IBM Fibre Channel Endpoint Security
(IFCES) and how they differ from the usual node to node communication certificate. We
provide an example of how to create and check an IFCES suitable certificate and describe the
procedures to exchange or upgrade the certificates in an IFCES environment.

© Copyright IBM Corp. 2020. All rights reserved. 85

6.1 Introduction to the certificates used with IFCES
This section is a brief introduction to digital certificates, followed by special considerateness
for the DS8900F.

6.1.1 What are digital certificates used for

Digital certificates are used to establish trust between two partners in a computer network
and to set up encrypted communication. With IFCES, the two endpoints (IBM Z CPC and
DS8900F storage system) have to be able to establish trusted and encrypted communication
with the External Key Manager. To do so, they use the industry standard Transport Layer
Security (TLS) version 1.2 protocol, the most common form of digital certificate exchange.
To understand the process of establishing trust using certificates, requires also some basic
knowledge of asymmetrical encryption. Asymmetrical encryption uses two keys. One of them
can decrypt the data that the other one encrypted, and vice versa. They are referred to as a
Public - Private Key pair:
򐂰 You give your Public Key to anybody you want to send you encrypted data. The Public Key
cannot be used to decrypt the data that was encrypted with it.
򐂰 Only your Private Key can decrypt data that was encrypted with your Public Key.

This way, anybody can send you an encrypted message and you are the only one who can
read it.

On the other hand, you can encrypt a message with your Private Key. Everybody who has
your Public Key can decrypt and read it. You can use this method to prove to everybody that a
message originated from you. This is called digital signing.

We use a simplified example to explain how digital certificates work. Two nodes, we call them
Alice and Bob, want to set up trusted communication. Each of them has its own certificate (A
and B), as shown in Figure 6-1.

Figure 6-1 Computers who want to set up trusted communication

In order to trust Bob, Alice needs a copy of Bob’s certificate. But how can Alice be sure that
the copy she received really is from Bob, and that no one has either modified it or planted a
fake certificate? To remove this uncertainty, Bob and Alice use a third party which they both
trust, called Certificate Authority (CA).

86 IBM Fibre Channel Endpoint Security

As shown in Figure 6-2, Bob has his certificate digitally signed by the CA’s Private Key. After
signing, a certificate is deemed unchangeable. Alice receives the signed certificate. If she can
decrypt it with the CA’s Public Key, she can be sure it is good. If either her copy of the CA
Public Key or the received certificate were corrupted, Alice would not be able to read the
certificate and therefore would not trust the sender.

Figure 6-2 Digital certificate signing

This way, Alice and Bob can set up secure communication from Bob to Alice:
򐂰 Alice trusts Bob because she has his signed certificate.
򐂰 Bob can use Alice’s Public Key to send encrypted data, which only Alice can read, using
her Private Key.

For two-way communication, Alice and Bob initiate the same process in the opposite
direction, too.

Note: Of course the communication between the nodes (Alice and Bob) and the CA must
also be secured.

Asymmetric encryption is relatively slow and computing intensive. Therefore it is usually only
used to establish a trusted communication session between two nodes. The first thing Alice
and Bob will do now, is to create a common symmetric session key. They can share it using
their asymmetric encryption and continue their session using symmetric encryption. To avoid
that the symmetric session key is intercepted and abused, Alice and Bob have to make sure
that a session stays active only as long as it is really needed, and that they create a unique
key for each session that they set up.

Note: To establish trusted and secure communication in IFCES, both endpoints exchange
signed certificates with the External Key Manager. Whenever they have to communicate,
they use asymmetrical encryption and set up a session.

Different methods are used to exchange certificates between the IBM Z CPC and SKLM,
and IBM DS8900F and SKLM. The process also differs between initial implementation or
subsequent replacement of a certificate. See Chapter 4, “Implementation” on page 35 and
6.3, “Changing digital certificates” on page 92 for details.

Chapter 6. Managing Certificates 87

6.1.2 Special considerations for the DS8900F encryption certificate
The certificate for External Key Manager communication on the DS8900F side, also called
the DS8900F encryption certificate, requires some special considerations. In addition to
establish trusted communication with the External Key Manager, it is also used during
creation of the peer-to-peer device groups (see 2.1.1, “External key manager” on page 18).
For that it must contain the Fibre Channel WWNN of the system in a special field.

This certificate is the DS8900F own certificate, which it uses to identify itself to the External
Key Manager. By default, every DS8900F has an active encryption certificate (GEN2,
SHA256, 2048 bit key) that already contains the WWNN, and therefore is ready for use with
IFCES. This certificate is known and trusted by the External Key Manager, and you can use it
without further action.

There also is another, stronger, IFCES ready certificate available on each DS8900 (GEN3,
SHA 512, 4096 bit key). It is not in use by default. If you consider replacing the GEN2 with the
GEN3 certificate, see 6.3.3, “Changing the Digital Certificate for the DS8900F” on page 103.

If you need to replace the pre installed, self-signed certificates with one that is signed by an
external (or your organization’s) CA, you also have to follow the instructions in “Modify
customer defined certificate for use with IFCES” on page 88.

6.2 Modify customer defined certificate for use with IFCES

The certificate that is used to identify an a DS8900F storage system to the External Key
Manager is called encryption certificate. They External Key Manager must have a copy of this
certificate to be able to verify communication requests from the owning DS8900F. The same
certificate is used for all three types of encryption a DS8900F system supports:
򐂰 Data at rest (DAR) encryption
򐂰 Transparent Cloud Tiering (TCT) encryption
򐂰 IBM Fibre Channel Endpoint Security

For DAR and TCT encryption, the encryption certificate can be a standard X509 certificate.
For IFCES, the certificates must also contain the Fibre Channel WWNN of the DS8900F
system. As described in 2.1.1, “External key manager” on page 18, the External Key Manager
uses the WWNNs of the endpoint devices to associate them as a trusted endpoint device pair
in its peer to peer device groups. The method to pass the WWNN to the External Key
Manager is by including it in the encryption certificate using the Subject Alternative Name
(SAN) field. The use of the SAN field to provide the Fibre Channel WWNN is standardized.

Because a certificate is digitally signed, it cannot be changed after creation. Therefore, you
cannot add the SAN field to an already existing certificate. You have to create a new one.

In our example, we use the popular openssl command for all certificate related actions.

88 IBM Fibre Channel Endpoint Security

 Note: In the following example, we describe a method to create a Certificate Signing
Request (CSR) for a DS8900F certificate that contains the WWNN and can be used with
IFCES. Use this CSR to get a signed certificate from an outside Certificate Authority (CA)
or your organization’s own CA.

We amend the example with the creation of a self-signed certificate to be able to show how
you can check whether the SAN field is populated properly. Self-signed certificates are
considered insecure and suitable only for test purposes. We recommend not to use
self-signed certificates in production environments.

At the end, we show how you can package the new certificate and its associated private
key into a PKCS12 container, because you need it to import the new certificate into the
DS8900F.

The first step is to create a certificate signing request (CSR). In this step, you provide the
necessary information about the DS8900F. You need the Storage Image ID and the WWNN of
the storage system for which you create the certificate. You can determine them by using the
lssi DSCLI command, as shown in Example 6-1. The blue string (75ACA91) is the Storage
Image ID and the red one (5005076303FFD13E) is the WWNN.

Example 6-1 Determine the Storage Image ID and the WWNN of a DS8000
dscli> lssi
Name ID Storage Unit Model WWNN State ESSNet
========================================================================================
IBM.2107-75ACA91 IBM.2107-75ACA91 IBM.2107-75ACA90 980 5005076303FFD13E Online Enabled

Example 6-2 shows how you can create a X509 CSR that contains the required information in
the SAN field, using the openssl command.

Example 6-2 Adding a WWNN to a X509 certificate signing request

openssl req -new -sha256 -x509 -nodes -out csr_file.csr -newkey rsa:2048 -keyout
key_file.key -config <(
cat << EOF
[req]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn

[ dn ]
C = US
O = ACME Corp
CN = 2107-75ACA91
UID = DS8K-2107-75ACA91

[ req_ext ]
subjectAltName = @alt_names

[ alt_names ]
otherName = 1.2.840.114402.1.1.1;UTF8:50:05:07:63:03:FF:D1:3E
EOF
)

Chapter 6. Managing Certificates 89

 Modify the command to meet your configuration needs. Change the red fields according to
your preferences or organizational requirements:
-out parameter The name of the CSR file to create
-keyout parameter The name of the key file to create
C field The country code for your organization
O field The name for your organization

Modify the blue fields to match the DS8900F system that you want to create the certificate for:
CN field Replace the rightmost seven characters with the Storage Image ID
UID field Replace the rightmost seven characters with the Storage Image ID
otherName field Replace the rightmost 23 characters with the WWNN. The WWNN
itself has a length of 16 hexadecimal characters. You must enter it with
a colon (“:”) after each character pair. Make sure you do not change
any other part of the otherName field

Note: Using a command, such as that shown in Example 6-2, you generate a CSR using a
SHA256 signature and key length of 2048 bits. In the DS8900F user interfaces, this would
be referred to as a GEN2 certificate. You can also create requests for stronger (GEN3)
certificates by specifying SHA512 and a key length of 4096 bits.

Now you can use the CSR and key files to request a signed certificate from the CA approved
in your organization. We continue with the generation of a self signed certificate, as shown in
Example 6-3. Both methods provide you with a X509 certificate.

Example 6-3 Create self signed certificate from CSR

$ openssl x509 -req -days 365 -in csr_file.csr -signkey key_file.key -out
cert_file.pem -extensions req_ext -extfile <(
cat << EOF
[ req_ext ]
subjectAltName = @alt_names

[ alt_names ]
otherName = 1.2.840.114402.1.1.1;UTF8:50:05:07:63:03:FF:D1:3E
EOF
)

As before, adapt the red fields to your organization’s requirements. You have to provide the
definitions for the SAN field again, in a slightly different manner, as highlighted in blue. This is
due to a quirk in the openssl command. It cannot carry over X509 extensions from a CSR to
the actual certificate generation.

The resulting file with the ending “.pem” contains the new X509 certificate. You can display its
contents using the command shown in Example 6-4. We show only the fields that are
significant for our special case.

Example 6-4 Display a X509 certificate

$ openssl x509 -text -noout -in cert_file.pem

Certificate:
Data:
Version: 3 (0x2)

90 IBM Fibre Channel Endpoint Security

 Serial Number: 12555499258733268688 (0xae3e180dce7a0ed0)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, O=ACME Corp, CN=2107-75ACA91/UID=DS8K-2107-75ACA91
Validity
Not Before: Sep 5 16:43:26 2019 GMT
Not After : Sep 4 16:43:26 2020 GMT
Subject: C=US, O=ACME Corp, CN=2107-75ACA91/UID=DS8K-2107-75ACA91
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
....
X509v3 extensions:
X509v3 Subject Alternative Name:
othername:<unsupported>
Signature Algorithm: sha256WithRSAEncryption
...

It is important that your new certificate has a version number of 3 and contains the X509v3
extension Subject Alternative Name, as shown in blue. The printout of the certificate doesn’t
show the WWNN of the DS8900F yet, because openssl cannot interpret the fields properly.

To verify the WWNN, you can use the asn1parse option of the openssl command. You can do
this in two steps. You first issue a asn1parse command against the whole certificate, as shown
in Example 6-5.

Example 6-5 Find the SAN field in the raw X509 content
$ openssl asn1parse -in cert_file.pem
0:d=0 hl=4 l= 878 cons: SEQUENCE
4:d=1 hl=4 l= 598 cons: SEQUENCE
...
557:d=5 hl=2 l= 3 prim: OBJECT :X509v3 Subject Alternative Name
562:d=5 hl=2 l= 42 prim: OCTET STRING [HEX
DUMP]:3028A02606092A864886FD62010101A0190C1735303A30353A30373A36333A30333A46463A44
313A3345
..-

Look through the output of the command until you find the term X509v3 Subject Alternate
Name and record the number in front of the next line in the output. Then you issue another
asn1parse command with the -strparse option against this specific line, as shown in
Example 6-6.

Example 6-6 Parse the content of the SAN data field

openssl asn1parse -in cert_file.pem -strparse 562
0:d=0 hl=2 l= 40 cons: SEQUENCE
2:d=1 hl=2 l= 38 cons: cont [ 0 ]
4:d=2 hl=2 l= 9 prim: OBJECT :1.2.840.114402.1.1.1
15:d=2 hl=2 l= 25 cons: cont [ 0 ]
17:d=3 hl=2 l= 23 prim: UTF8STRING :50:05:07:63:03:FF:D1:3E

In the output, you should find the WWNN that you defined for the CSR in the UTF8STRING field.

Chapter 6. Managing Certificates 91

 After you have the signed X509 certificate, the final step is to create a PKCS12 container that
contains the certificate itself and the private key that is associated with it. You can use an
openssl command again, as shown in Example 6-7.

Example 6-7 Create PKCS12 container from certificate and key file
$openssl pkcs12 -export -out certificate.p12 -inkey key_file.key -in cert_file.pem
Enter Export Password:
Verifying - Enter Export Password:

You created the file containing the required private key together with the signing request in the
first step. The command prompts you for a password. You can choose any password that you
want, but make sure to memorize it, because you will need it again.

The PKCS12 container is the file format you use to install a new encryption certificate in the
DS8900F. Follow the instructions provided in “Changing the Digital Certificate for the
DS8900F” on page 103 to upload the package to the DS8900F HMC and install the certificate
and private key.

6.3 Changing digital certificates

Depending on your organization’s security policies, it might be necessary to change the digital
certificates used by the External Key Manager, IBM Z HMC, and DS8900F.

There are a few examples of when you may need to change digital certificates:
򐂰 Upgrade the DS8900F certificate from Gen 2 to Gen3
򐂰 Implement customer defined certificates
򐂰 Whenever certificates are due to expire or already expired

In this section, we provide the procedure how to change digital certificates on External Key
Manager (SKLM servers), IBM z HMC, and DS8900F.

6.3.1 Changing the SSL/KMIP digital certificate for SKLM

The SKLM key servers have their own digital certificate used for authentication to the IBM Z
CPC and the DS8900F. In case you have to change this certificate, for example because it is
due to expire or already expired, you generate the new one and export it to both the IBM Z
HMC and DS8900F. Failing to do so generates error messages on the IBM Z HMC and
DS8900F, indicating that the communication to the SKLM servers cannot be established.

Important: The DS8900F uses the same SKLM SSL/KMIP digital certificate for all
DS8000 encryption types, such as date at rest (DAR), Transparent Cloud Tiering (TCT),
and IBM Fibre Channel Endpoint Security (IFCES). Therefore, changing this certificate
affects all encryption types enabled on a DS8900F system.

To change the digital certificate on the SKLM servers, perform the steps in the next sections,
starting with changing the certificates on the SKLM key servers and followed by the actions
required on the IBM Z HMC and DS8900F.

92 IBM Fibre Channel Endpoint Security

SKLM key server

Note: For IFCES, your SKLM server has to be configured in multi-master mode.
Configuration changes done on one node are automatically replicated to all others.
Therefore, you normally have to perform the following steps only with one SKLM node.

Complete the following steps:

1. Log on to the SKLM key server and from the main menu select the Configuration tab and
then SSL/KMIP.
You can create a new self-signed certificate or customer-defined certificate (Request
certificate from a third party provider). In our example, we selected the second option,
as shown in Figure 6-3.
2. When creating a new certificate, you have to provide a certificate label (required for DAR
encryption), description, and certificate validity based on your security policy.

Figure 6-3 SKLM - change digital certificate

3. When the SKLM certificate is changed, export it as described in “Export SKLM server
SSL/KMIP certificate” on page 46 and save it to your workstation. This certificate must be
imported to the IBM Z HMC and DS8900F systems, as described in the following sections.
4. Also log on to the other SKLM nodes in your multi-master configuration, and check if they
also use the new certificate.

Chapter 6. Managing Certificates 93

 IBM Z HMC
Complete the following steps:
1. Log on to the IBM Z HMC, select your host (Z system) and from the Configuration menu
select Manage Key Manager Connections (follow steps 1 - 3 described in 4.1.1, “Define
External Key Servers and export certificate to SKLM” on page 36).
2. From the Action section, click Import key manager certificate as shown in Figure 6-4.

Figure 6-4 Import key manager certificate

3. This brings up a panel as shown in Figure 6-5, which guides you through the import
process. You have to select from where you want to import the certificate and from which
SKLM server. If the connection between the IBM Z HMC and the SKLM server is in
operation, you can select the option Import from key manager. In this case, the HMC
retrieves the certificate from the key server directly. Otherwise, you have to select Import
from file system and provide the certificate file that you exported from the key server as
described in “SKLM key server” on page 93.

Figure 6-5 Select SKLM server

94 IBM Fibre Channel Endpoint Security

4. Select the SKLM server that you want to import the certificate for and click Next.
5. The launched panel shows a list of all Z systems that have key server configured to them,
as shown in Figure 6-6. Select all Z systems for which you want to import the certificate
(normally all Z systems using this specific key server). Click Import to apply the selected
SKLM key manager certificate into the trust stores of the selected Z systems.

Figure 6-6 Select Z systems

Note: Normally, all SKLM servers in a multi-master configuration use the same certificate.
Therefore, you should have to import the certificate for only one of them. Check the IBM Z
HMC Endpoint Security overview panel, as shown in Figure 4-4 on page 38, whether all
SKLM servers are connected properly. If not, repeat the steps in this section for the
affected one.

DS8900F
After importing the new SKLM server certificate to the IBM Z CPC system, you also have to
import it into the DS8900F.

Important: The same SKLM SSL/KMIP digital certificate is used for all DS8000 encryption
types, that is, data at rest (DAR), Transparent Cloud Tiering (TCT), and IBM Fibre Channel
Endpoint Security (IFCES). Therefore, you can update the SKLM certificate on the
DS8900F either from the DAR or Endpoint Security panel. In our example below we do it
from the Endpoint Security panel.

1. Logon to the DS8900F GUI and select Settings → Security → Endpoint Security.
Expand the Encryption Communications Certificate section, as shown in Figure 6-7 on
page 96.
2. The defined SKLM servers are listed in the Key Server Communications Certificates
section. Select Update Certificate for the SKLM server with the new certificate.

Chapter 6. Managing Certificates 95

 Figure 6-7 DS8900F GUI - update SKLM digital certificate

3. You are prompted for the certificate file name, as shown in Figure 6-8. Provide the
certificate file that you exported from the key server, as described in “SKLM key server” on
page 93.

Figure 6-8 DS8900F GUI - provide SKLM certificate

4. Click Update and wait for the task completion window, as shown in Figure 6-9.

Figure 6-9 DS8900F GUI - update key server certificate task window

96 IBM Fibre Channel Endpoint Security

 The final certificate update confirmation displays, as shown in Figure 6-10.

Figure 6-10 DS8900F GUI - update key server certificate completed

Note: Normally, all SKLM servers in a multi-master configuration use the same certificate.
Therefore, you should have to import the certificate for only one of them. Check the
DS8900F Endpoint Security overview panel, as shown in Figure 4-31 on page 54, to
determine whether all SKLM servers are connected properly. If not, repeat the steps in this
section for the affected one.

6.3.2 Changing the digital certificate on the IBM Z HMC

You can either modify (edit) or replace the digital certificates of your IBM Z servers on the IBM
Z HMC. Modifying only the existing certificate enables you to change the expiration date. The
other option is to request a signed certificate from a certificate authority and import it to the
HMC. The IBM Z HMC can export the new or modified certificate directly to the SKLM
servers. No extra steps are required on the SKLM servers.

For both options, you log on to the HMC and select the Z server for which you want to change
the certificate. Open the Configuration menu and select Manage Key Manager
Connections (as described in steps 1 - 3 in 4.1.1, “Define External Key Servers and export
certificate to SKLM” on page 36).

Editing the digital certificate

Complete the following steps:
1. From the Action section, click Edit certificates to invoke a new wizard (Figure 6-11).

Figure 6-11 IBM Z HMC - edit certificates

Chapter 6. Managing Certificates 97

 2. Select the Z system to edit its certificate, as shown in Figure 6-12. You can select more
than one Z system if they are connected to the same SKLM servers. Continue by clicking
Next.

Figure 6-12 IBM Z HMC - choose Z systems

3. The Edit Certificates panel as shown in Figure 6-13 only enables you to change the
certificate’s expiry date. You can find more information about this certificate by expanding
See additional certificate details. Click Next to continue.

Figure 6-13 IBM Z HMC - edit certificates - make changes

98 IBM Fibre Channel Endpoint Security

4. A warning message pops up (Figure 6-14), indicating the authentication to the defined key
managers will fail until you export the modified certificate to SKLM servers. By clicking
Save, the modified self-signed certificate is created and saved.

Figure 6-14 IBM Z HMC - save certificate

5. The next panel guides you to export the certificate to the SKLM servers, as shown in
Figure 6-15. Select Export directly to key managers if there is a LAN connection to your
SKLM servers. Click Export.

Figure 6-15 IBM Z HMC - export Certificates to Key Manager

6. Provide the SKLM server credentials and click Connect and export to complete the
export (Figure 6-16).

Figure 6-16 z HMC - credentials for SKLM server

Chapter 6. Managing Certificates 99

 When the certificate is exported to the first SKLM key server, you are prompted for the
credentials of the second SKLM server, and so on. After the certificate is exported to all of the
SKLM servers, you get an information message, as shown in Figure 6-17.

Figure 6-17 IBM Z HMC - export certificates confirmation message

Create certificate signing request and import signed

Complete the following steps:
1. In the Action section, click Create certificate signing request to invoke a new wizard
(Figure 6-18).

Figure 6-18 IBM Z HMC - create certificate signing request

100 IBM Fibre Channel Endpoint Security

2. Select the Z system to create the certificate for, as shown in Figure 6-19. You can select
more than one Z system if they are connected to the same SKLM servers. Continue by
clicking Next.

Figure 6-19 IBM Z HMC - select systems

3. The Export Certificate Signing Request panel displays, as shown in Figure 6-20. Select
the export option that you want. In our example, we choose e-mail. Click Export to
continue.

Figure 6-20 IBM Z HMC - export certificate signing request via e-mail

Chapter 6. Managing Certificates 101

 4. In the next screen (Figure 6-21), provide the valid e-mail address that you want this
certificate to be sent to. Click Send.

Figure 6-21 IBM Z HMC - provide e-mail address

The addressee receives a certificate signing request with a file name like this:
<server_name>.csr.

After you receive the signed certificate back from your CA, you have to import it both to the
IBM Z HMC and the key managers (SKLM servers). You can perform these steps in two ways:
1. Import the new certificate into the IBM Z HMC first, then use the HMC’s built in capability
to export its certificate to the SKLM servers. This results in an interruption of the
connections to the key managers until the second step is complete:
a. In the list of actions, click Import signed certificate. This brings up a panel where you
can provide the location of the new certificate file. It will also show a warning about the
interruption of the connections to the key managers. Select the correct file and
continue. From this point, communication between the HMC and the SKLM server is
impossible.
b. In the list of actions, click Export certificates to key manager. This displays the panel
as shown by Figure 4-12 in “Define External Key Servers and export certificate to
SKLM” on page 36. Follow the described procedure to provide the new certificate to
the SKLM servers and re-establish connections.
2. Manually import the new certificate into the SKLM server first, then import it into the IBM Z
HMC. This way, the communication between HMC and key managers will not be
interrupted.
a. Import the signed certificate to SKLM by following equivalent steps as described in
“Import the new DS8900F certificate to SKLM” on page 104 and “Modify the
peer-to-peer device groups” on page 105.
b. In the list of actions, click Import signed certificate. This produces a panel where you
can provide the location of the new certificate file. It also shows a warning about the
interruption of the connections to the key managers. Select the correct file and
continue. From this point, communication between the HMC and the SKLM server is
impossible.

102 IBM Fibre Channel Endpoint Security

6.3.3 Changing the Digital Certificate for the DS8900F
When you change the currently active DS8900F certificate, you must also import the new
certificate to the SKLM servers. In addition, the peer-to-peer device groups the DS8900F
belongs to must be updated with the new certificate.

The connection between the DS8900F and the SKLM server will be interrupted after you
change the DS8900F certificate, and resumed after you complete the changes in the SKLM
server.

Note: The DS8900F uses the same digital communication certificate for all DS8000
encryption types, such as date at rest (DAR), Transparent Cloud Tiering (TCT) and IBM
Fibre Channel Endpoint Security (IFCES). Therefore, changing this certificate may affect
other encryption types enabled on a DS8900F system. See IBM DS8880 Encryption for
data at rest and Transparent Cloud Tiering (DS8000 Release 8.5), REDP-4500, for details.

If data at rest encryption is not used on a DS8900F, the communication certificate cannot
be changed.

Update the DS8900F communication certificate

To change the digital certificate that is used by the DS8900F to communicate with the SKLM
key servers, complete the following steps:
1. Navigate to the Endpoint Security menu in the DS8900F GUI and select Update
Certificate as shown in Figure 6-22.

Figure 6-22 DS8900F GUI - Encryption Communicate Certificate

Chapter 6. Managing Certificates 103

 2. Three options are displayed. Choose between the System defined Gen 2 certificate
(default), the System defined Gen 3 certificate, or a Customer defined certificate, as
shown in Figure 6-23.

Figure 6-23 DS8900F GUI - Update DS8000 Encryption Certificate

Note: The built in Gen2 and Gen3 certificates already contain the DS8900F’s WWNN in
the Subject Alternate Name field and are therefore ready for use with IFCES. If you want to
import your own Customer defined certificate, you have to make sure it also contains the
DS8900F WWNN. Refer to “Special considerations for the DS8900F encryption certificate”
on page 88 for a method to create an IFCES ready DS8900F certificate.

Import the new DS8900F certificate to SKLM

After you update the DS8900F certificate, you must export and transfer it to the SKLM server
import directory.

Note: You need a method to transfer the certificate file that you exported from the
DS8900F to the file system of the server that the SKLM runs on. You also need a user ID
that has sufficient access to add a file to the SKLM import directory. On UNIX systems, this
defaults to /opt/IBM/WebSphere/AppServer/products/sklm/data.

1. Log on to the SKLM key server, and from the main menu go to Advanced Configuration
and select Client Device Certificates (Figure 6-24).

Figure 6-24 SKLM - Advanced Configuration -> Client Device Certificates

104 IBM Fibre Channel Endpoint Security

2. From the table header, click Import as shown in Figure 6-25.

Figure 6-25 SKLM - Import certificate

3. Provide a unique certificate name which you can recognize easily, for example by
including the DS8900F serial number. Click Browse and locate the certificate that you
previously copied to the SKLM import directory. Finish the import by clicking Import.

Figure 6-26 SKLM - Import certificate details

Modify the peer-to-peer device groups

Complete the following steps:
1. In the welcome tab of SKLM, filter for the DS8900F WWNN in the Key and Device
Management section. The DS8900F WWNN should appear in at least two peer-to-peer
device groups. One is the diagnostic device group with the DS8900F WWNN repeated
twice. The others are the IBM Z CPC - DS8900F association device groups whose names
consist of the WWNN of the IBM Z CPC (as owner) and the WWNN of the DS8900F (as
partner). There is one device group for each IBM Z CPC - DS8900F pairing.

Chapter 6. Managing Certificates 105

 2. Start with the z/DS8900F device groups. Highlight one, right-click, and select Manage
Keys and Devices as shown in Figure 6-27.

Figure 6-27 SKLM - Manage Keys and Devices

3. From the next screen highlight the second item, which is the Partner device type,
right-click and select Modify, as shown in Figure 6-28.

Figure 6-28 SKLM - Modify Device Group

106 IBM Fibre Channel Endpoint Security

4. Next, browse for the new DS8900F certificate that you previously imported and select
Modify, as shown in Figure 6-29.

Figure 6-29 SKLM - Modify Device Certificate

5. Repeat these steps for all device groups the DS8900F belongs to.
6. Finally, to the welcome screen, filter for the DS8900F WWNN again, and highlight the
diagnostic device group, whose name begins with the letter “D” followed by the DS8900F
WWNN repeated twice. Right-click and select Manage Keys and Devices.
7. For this group, modify both the owner and the partner must be modified. For each,
right-click and modify the certificate.

Chapter 6. Managing Certificates 107

108 IBM Fibre Channel Endpoint Security
Related publications

The publications listed in this section are considered particularly suitable for a more detailed
discussion of the topics covered in this book.

IBM Redbooks
The following IBM Redbooks publications provide additional information about the topic in this
document. Note that some publications referenced in this list might be available in softcopy
only.
򐂰 IBM DS8900F Product Guide, REDP-5554
򐂰 IBM DS8910F Model 993 Rack Mounted Storage System, REDP-5566
򐂰 IBM DS8880 and IBM Z Synergy, REDP-5186
򐂰 IBM DS8900F Architecture and Implementation, SG24-8456
򐂰 IBM z15 Technical Introduction, SG24-8850

You can search for, view, download or order these documents and other Redbooks,
Redpapers, Web Docs, draft and additional materials, at the following website:
ibm.com/redbooks

Other publications
These publications are also relevant as further information sources:
򐂰 IBM DS8900F Introduction and Planning Guide, SC27-9560
򐂰 IBM DS8000 Series Command-Line Interface User’s Guide, SC27-9562
򐂰 IBM Security Key Lifecycle Manager Installation and Configuration Guide, SC27-5335
򐂰 IBM Security Key Lifecycle Manager Quick Start Guide, GI13-2316

Online resources
These websites are also relevant as further information sources:
򐂰 IBM Support: Fix Central:
https://www.ibm.com/support/fixcentral/swg/selectFixes?parent=Enterprise%20Stor
age%20Servers&product=ibm/Storage_Disk/DS8900F
򐂰 DS8000 IBM Knowledge Center:
https://www.ibm.com/support/knowledgecenter/ST5GLJ/
򐂰 DS8900F IBM Knowledge Center:
https://www.ibm.com/support/knowledgecenter/SSHGBU_9.0.0

© Copyright IBM Corp. 2020. All rights reserved. 109

Help from IBM
IBM Support and downloads
ibm.com/support

IBM Global Services

ibm.com/services

110 IBM Fibre Channel Endpoint Security

IBM Fibre Channel Endpoint Security
(0.2”spine)
0.17”<->0.473”
90<->249 pages
Back cover

SG24-8455-00

ISBN 073845835x

Printed in U.S.A.

®
ibm.com/redbooks