18CSE354T – NETWORK

SECURITY

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 UNIT - 2

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 SLO-1 :
Overview of IP security (IPsec).

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 Overview of IP security
• To secure the network infrastructure from
unauthorized monitoring and control of network
traffic and the need to secure end user-to-end-user
traffic using authentication and encryption
mechanisms.
• To provide security, the Internet Architecture Board
(IAB) included authentication and encryption as
necessary security features in the next-generation IP,
which has been issued as IPv6.
• Designed to be usable both with the current IPv4 and
the future IPv6.

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 Applications of IPsec
• IPsec provides the capability to secure communications
across a LAN, across private and public WANs, and across
the Internet. Examples of its use include:
• Secure branch office connectivity over the Internet:
– A company can build a secure virtual private network over the
Internet or over a public WAN.
– This enables a business to rely heavily on the Internet and reduce
its need for private networks, saving costs and network
management overhead.
• Secure remote access over the Internet:
– An end user whose system is equipped with IP security protocols
can make a local call to an Internet Service Provider (ISP) and gain
secure access to a company network.
– This reduces the cost of toll charges for traveling employees and
telecommuters.
18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2
 Applications of IPsec
• Establishing extranet and intranet connectivity with
partners:
– IPsec can be used to secure communication with other
organizations, ensuring authentication and confidentiality
and providing a key exchange mechanism.
• Enhancing electronic commerce security:
– Even though some Web and electronic commerce
applications have built-in security protocols, the use of
IPsec enhances that security.
– IPsec guarantees that all traffic designated by the network
administrator is both encrypted and authenticated, adding
an additional layer of security to whatever is provided at
the application layer.

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 An IPSec VPN Scenario
• Combined
authentication
/encryption
function
called
encapsulating
security
payload (ESP)

• IPsec networking device

will typically encrypt all
traffic going into the
WAN and decrypt traffic
coming from the WAN

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2
 Benefits of Ipsec
• IPSec in a firewall/router provides strong security to
all traffic crossing the perimeter
• IPSec in a firewall/router is resistant to bypass traffic
as long as all traffic is IP
• IPSec is below transport layer, hence transparent to
applications
• IPSec can be transparent to end users
• IPSec can provide security for individual users
• IPSec secures routing architecture

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 IPSec documents
IPsec encompasses three functional areas:
i. authentication
ii. confidentiality
iii. key management
– dozens of RFCs and draft IETF documents, making this the most complex
and difficult to understanding of all IETF specifications
The documents can be categorized into the following groups
– Architecture: (general concepts, security requirements, definitions, and
mechanisms defining IPsec technology - The current specification is RFC
4301)
– Authentication Header (AH): (AH is an extension header to provide
message authentication.-RFC 4302)
– Encapsulating Security Payload (ESP): (combined encryption/
authentication. RFC 4303)
– Internet Key Exchange (IKE) : key management schemes for use with IPsec.
RFC 7296
– Cryptographic algorithms: cryptographic algorithms for encryption
– Other: security policy and management information base (MIB) content.

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 IPSec Services

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 IP security architecture

DOI - Domain of Interpretation

18CSE354T – NETWORK SECURITY S - 1 / UNIT - 2

 Seven Groups

• Architecture
• Encapsulating Security Payload (ESP)
• Authentication Header (AH)
• Encryption algorithms
• Authentication algorithms
• Key management
• Domain of Interpretation (DOI)
 IP Sec Policy
• Fundamental to the operation of IPsec is the
concept of a security policy applied to each IP
packet that transits from a source to a
destination.
• IPsec policy is determined primarily by the
interaction of two databases,
1. Security Association Database (SAD)
2. Security Policy Database (SPD)
 overview of SA & SP databases

• Provides Security services – one way connection ( S-->R)

• a peer relationship - two security association needed
 Security association Parameters
A security association is uniquely identified by three parameters.
■ Security Parameters Index (SPI):
• A 32-bit unsigned integer assigned to this SA and having local
significance only.
• The SPI is carried in AH and ESP headers to enable the receiving
system to select the SA under which a received packet will be
processed.
■ IP Destination Address:
• This is the address of the destination endpoint of the SA, which
may be an end-user system or a network system such as a
firewall or router.
■ Security Protocol Identifier:
• This field from the outer IP header indicates whether the
association is an AH or ESP security association.
C S
SAc SAs
• Hence, in any IP packet, the security
association is uniquely identified by the
Destination Address in the IPv4 or IPv6 header
and the SPI in the enclosed extension header
(AH or ESP).
 IP Security Policy

IPSec policy is determined by the interaction of

two databases ,

• Security Association Database(SAD)

• Security Policy Database(SPD)
 Security Association Database(SAD)

• All Security Association’s (SA) maintained in a

Data Base called SAD – Security Association
Database.
 Security Association Database
• A security association is normally defined by the following
parameters in an SAD entry.
■ Security Parameter Index: A 32-bit value selected by the
receiving end of an SA to uniquely identify the SA. In an SAD
entry for an outbound SA, the SPI is used to construct the
packet’s AH or ESP header. In an SAD entry for an inbound SA,
the SPI is used to map traffic to the appropriate SA.
■ Sequence Number Counter: A 32-bit value used to generate
the Sequence Number field in AH or ESP headers
■ Sequence Counter Overflow: A flag indicating whether
overflow of the Sequence Number Counter should generate an
auditable event and prevent further transmission of packets on
this SA (required for all implementations).
■ Anti-Replay Window: Used to determine whether an
inbound AH or ESP packet is a replay
 Security Association Database
■ AH Information: Authentication algorithm, keys, key lifetimes,
and related parameters being used with AH (required for AH
implementations).
■ ESP Information: Encryption and authentication algorithm, keys,
initialization values, key lifetimes, and related parameters being
used with ESP (required for ESP implementations).
■ Lifetime of this Security Association: A time interval or byte
count after which an SA must be replaced with a new SA (and new
SPI) or terminated, plus an indication of which of these actions
should occur (required for all implementations).
■ IPsec Protocol Mode: Tunnel, transport, or wildcard.
■ Path MTU: Any observed path maximum transmission unit
(maximum size of a packet that can be transmitted without
fragmentation) and aging variables (required for all
implementations).
 IPv4 Header
The IPv4 header is defined in RFC 791. Its fields
are
 IPv6 Header
• The IPv6 header is defined in RFC 2460, and its fields are

• In IPv6, the equivalent field to IPv4's PROTOCOL is NEXT HEADER.

• It has the same values defined as the IPv4 PROTOCOL field, so ESP=50 and
AH=51. IPv6-style extension headers (roughly equivalent to OPTIONS in
the IPv4 header) are encoded as
 IPv6 Header
• In IPv6, the equivalent field to IPv4's PROTOCOL is NEXT HEADER.
• It has the same values defined as the IPv4 PROTOCOL field, so ESP=50 and
AH=51. IPv6-style extension headers (roughly equivalent to OPTIONS in
the IPv4 header) are encoded as

• LENGTH OF THIS HEADER is in units of 8-octet chunks, not counting the

first 8-octet chunk.
• AH looks like an IPv6 extension header, but its PAYLOAD LENGTH is in units
of 4- octet chunks instead of 8-octet chunks (and like other IPv6 extension
headers, doesn't count the first 8- octets).
• This violates one of the protocol folklore rules, which is that the LENGTH
field should always be defined the same way for all options, so that it is
easy to skip over unknown options.
 IP packet Processing
• IPsec is executed - packet-by-packet basis.
When IPsec is implemented,
– each outbound IP packet is processed - before
transmission
– each inbound packet is processed - logic after
reception before passing - next higher layer (e.g.,
TCP or UDP).
 OUTBOUND PACKETS

• Figure highlights the main elements of IPsec processing for

outbound traffic.
• A block of data from a higher layer, such as TCP, is passed down to
the IP layer and an IP packet is formed, consisting of an IP header
and an IP body.
TCP
IP Layer --- IP packet (IP – Header and body)
• Then the following steps occur:
1. IPsec searches the SPD for a match to this packet.
2. If no match is found, then the packet is discarded and an error message is
generated.
3. If a match is found, further processing is determined by the first matching
entry in the SPD. If the policy for this packet is DISCARD, then the packet is
discarded. If the policy is BYPASS, then there is no further IPsec processing; the
packet is forwarded to the network for transmission.
If Policy – DISCARD ( discard) , BYPASS – forward without processing
Processing Model for Outbound Packets

error message
Processing Model for Outbound Packets
4. If the policy is PROTECT, then a search is made
of the SAD for a matching entry. If no entry is
found, then IKE is invoked to create an SA with
the appropriate keys and an entry is made in the
SA.
5. The matching entry in the SAD determines the
processing for this packet. Either encryption,
authentication, or both can be performed, and
either transport or tunnel mode can be used.
The packet is then forwarded to the network for
transmission.
Processing Model for Inbound Packets
 Processing Model for Inbound Packets
1. IPsec determines whether this is an unsecured IP packet or
one that has ESP or AH headers/trailers, by examining the IP
Protocol field (IPv4) or Next Header field (IPv6).
2. If the packet is unsecured, IPsec searches the SPD for a match
to this packet. If the first matching entry has a policy of BYPASS,
the IP header is processed and stripped off and the packet body
is delivered to the next higher layer, such as TCP. If the first
matching entry has a policy of PROTECT or DISCARD, or if there is
no matching entry, the packet is discarded.
3. For a secured packet, IPsec searches the SAD. If no match is
found, the packet is discarded. Otherwise, IPsec applies the
appropriate ESP or AH processing.
Then, the IP header is processed and stripped off and the packet
body is delivered to the next higher layer, such as TCP.
 Encapsulating Security Payload (ESP)

• ESP can be used to provide confidentiality,

data origin authentication, connectionless
integrity, an anti-replay
• The set of services provided depends on
options selected at the time of Security
Association (SA) establishment and on the
location of the implementation in a network
topology.
• ESP can work with a variety of encryption and
authentication algorithms
 ESP Packet Format
Top-level format of an ESP packet. It contains the following fields
• Security Parameters Index(32bits)
– Identifies a security association
• Sequence Number(32bits)
– A monotonically increasing counter value
• Payload Data(variable)
– This is a transport-level segment (transport mode) or IP packet (tunnel mode) that is
protected by encryption.
• Padding(0-255bytes)
• Pad Length(8bits)
– the number of pad bytes immediately preceding
• Next Header(8bits)
– Identifies the type of data contained in the payload data field by identifying the first
header in that payload
• Integrity Check Value (variable):
– A variable-length field (must be an integral number of 32-bit words) that contains the
Integrity Check Value computed over the ESP packet minus the Authentication Data
field.
• Authentication Data(variable)
– Contains ICV computed over the ESP packet
ESP Packet Format
 ESP Packet Format
• An initialization value (IV), or nonce, is present if this
is required by the encryption or authenticated
encryption algorithm used for ESP.
• If tunnel mode is being used, then the IPsec
implementation may add traffic flow confidentiality
(TFC) padding after the Payload Data and before the
Padding field.
 Encryption and Authentication Algorithms

• The Payload Data, Padding, Pad Length, and Next Header fields are encrypted
by the ESP service.
• If the algorithm used to encrypt the payload requires cryptographic
synchronization data, such as an initialization vector (IV), then these data may
be carried explicitly at the beginning of the Payload Data field.
• If included, an IV is usually not encrypted, although it is often referred to as
being part of the ciphertext.
• The ICV field is optional. It is present only if the integrity service is selected
and is provided by either a separate integrity algorithm or a combined mode
algorithm that uses an ICV.
• The ICV is computed after the encryption is performed. This order of
processing facilitates rapid detection and rejection of replayed or bogus
packets by the receiver prior to decrypting the packet, hence potentially
reducing the impact of denial of service (DoS) attacks.
• It also allows for the possibility of parallel processing of packets at the
receiver that is decryption can take place in parallel with integrity checking.
• Note that because the ICV is not protected by encryption, a keyed integrity
algorithm must be employed to compute the ICV.
 AUTHENTICATION HEADER
• Provides support for data integrity and
authentication of IP packets
• Guards against replay attacks
 AUTHENTICATION HEADER
• Next Header(8bits)
– Identifies the type of header immediately following this
header
• Payload Length(8bits)
– Length of AH in 32-bit words minus 2
• Reserved(16bits)
• Security Parameters Index(32bits)
– Identifies a security association
• Sequence Number(32bits)
– A monotonically increasing counter value
• Authentication Data(variable)
– Contains ICV( Integrity Check value) or MAC
 Padding
The Padding field serves several purposes:
■ If an encryption algorithm requires the plaintext to be a
multiple of some number of bytes (e.g., the multiple of a
single block for a block cipher), the Padding field is used to
expand the plaintext (consisting of the Payload Data, Padding,
Pad Length, and Next Header fields) to the required length.
■ The ESP format requires that the Pad Length and Next
Header fields be right aligned within a 32-bit word.
Equivalently, the ciphertext must be an integer multiple of 32
bits. The Padding field is used to assure this alignment.
■ Additional padding may be added to provide partial traffic-
flow confidentiality by concealing the actual length of the
payload.
 Transport and Tunnel Modes
• Figure(a) shows two ways in which the IPsec ESP service can be
used. In the upper part of the figure, encryption (and optionally
authentication) is provided directly between two hosts.
• Figure (b) shows how tunnel mode operation can be used to set
up a virtual private network. In this example, an organization
has four private networks interconnected across the Internet.
• Hosts on the internal networks use the Internet for transport of
data but do not interact with other Internet-based hosts.
• By terminating the tunnels at the security gateway to each
internal network, the configuration allows the hosts to avoid
implementing the security capability.
• The former technique is supported by a transport mode SA,
while the latter technique uses a tunnel mode SA
Transport Mode
Transport Mode
Tunnel Mode
Transport & Tunnel Modes

Transport mode

Tunnel mode
Transport & Tunnel Mode
 Transport vs Tunnel Mode ESP

• Transport mode is used to encrypt and

optionally authenticate IP data
– data protected but header left in clear
– good for ESP host to host traffic
• Tunnel mode encrypts the entire IP packet
– add new header for next hop
– good for VPNs, gateway to gateway security
Transport mode and Tunnel mode
Protocol Operation for ESP
• IPSec tunnel mode is the default mode.

• Tunnel mode is used to encrypt traffic between secure IPSec Gateways( for
example two Cisco routers connected over the Internet via IPSec VPN.)

• Traffic from the client is encrypted, encapsulated inside a new IP packet and
sent to the other end.

• In tunnel mode, an IPSec header (AH or ESP header) is inserted between the
IP header and the upper layer protocol
68
IPSec Tunnel mode with ESP header:

ESPisisidentified
ESP identified in New
in the the IP
New IP header
header with
with an IP an IPIDprotocol
protocol of 50.
ID of 50.

69
 IPSec Tunnel mode with AH header:

The AH can be applied alone or together with the ESP, when IPSec is in tunnel
mode.

AH’s job is to protect the entire packet.

The AH does not protect all of the fields in the New IP Header because some
change in transit, and the sender cannot predict how they might change.

The AH protects everything that does not change in transit.

AH is identified in the New IP header with an IP protocol ID of 51.

70
• Transport mode provides the protection of our data, also known as IP
Payload, and consists of TCP/UDP header + Data, through an AH or ESP
header.

• The payload is encapsulated by the IPSec headers and trailers.

• The original IP headers remain intact, except that the IP protocol field is
changed to ESP (50) or AH (51), and the original protocol value is saved
in the IPsec trailer to be restored when the packet is decrypted.

• IPSec transport mode is usually used when another tunneling protocol

(like GRE) is used to first encapsulate the IP data packet, then IPSec is
used to protect the GRE tunnel packets. IPSec protects the GRE tunnel
traffic in transport mode. 71
 IPSec Transport mode with ESP header:

• the original IP Header is moved to the front.

• Placing the sender’s IP header at the front (with minor changes

to the protocol ID), proves that transport mode does not
provide protection or encryption to the original IP header .

• ESP is identified in the New IP header with an IP protocol

ID of 50.
72
 IPSec Transport mode with AH header:

▪ The AH can be applied alone or together with the ESP when IPSec is in
transport mode.

▪ AH’s job is to protect the entire packet, however, IPSec in transport

mode does not create a new IP header in front of the packet but places a
copy of the original with some minor changes to the protocol ID
therefore not providing essential protection to the details contained in
the IP header (Source IP, destination IP etc).

▪ AH is identified in the New IP header with an IP protocol ID of 51.

73
 Combining Security Associations
• SA’s can implement either AH or ESP (but not
both)
• To implement both need to combine SA’s
– form a SA bundle
– may terminate at different or same endpoints
– combined by
• transport adjacency
• iterated tunneling
• issue of authentication & encryption order
Transport mode

Tunnel mode
• Case 1. All security is provided between end
systems that implement IPsec. For any two end
systems to communicate via an SA, they must
share the appropriate secret keys. Among the
possible combinations are
a. AH in transport mode
b. ESP in transport mode
c. ESP followed by AH in transport mode (an ESP SA
inside an AH SA)
d. Any one of a, b, or c inside an AH or ESP in tunnel
mode
• Case 2. Security is provided only between gateways (routers,
firewalls, etc.) and no hosts implement IPsec. This case
illustrates simple virtual private network support. The security
architecture document specifies that only a single tunnel SA is
needed for this case. The tunnel could support AH, ESP, or ESP
with the authentication option. Nested tunnels are not
required, because the IPsec services apply to the entire inner
packet.
• Case 3. This builds on case 2 by adding end-to-end security.
The same combinations discussed for cases 1 and 2 are
allowed here. The gateway-to-gateway tunnel provides either
authentication, confidentiality, or both for all traffic between
end systems. When the gateway-to-gateway tunnel is ESP, it
also provides a limited form of traffic confidentiality.
Individual hosts can implement any additional IPsec services
required for given applications or given users by means of
end-to-end SAs.
• Case 4. This provides support for a remote
host that uses the Internet to reach an
organization’s firewall and then to gain access
to some server or workstation behind the
firewall.
• Only tunnel mode is required between the
remote host and the firewall. As in case 1, one
or two SAs may be used between the remote
host and the local host.
 AH – TRANSPORT & TUNNEL MODE
AH Transport Mode

AH Tunnel Mode
 ESP – TRANSPORT & TUNNEL MODE
• Transport Mode

⮚ Tunnel Mode
Summary – AH & ESP
Transport-Mode versus Tunnel-Mode Encryption