IP Security

Fig 1 shows TCP/IP protocol suite used in Internet where layer 3 (network layer) is referred to
as Internet Protocol (IP). IP Security (IPSec) provides authentication and confidentiality to
packets at this layer. IPSec is useful as security provided at higher layers i.e., application and
transport is not sufficient in few cases e.g.
(i) PGP and S/MIME provides security only to E-mail application not to all client/server
applications. Thus, IPSec is useful to such applications.
(ii) Though applications that utilize TCP as underlying protocol are protected by SSL/TLS
but not all applications use TCP some use UDP as well.
(iii)Routing protocols works at layer 3 (IP layer) and hence require security at this layer
only.

Application

Transport

Layer 3 Network IPSec provides

authentication and
Data link confidentiality to packet
Physical here

Fig 1

 IPSec can be implemented in firewall or router (i.e. devices connecting two

internetworks). It can also be implemented in the end hosts/systems.
 IPSec provides security to packets moving outwards and verifies the packets coming
inwards at the point of its implementation.
 Fig 2 shows the cases of use of IPSec.
 IPSec Enabled System

ad
o
ayl
ISP

PP
re I
ecu
er S
H ead
(2)

ec
PS
er I
ead
H
IP
IPSec Enabled
System

IP Header IPSec Header Secure IP Payload

Public (Internet) or
private network

(1)

IP Header IP Payload
IP Header IP Payload

IPSecEnable IPSecEnable
d device d device

Figure 2 Use of IP Security in an example network

Case 1: Securing two different company offices (LANs) located at different locations and
connected through public WANs (via routers). In this case, the traffic (IP packets) moving out
via router1 are protected by IPSec while it is verified at router2. The IPSec header is added to
outgoing packets at router1 while IPSec header is stripped off at router2. Thus, company’s
data remains safe despite of using public network.

Case2: Secure remote access to end user. The user’s end system (system1) uses IPSec to send
packets safely to access a distant network or Internet Service Provider (ISP) to utilize their
resources and thus saving physical commuting or travelling cost to distant network or ISP.

Apart from these cases, IPSec enhances security for existing E-commerce applications and
also establishes secure communication between partners over extranets and intranets.

 IPSec offers following benefits

(i) IPSec secures all the traffic that leaves the periphery of an organization through
firewall or router.
(ii) IPSec analyses all the incoming traffic to an organization from outside world and
can filler the undesired ones.
(iii)IPSec is transparent to applications as it operates below the transport layer
(TCP/UDP) and involves no up-gradation of existing client or server software.
(iv) IPSec operates independent of users and hence, remains unaffected by users joining
or leaving an organization. Also, it does not require any user training to educate the
users and operators.
(v) Individual user level security can be provided by IPSec, if desired by setting up
virtual secure subnetwork in an organization.
(vi) IPSec protects routing mechanism providing associations between routers such that
an attacker can not cause any harm to routing mechanism. IPSec ensures following
in routing:
(a) Provides authenticity to new router advertisements by which a router realizes its
presence.
(b) Provides authenticity to advertisements between any two neighboring routers.
(c) Non forging of routing update packet.
(d) Reply of receipt (redirect messages) from the router to whom request was made.

ARCHITECTURAL DESCRIPTION OFIP SECURITY

 IPSec specification mentions several documents (RFCs), worth mentioning are the
following ones:
RFC 2401 that gives overview of entire IPSec architecture; RFC 2402 provides
authentication extension details of IPSec; RFC 2406 provides encryption extension
details of IPSec and RFC 2408 that provides key management description.
 Features provides by IPSec are optional in IPv4 while compulsory in IPv6.
 These features are added in the form of headers placed after the main IP headers.
 A classification of documents pertaining to IPSec, published by IP Security protocol
working group (of IETF) is shown in Fig 3.
 Under this classification, basic definitions, requirements and mechanism is presented
under the Architecture head; Encapsulating Security Payload (ESP) format, issues are
covered under ESP protocol head, while that of Authentication Header (AH) are
 covered under AH protocol head; Encryption algorithm and their use is presented under
Encryption Algorithm head while Authentication Algorithm and their use is presented
under Authentication Algorithm head; Key management issues and schemes are covered
in Key management document; Domain of Interpretation (DOI) contains values and
terms required by other documents for relating with each other like algorithm identifies,
key lifetime etc [1].
 IPSec has two protocols:
(i) Authentication Header (AH) protocol
(ii) Encapsulating Security Payload (ESP) protocol
 IPSec provides following services via these protocol:
(i) Access control
(ii) Message integrity independent of connection
(iii)Origin authenticity
(iv) Protection against replays
(v) Confidentiality
(vi) Traffic flow (confidentiality) management
 An IP enabled system selects:
(i) Security protocol among AH and ESP
(ii) Algorithm(s) for providing desired service
(iii)Cryptographic keys as per the selected protocol
 Table 1 shows the services provided by each of AH and ESP.
 ESP has two options: with authentication and without authentication.
 AH does not provide confidentiality and traffic flow (confidentiality) management.
 ESP (without authentication) does not provide message integrity and origin authenticity.
 ESP (with authentication) provides all services.
 IPSec (Both AH and ESP) provides access control by checking the Security Association
(SA) from the Security Association Database (SAD). If no SA is found in the database,
packet is dropped.
 IPSec (Both AH and ESP) provides replay protection using sequence numbers and
sliding receiver window.
 SA,SAD and sliding receiver window are discussed next.

Table 1 services provided by each of AH and ESP

Access Message Origin Replay Confid- Traffic flow
Control Integrity Authenticity Protection entiality (confidentiality)
Management
AH Y Y Y Y
ESP (without Y Y Y Y
authentication-
only
encryption)
ESP (with Y Y Y Y Y Y
authentication)
 Architecture (general concepts, requirements,
definitions etc.) related documents

Documents related to use of Documents related to use, issues

ESP protocol and packet format of AH protocol
Optional

Documents related to Documents related to

packet encryption packet authentication
algorithm algorithm

Domain of interpretation (DOI) documents

(contains values required by other documents)

Documents for Key management schemes

Figure 3 IPSec Document Overview (RFC 2401)

Notion of Security Association

 A security association (SA) is a logical relationship or contract between two
communicating hosts. An SA helps in creating a secure channel between these hosts.
 It is one way only i.e. from sender to receiver. For securing traffic in opposite direction
another security association is required. Thus, for a two-way secure exchange, two SAs
are required.
 Fig 4 shows an example where confidentiality between sender S and destination D is
required. Thus, S defines on outbound SA having encryption algorithm and symmetric
shared key for encrypting the message send to D. Similarly,D defines an inbound SA
having same encryption algorithm and corresponding symmetric shared key for
decrypting the message.

Sender S Encrypted

Message

Algorithm – DES Algorithm – DES

Key – xyz Key – xyz

Fig 4 An example showing confidentiality between sender S and destination D

 An SA may be complex depending upon whether:
(i) SA for integrity, authentication along with encryption is required.
(ii) Different algorithms and parameters that exist for different protocols.
(iii)Communication is required between multiple senders and receiver.
In case (iii), each host should have both inbound and outbound SAs.
 Set of SAs are put together in a database termed as Security Association Database
(SAD).
 SAD is basically a matrix with each row having a single SA.
 There are two SADs: inbound SAD and outbound SAD for keeping inbound and
outbound SAs.
 Fig 5 shows one such SAD.
 An SA kept in SAD is uniquely identified by:
(i) Security Parameter Index (SPI)
 SPI is a 32 bit number or string assigned to the SA.
 It is determined during SA negotiation on the hosts.
 It is placed in the IPSec header (AH or ESP) so that receiver may identify the SA
for this packet from SAD.
 SPI remains same in all the packets tied to same SA.
(ii) Destination Address (DA) - Destination address of host (end user
system/firewall/router). Thus, SAs are unique per DA.
(iii) Protocol (P) - IPSec protocols: either AH or ESP.
 Hence, An SA is uniquely identified by SPI,DA,P combination.
 Each SA has associated parameters (remaining entries of a row in SAD) [2]. These are:
(i) Sequence Number Counter: A 32 bit counter used for generating sequence
number to be kept in sequence number field of IPSec header.
(ii) Sequence Number Overflow: Flag used to identify the sequence number
overflow and define the action taken by host in such circumstances.
(iii) Anti-Reply Window: A window of size n used to identify the replayed IPSec
packet at the receiver.
(iv) AH Information: Information for AH protocol like algorithm, keys, key
lifetime etc.
(v) ESP Information: Information for ESP protocol like encryption algorithm,
authentication algorithm, keys, key lifetime, Initial Vectors (IV) etc.
(vi) SA Lifetime: Defines lifetime of the associated SA.
(vii) IPSec Mode: Either transport or tunnel mode of IPSec.
(viii) Path MTU:Defines maximum packet size (without fragmentation) and aging
variables.
 A key distribution mechanism is linked to authentication and encryption mechanisms
via SPI
 <SPI, DA, P> Seq. Over Anti Info AH Info ESP Lifeti- IPSec Path
no flow replay protocol protocol me of mode MTU
flag window SA

An SA is uniquely identified by SPI, DA, P Tunnel or

Transport
SPI: Security Parameter Index Generated for AH or
DA: Destination Address ESP header Defines options upon
P: Protocol (Either AH or ESP) sequence overflow

Figure 5 Attributes for a single SA in a Security Association Database (SAD)

Security Policy Description

 A security policy (SP) in IPSec implies the type of security to be applied to IP packet
upon arrival or departure.
 The SP is kept in a Security Policy Database (SPD).
 Each entry in SPD is indexed based upon sextuple (termed as selectors): source address,
destination address, name, protocol, source port and destination port.
 Here, source and destination address are unicast or multicast IP addresses; name refers
to DNS entity; protocol refers to AH or ESP; source and destination port addresses refer
to port addresses of processes running on source and destination systems.
 An example SPD is shown in Fig 6.
 An inbound and outbound SPD is created on IPSec enabled host.
 An outbound SPD is consulted (SPD procedure is called) when a user packet is send by
transport layer to network layer (Fig 7).
 In this case outbound SPD is searched based upon sextuple index.
 Depending upon security policy either the packet is dropped (if it cannot be send further
as per policy) or bypassed (if no policy exists for this index) or IPSec is applied.
 IPSec application results in two cases:
(i) If outbound SA already exists then based upon SA index (triplet), the corresponding
SA is selected from out bound SAD. It is then followed by formation of AH or ESP
header (utilizing encryption and/or authentication) and packet transmission.
(ii) If outbound SA does not exists then Internet Key Exchange (IKE) protocol is
invoked so as to create new outbound and inbound SA for outbound SAD and
inbound SAD respectively for this traffic.
 Upon packet arrival at the receiver, the inbound SPD is searched using sextuple index
(Fig 8).
 Depending upon security policy either the received packet is dropped (if it cannot be
send further as defined by policy) or bypassed (if no policy exists for this index) or
IPSec is applied.
 IPSec application results in two cases:
(i) If inbound SA exists then based upon SA index, the corresponding SA is selected
from inbound SAD. It is then followed by verification of AH or ESP header (using
decryption and/or authentication) and removal of AH or ESP header. The extracted
packet is then delivered to transport layer.
(ii) If no inbound SA exists, packet is discarded [2].
 Index Policy
Identifies the type of security
to be applied on incoming or
outgoing packet

Index = <SA: Source Address, DA: Destination Address, Name: DNS Entity,
P: Protocol, SPort: Source Port, DPort: Destination Port>

Figure 6 Security Policy Database (SPD)

Jai

Higher (Transport) layer

An outgoing packet consulted in the

outbound SPD

Index Policy

<SA, DA,…..> policy

Apply
By Pass
Drop
X
Pass packet to IP layer Search the outbound SAD
and then through Data- SA found SA not found
link and Physical layers
to Veeru IPSec layer Internet Key
Exchange
(IKE)

Figure 7 Outbound processing

 Veeru

To higher
(Transport) layer

IPSec layer Discard

X
SA found SA not found

Search the inbound SAD

X By Pass
Discard Apply

Index Policy

<SA, DA,…..> policy

An inbound SPD searched for policy

An Incoming packet from Jai Travels via lower

layers (Data-link and Physical) to IP layer

From Jai

Figure 8 Inbound processing

Application layer Application layer

Transport layer Transport layer

IPSec layer Network layer

Network layer IPSec layer

New Network layer

(a) Transport Mode
(b) Tunnel Mode

Figure 9 Transport mode versus tunnel mode

Operational Modes of IPSec

 IPSec operates in: transport mode or tunnel mode.
 In transport mode, transport layer packet (UDP or TCP segment or ICMP packet) is
encapsulated by IPSec via addition of IPSec header and trailer. This encapsulated data is
then added to the network layer packet as IP payload (Fig 9(a)).
 IPSec does not protect the IP header in transport mode, only the transport layer
packet/information (IP payload) is protected [2].
 This transport mode is used to protect end to end (host to host e.g. client-server or
workstation-workstation) data (Fig 9(a) and 10) via encryption/decryption or
authentication.
 In tunnel mode, the entire IP packet including the IP header (old) is protected (Fig
11.9(b)). IPSec adds header/trailer to network layer packet and then adds a new IP
header to it. Thus, tunnel mode is used to protect end to intermediate data (Fig 10).
 The new IP header and old IP header contains different information.
 The tunnel mode is used between.
(i) Two routers
(ii) A host and a router
(iii)A router and a host
 Thus, it appears that the protected packet is going through an imaginary tunnel (safe
channel) on insecure network.
 In comparison, IP Sec lies in between transport layer & network layer in transport mode
where as in between network and new network layer in tunnel mode (Fig 9).
 ESP in transport mode protects (encrypts and optionally authenticates) the IP payload
only where as in tunnel mode protects the entire inner IP packet including the inner IP
header.
 AH in transport mode authenticates IP payload along with selected parts of IP header
where as in tunnel mode authenticates the entire inner IP packet and selected parts of
outer IP header.

End-to-end
authentication
Distant Host
Server or
LAN
Switch

Internet
End-to-end
authentication
Internal Router/firewall
Host
End-to-intermediate
authentication

Distant Host

Figure 10 Two different authentication types (End-to-End and End-to-Intermediate)

 1 2 3
(a) IPv4
(i) Before applying AH

*
Authenticated except for mutable fields

1: Original IP header 1 AH 2 3
2: TCP header
(ii) Transport mode
3: Data
Authenticated except for mutable fields in the new
IP header

New IP header AH 1 2 3
(iii) Tunnel mode

(b) IPv6 1 2 3 4

(i) Before applying AH

*
Authenticated except for mutable fields

Hop-by-hop,
destination
1: Original IP header 1 options**, routing,
AH Destination 2 3 4
2: Extension header fragment extension options**
(if present) headers
3: TCP header (ii) Transport mode
4: Data
Authenticated except for mutable fields in the new IP header

New IP header extension header AH 1 2 3 4

(iii) Tunnel mode

Figure 11 AH Authenticationin IPv4 and IPv6

* Mutable fields are set to zero for MAC calculation during authentication.
** Destination options may lie before or after the AH in IPv6 using IPSec.

11.4AUTHENTICATION HEADER (AH) PROTOCOL

 Authentication Header (AH) protocol enhances security by addition of Authentication Header

that provides source authentication and IP payload integrity.
 For this, AH protocol calculates Message Digest (MD) or hash using symmetric key and hash
function.
 The calculated MD is embedded into Authentication Header and then the IP packet is
transferred to receiver.
 Receiver verifies the authenticity and integrity of the packet.
 The position of AH depends upon the version (IPv4 or IPv6) and mode (transport or tunnel).
 AH protocol does not provide privacy.
 Fig 111 shows IPv4 and IPv6 before applying AH and after applying AH (for transport and
tunnel mode) [1].
 The IPSec Authentication Header (including fields) is shown in Fig 12. It consists of:
(i) Next Header - type of IP packet payload (TCP/UDP/ICMP/OSPF). IPSec copies the
protocol field value of original IP packet header into next header field. The protocol field
value in new IP header is changed to 51 to indicate that the packet contains AH.
(ii) Payload Length - it represents AH length in 4- byte multiples excluding first 8 bytes.
(iii) Reserved– 16 bits kept aside for future use.
(iv) Security Parameter Index (SPI) - required for Security Association (SA) and is like virtual
circuit identifier and remains same for a particular SA connection.
(v) Sequence Number – provides sequencing of IPSec packets and prevents replays using anti-
replay service. The sequence number changes for retransmitted packet and does wrap
around. Upon reaching its maximum value, a new connection is initiated.
(vi) Authentication Data (variable size- Integral of 32 bit words) - used for keeping Integrity
Check Value (ICV) for a packet.

Bit: 0 8 16 31
Next header Payload length RESERVED

Security parameters index (SPI)

Sequence number

Authentication data (variable)

Figure 12 IPSec Authentication Header

 Individual packets inside window are marked
or unmarked depending upon either valid
packet received or not received

N N+1

Window having size W

It is shifted towards right upon receipt of valid
packet

Figure 13 Anti-Replay Mechanism adopted by IPSec

AH protocol service against Replays (Fig 13): For implementing successful anti-replay service
sender and receiver must take following actions:

Actions taken by the Sender:

 Sender initializes the sequence number to zero at the time of establishing a new Security
Association (SA).
 This value is incremented and put in the IPSec header whenever a new packet is send.
32
 Thus, initial value of any assigned sequence number is 1 and its maximum value is 2 -1.
 When max value is reached; The SA is terminated and new SA in established.

Actions taken by Receiver:

 Receiver implements a window of size W (default W = 64).

Note:As packet may be received out of order due to connectionless and unreliable nature of IP,
the notion of window is used.

 For a successfully received largest sequence number N, the left and right edges of window are
represented as (N - W + 1) and (N) respectively.
 Any received packet whose MAC is verified and has -
(i) Sequence number that lies in the window i.e., within left and right edges of window, is
treated as valid and hence, the corresponding window slot is marked.
(ii) Sequence number that lies outside window and just after right edge, the window is advanced
(new right edge = previous right edge + 1) and the corresponding window slot is marked.
 Packet is discarded when:
(i) Authentication of packet fails(due to wrong MAC).
(ii) Sequence number of received packet lies towards left of the window i.e. before (N – W + 1).

Use of Integrity Check Value (ICV) by AH protocol

 The MAC or truncated codes produced by MAC algorithm (e.g. MD5/SHA-1 produced 96 bit
truncated code) are kept in authentication data field of IPSec header and termed as ICV.
 The default length of authentication data field is 96 bits.
 The MAC in AH is calculated using:
(i) IP header fields that do not change during transmission (e.g. source and destination address)
while ignoring the fields that may change (mutable fields) e.g. TTL field (these fields are set
to zero while calculating MAC).
(ii) Authentication data field in AH with value zero at both sender and receiver.
(iii) Upper level protocol data (IP payload, may alsoinclude inner IP packet in tunnel mode).