Cryptography & Network Security CSE

Network Security - I:
Security at application layer: PGP and S/MIME
Security at Transport Layer: SSL and TLS
Network Security - II:
Security at the Network Layer: IPSec, System Security
Security at Application layer and Transport Layer
 Various business services are now offered online through client-server applications.
 The most popular forms are web application and e-mail.
 In both applications, the client communicates to the designated server and obtains services.
 While using a service from any server application, the client and server exchange a lot of
information on the underlying intranet or Internet.
 We are aware that these transactions are vulnerable to various attacks.
 Network security entails securing data against attacks while it is in transit on a network.
E-mail Security
 Now-a-days, e-mail has become very widely used network application.
 Email is one of the most widely used and regarded network services.
 Currently message contents are not secure, may be inspected either in transit or by suitably
privileged users on destination system.
 Sending an e-mail is a one-time activity.
 In e-mail, there is no session. It is unidirectional message.

E-mail Architecture

UA- User Agent is useful to prepare the messages.

1. MTA- Message Transfer Agent is useful to send messages to mail server. This is Push program.
2. MAA-Message Access Agent is useful to receive messages from mail server. This is Pull
program.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Cryptographic algorithms
In e-mail security, the sender of the message needs to include the name or identifiers of the algorithms
used in the message.
Cryptographic secrets
In e-mail security, the encryption/decryption is done using a symmetric key algorithm, but the secret key
to decrypt the message is encrypted with the public key of the receiver and is sent with the message.

E-mail Security Protocols

 PGP (Pretty Good Privacy) – used for personal mailing and
 S/MIME (Secure/Multipurpose Internet Mail Extension) - used for commercial mailing

PGP (Pretty Good Privacy)

 Provides a confidentiality and authentication service.
 It can be used for electronic mail and file storage applications.
 Developed by Phil Zimmermann.
 Selected the best available cryptographic algorithms as building blocks.
 Integrated these algorithms into a general-purpose application that is independent of operating
system and processor and that is based on a small set of easy-to-use commands.
 Made the package and its documentation, including the source code, freely available via the
Internet, bulletin boards, and commercial networks. Entered into an agreement with a company to
provide a fully compatible, low–cost commercial version of PGP

PGP Growth
It is available free world wide inversions that run on a variety of platforms.
• The commercial version satisfies users who want a product that comes with vendor support
• It is based on algorithms that have survived extensive public review and are considered
extremely secure.
• It has a wide range of applicability.
• It was not developed by, nor is it controlled by, any governmental or standards organization
• Is now on an Internet standards track, however it still has an aura of an anti establishment
endeavor.

PGP Notation
Ks=session key used in symmetric encryption scheme.
PRa = private key of user A, used in public-key encryption scheme
PUa = public key of user A, used in public-key encryption scheme
EP = public-key encryption
DP = public-key decryption
EC = symmetric encryption
DC = symmetric decryption
H = hash function
||=concatenation
Z=compression using ZIP algorithm
R64=conversion to radix64 ASCII format1

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

PGP Operation–Authentication:

1. Sender creates a message.

2. SHA-1 used to generate 160-bit hash code of message.
3. Hash code is encrypted with RSA using sender's private key, result is attached to message.
4. Receiver uses RSA or DSS with sender's public key to decrypt and recover hash code.
5. Receiver generates new hash code for message and compares with decrypted hash code, if
match, message is accepted as authentic.

PGP Operation – Confidentiality:

1. Sender generates message and random 128-bit number to be used as session key for this
message only.
2. Message is encrypted, using 3DES with session key.
3. Session key is encrypted using RSA with recipient's public key, then attached to message.
4. Receiver uses RSA with its private key to decrypt and recover session key.
5. Session key is used to decrypt message.

PGP Operation – Confidentiality & Authentication

Use both services on same message.
Create signature & attach to message, encrypt both message & signature, attach RSA encrypted
session key.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

PGP Operation–Compression

As a default, PGP compresses the message after applying the signature but before encryption. This
has the benefit of saving space both for e-mail transmission and for file storage.
The placement of the compression algorithm, indicated by Z for compression and Z -1 for
decompression. So can store uncompressed message & signature for later verification & because
compression is non-deterministic uses ZIP compression algorithm.

PGP Operation–Email Compatibility

 When PGP is used, at least part of the block to be transmitted is encrypted. If only the
signature service is used, then the message digest is encrypted (with the sender’s private key). If the
confidentiality service is used, the message plus signature (if present) are encrypted (with a one- time
symmetric key).
 Thus, part or the entire resulting block consists of a stream of arbitrary 8-bit octets.
 However, many electronic mail systems only permit the use of blocks consisting of
ASCII text.
 To accommodate this restriction, PGP provides the service of converting the raw 8-bit
binary stream to a stream of printable ASCII characters. The scheme used for this purpose is radix-64
conversion.
 Each group of three octets of binary data is mapped into four ASCII characters. This
format also appends.

S/MIME (Secure/Multipurpose Internet Mail Extensions)

Secure/Multipurpose Internet Mail Extension (S/MIME) is a security enhancement to the MIME
Internet e-mail format standard based on technology from RSA Data Security. it appears likely that
S/MIME will emerge as the industry standard for commercial and organizational use, while PGP will
remain the choice for personal e-mail security for many users. S/MIME is defined in a number of
documents.
S/MIME support in many mail agents:
Ex: MS-Outlook, Mozilla, Mac-Mail, etc
To understand S/MIME, we need first understand mail format that it uses, namely MIME. We have
to learn about RFC5322 (Internet Message Format).

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

RFC5322
• Defines a format for text messages that are sent using electronic mail.
• Messages are viewed as having an envelope and contents.
• Envelope contains whatever information is needed to accomplish transmission & delivery.
• The contents compose the object to be delivered to the recipient.
• RFC5322 standard applies only to the contents.
The content standard includes a set of header fields that may be used by the mail system to create the
envelope. The overall structure of a message that conforms to RFC 5322 is very simple. A message
consists of some number of header lines (the header) followed by unrestricted text (the body). The
header is separated from the body by a blank line. Put differently, a message is ASCII text, and all
lines up to the first blank line are assumed to be header lines used by the user agent part of the mail
system. A header line usually consists of a keyword, followed by a colon, followed by the keyword’s
arguments; the format allows along line to be broken up into several lines. The most frequently used
keywords are From, To, Subject and Date. Here is an example message:
Date: October 8, 2009 2:15:49PM EDT
From: “William Stallings”
<ws@shore.net>Subject: The Syntax in RFC 5322
To: Smith@Other-host.com
Cc: Jones@Yet-Another-Host.com
Hello. This section begins the actual message body, which is delimited from the
message heading by a blank line.

Multipurpose Internet Mail Extensions (MIME):

An extension to the RFC5322 framework that is intended to address some of the problems and
limitations of the use of Simple Mail Transfer Protocol (SMTP) lists the following limitations of the
SMTP/5322 scheme.
1. SMTP cannot transmit executable files or other binary objects.
2. SMTP cannot transmit text data that includes national language characters, because these are
represented by 8-bit codes with values of 128 decimal or higher, and SMTP is limited to 7-bit ASCII.
3. SMTP servers may reject mail message over a certain size.
4. SMTP gateways that translate between ASCII and the character code EBCDIC do not use a
consistent set of mappings, resulting in translation problems.
MIME is intended to resolve these problems in a manner that is compatible with existing RFC5322
implementations. The specification is provided in RFCs 2045 through 2049.
The MIME specification includes the following elements.
1. Five new message header fields are defined, which may be included in an RFC5322
header. These fields provide information about the body of the message.
2. A number of content formats are defined, thus standardizing representations that
support multimedia electronic mail.
3. Transfer encodings are defined that enable the conversion of any content format into a
form that is protected from alteration by the mail system.
The Five Header Fields Defined in MIME:
• MIME-Version: Must have the parameter value 1.0. This field indicates that the
message conforms to RFCs 2045 and 2046.
• Content-Type: Describes the data contained in the body with sufficient detail that the

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

receiving user agent can pick an appropriate agent or mechanism to represent the data to the user
or otherwise deal with the data in an appropriate manner.
• Content-Transfer-Encoding: Indicates the type of transformation that has been used
to represent the body of the message in a way that is acceptable for mail transport.
• Content-ID: Used to identify MIME entities uniquely in multiple contexts.
• Content-Description: A text description of the object with the body; this is useful
when the object is not readable (e.g., audio data).

MIME Content Types

MIME Transfer Encodings:

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

S/MIME Functionality: S/MIME provides the following functions.

• Enveloped data: This consists of encrypted content of any type and encrypted content
encryption keys for one or more recipients.
• Signed data: A digital signature is formed by taking the message digest of the content
to be signed and then encrypting that with the private key of the signer. The content plus
signature are then encoded using base64 encoding. A signed data message can only be viewed
by a recipient with S/MIME capability.
• Clear-signed data: As with signed data, a digital signature of the content is formed.
However, in this case, only the digital signature is encoded using base64. As a result, recipients
without S/MIME capability can view the message content, although they cannot verify
thesignature.
• Signed and enveloped data: Signed-only and encrypted-only entities may be nested,
so that encrypted data may be signed and signed data or clear - signed data may be encrypted.
S/MIME Messages:

• S/MIME secures a MIME entity with a signature, encryption, or both. forming a

MIME wrapped
Public Key Cryptography Standards (PKCS) object have a range of content-types: enveloped data,
signed data, clear-signed data, registration request, certificate only message.
S/MIME Content Types

S/MIME Certificate Processing:

• S/MIME uses public-key certificates that conform to version 3 of X.509.
• The key-management scheme used by S/MIME is in some ways a hybrid between a
strict X.509 certification hierarchy and PGP’s web of trust.
• S/MIME managers and/or users must configure each client with a list of trusted keys
and with certificate revocation lists.
The responsibility is local for maintaining the certificates needed to verify incoming signatures and to
encrypt outgoing messages.
• The certificates are signed by certification authorities.
User Agent Role: An S/MIME user has several key-management functions to perform
• Key generation: The user of some related administrative utility (e.g., one associated
with LAN management) MUST be capable of generating separate Diffie-Hellman and DSS key

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

pairs and SHOULD be capable of generating RSA key pairs. Each key pair MUST be generated
from a good source of nondeterministic random input and be protected in a secure fashion. A
use agent SHOULD generate RSA key pairs with a length in the range of 768 to 1024 bits and
MUST NOT generates a length of less than 512 bits.
• Registration: A user’s public key must be registered with a certification authority in order to
receive an X.509 public-key certificate.
• Certificate storage and retrieval: A user requires access to a local list of certificates in
order to verify incoming signatures and to encrypt outgoing messages. Such a list could be
maintained by the user or by some local administrative entity on behalf of a number of users.

VeriSign Certificates There are several companies that provide certification authority (CA) services.
For example, Nortel has designed an enterprise CA solution and can provide S/MIME support within
an organization. There are a number of Internet-based CAs, including VeriSign, GTE, and the U.S.
Postal Service.
Enhanced Security Services: three enhanced security services have been proposed in an
Internet draft. The three services are: Signed receipts, Security labels, secure mailing lists.

Transport Level Security

Web Security considerations
The World Wide Web is fundamentally a client/server application running over the Internet and
TCP/IP intranets. The following characteristics of Web usage suggest the need for tailored security
tools:
 The Internet is two-way. Unlike traditional publishing environments—even electronic
publishing systems involving tele-text, voice response, or fax-back— the Web is vulnerable to
attacks on the Web servers over the Internet.
 The Web is increasingly serving as a highly visible outlet for corporate and product
information and as the platform for business transactions. Reputations can be damaged and money
can be lost if the Web servers are subverted.
 Although Web browsers are very easy to use, Web servers are relatively easy to configure and
manage, and Web content is increasingly easy to develop, the underlying software is extraordinarily
complex. This complex software may hide many potential security flaws.
 A Web server can be exploited as a launching pad into the corporation’s or agency’s entire
computer complex. Once the Web server is subverted, an attacker may be able to gain access to data
and systems not part of the Web itself but connected to the server at the local site.
 Casual and untrained (insecurity matters) users are common clients for Web-based services.
Such users are not necessarily aware of the security risks that exist and do not have the tools or
knowledge to take effective countermeasures.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Web security Threats:

A Comparison of Threats on the Web

Web Traffic Security Approaches:
A number of approaches to providing Web security are possible.
1. One way to provide Web security is to use IP security (IPsec) (Network Level). The
advantage of using IPsec is that it is transparent to end users and applications and provides a
general- purpose solution. It includes filtering capability that filters the unwanted data.
2. Another relatively general-purpose solution is to implement security just above TCP
(Transport Level). The example of this approach is the Secure Sockets Layer (SSL) and the
follow-on Internet standard known as Transport Layer Security (TLS). At this level, there are
two implementation choices. For full generality, SSL (or TLS) could be provided as part of the
underlying protocol suite and therefore be transparent to applications. Alternatively, SSL can be
embedded in specific packages. For example, Netscape and Microsoft Explorer browsers come
equipped with SSL, and most Web servers have implemented the protocol.

3. Application-specific security services are embedded within the particular application.

Application level shows examples of this architecture. The advantage of this approach is that the
service can be tailored to the specific needs of a given application.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

SSL (Secure Socket Layer)

SSL probably most widely used Web security mechanism, and it is implemented at the Transport
layer. SSL is designed to make use of TCP to provide a reliable end-to-end secure service. Netscape
originated SSL. Version 3 of the protocol was designed with public review and input from industry
and was published as an Internet draft document. Subsequently, became Internet standard known as
TLS (Transport Layer Security).
SSL Architecture:

SSL is not a single protocol but rather two layers of protocol. Two important SSL concepts are the
SSL session and the SSL connection, which are defined in the specification as follows.
1. Connection: A connection is a transport that provides a suitable type of service. For
SSL, such connections are peer-to-peer relationships. Every connection is associated with one
session.
2. Session: An SSL session is an association between a client and a server. Sessions are
created by the Handshake Protocol. Sessions define a set of cryptographic security parameters,
which can be shared among multiple connections.

Figure: SSL Protocol stack

SSL Record Protocol:

SSL Record Protocol defines two services for SSL connections:

1. Confidentiality: The Handshake Protocol defines a shared secret key that is used for
conventional encryption of SSL payloads. The message is compressed before being
concatenated with the MAC and encrypted, with a range of ciphers being supported as shown.
2. Message Integrity: The Handshake Protocol also defines a shared secret key that is
used to form a message authentication code (MAC).

Figure: SSL Record Protocol Operation

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

It shows the overall operation of the SSL Record Protocol. The Record Protocol takes an application
message to be transmitted, fragments the data into manageable blocks, optionally compresses the
data, applies a MAC, encrypts, adds a header, and transmits the resulting unit in a TCP segment.
Received data are decrypted, verified, decompressed and reassembled before being delivered to
higher-level users.

Figure: SSL Record Format

The final step of SSL Record Protocol processing is to prepare a header consisting of the following
fields:
Content Type (8bits): The higher-layer protocol used to process the enclosed fragment.

Major Version (8bits): Indicates major version of SSL in use. For SSLv3, the value is 3. Minor
Version (8 bits): Indicates minor version in use. For SSLv3, the value is 0.

Compressed Length (16 bits): The length in bytes of the plaintext fragment (compressed
fragment if compression is used). The maximum value is 214+2048.

Change Cipher Spec Protocol:

 The Change Cipher Spec Protocol is one of the three SSL – specific protocols that use the
SSL Record Protocol.
 It is the simplest, consisting of a single message, which consists of a single byte with the
value.
 The sole purpose of this message is to cause the pending state to be copied into the current
state, which updates the cipher suite to be used on this connection.

SSL Alert Protocol:

The Alert Protocol is used to convey SSL – related alerts to the peer entity. As with other
applications that use SSL, alert messages are compressed and encrypted, as specified by the current
state. Each message in this protocol consists of two bytes; the first takes the value warning (1) or
fatal (2) to convey the severity of the message. This byte contains a code that indicates the specific
alert.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

SSL Handshake Protocol:

The most complex part of SSL is the Handshake Protocol. This protocol allows the server and client
to authenticate each other and to negotiate an encryption and MAC algorithm and cryptographic keys
to be used to protect data sent in an SSL record. The Handshake Protocol is used before any
application data is transmitted. The Handshake Protocol consists of a series of messages exchanged
by client and server.

The exchange can be viewed in 4 phases:

• Phase1: Establish Security Capabilities – this phase is used by the client to initiate a
logical connection and to establish the security capabilities that will be associated with it.
• Phase2: Server Authentication and Key Exchange – the server begins this phase by
sending its certificate if it needs to be authenticated.
• Phase 3: Client Authentication and Key Exchange- the client should verify that the
server provided a valid certificate if required and check that the server_hello parameters are
acceptable
• Phase 4: Finish - this phase completes the setting up of a secure connection. The client
sends a change_cipher_spec message and copies the pending Cipher Spec into the current
Cipher Spec. At this point the handshake is complete and the client and server may begin to
exchange application layer data.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Transport Layer Security (TLS) Protocol

In order to provide an open Internet standard of SSL, Internet Engineering Task Force (IETF)
released The Transport Layer Security (TLS) protocol in January 1999. TLS is defined as a proposed
Internet Standard in RFC 5246.
Salient Features

 TLS protocol has same objectives as SSL.

 It enables client/server applications to communicate in a secure manner by authenticating,
preventing eavesdropping and resisting message modification. TLS protocol sits above the
reliable connection-oriented transport TCP layer in the networking layer’s stack.
 The architecture of TLS protocol is similar to SSLv3 protocol. It has two sub protocols: the
TLS Record protocol and the TLS Handshake protocol.
 Though SSLv3 and TLS protocol have similar architecture, several changes were made in
architecture and functioning particularly for the handshake protocol.

Comparison of TLS and SSL Protocols:

1. Protocol Version − The header of TLS protocol segment carries the version number 3.1
to differentiate between number 3 carried by SSL protocol segment header.
2. Message Authentication − TLS employs a keyed-hash message authentication code
(HMAC). Benefit is that H-MAC operates with any hash function, not just MD5 or SHA, as
explicitly stated by the SSL protocol.
3. Session Key Generation − There are two differences between TLS and SSL protocol for
generation of key material.
• Method of computing pre-master and master secrets is similar. But in TLS
protocol, computation of master secret uses the HMAC standard and
pseudorandom function (PRF) output instead of ad-hoc MAC.
• The algorithm for computing session keys and initiation values (IV) is
different in TLS than SSL protocol.
4. Alert Protocol Message
• TLS protocol supports all the messages used by the Alert protocol of SSL,
except No certificate alert message being made redundant. The client sends
empty certificate in case client authentication is not required.
• Many additional Alert messages are included in TLS protocol for other
error conditions such as record-overflow, decode-error etc.
5. Supported Cipher Suites − SSL supports RSA, Diffie-Hellman and Fortezza cipher
suites. TLS protocol supports all suits except Fortezza.
6. Client Certificate Types − TLS defines certificate types to be requested in a certificate-
request message. SSLv3 support all of these. Additionally, SSL support certain other types of
certificate such as Fortezza.
7. Certificate Verify and Finished Messages
• In SSL, complex message procedure is used for the certificate-verify
message. With TLS, the verified information is contained in the handshake
messages itself thus avoiding this complex procedure.
• FinishedmessageiscomputedindifferentmannersinTLSandSSLv3.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

8. Padding of Data − In SSL protocol, the padding added to user data before encryption is
the minimum amount required to make the total data-size equal to a multiple of the cipher’s
block length. In TLS, the padding can be any amount that results in data-size that is a multiple of
the cipher’s block length, up to a maximum of 255 bytes.

Secure Shell Protocol (SSH):

The salient features of SSH are as follows−
 SSH is a network protocol that runs on top of the TCP/IP layer. It is designed to replace
the TELNET which provided unsecure means of remote logon facility.
 SSH provides a secure client/server communication and can be used for tasks such as file
transfer and e-mail.
 SSH 2 is a prevalent protocol which provides improved network communication security
over earlier version SSH1.

Figure: SSH Protocol stack

Transport Layer Protocol:

SSH protocol provides data confidentiality, server (host) authentication, and data integrity. It may
optionally provide at a compression as well.
Server Authentication− Host keys are asymmetric like public/private keys. A server uses a public
key to prove its identity to a client. The client verifies that contacted server is a―known ‖host from
the database it maintains. Once the server is authenticated, session keys are generated.
Session Key Establishment− After authentication, the server and the client agree upon cipher to be
used. Session keys are generated by both the client and the server. Session keys are generated before
user authentication so that usernames and passwords can be sent encrypted. These keys are generally
replaced at regular intervals (say, every hour) during the session and are destroyed immediately after
use.
Data Integrity− SSH uses Message Authentication Code (MAC) algorithms to for data integrity
check. It is an improvement over 32bit CRC used by SSH1.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

User Authentication Protocol:

In this part of SSH authenticates the user to the server. The server verifies that access is given to
intended users only. Many authentication methods are currently used such as, typed passwords,
Kerberos, public-key authentication, etc.
Connection Protocol:
This provides multiple logical channels over a single underlying SSH connection SSH Services: SSH
provides three main services that enable provision of many secure solutions. These services are
briefly described as follows:
Secure Command-Shell (Remote Logon) – It allows the user to edit files, view the contents of
directories, and access applications on connected device. Systems administrators can remotely
start/view/stop services and processes, create user accounts, and change file/directories permissions
and so on. All tasks that are feasible at a machine’s command prompt can now be performed securely
from the remote machine using secure remote logon.

Secure File Transfer − SSH File Transfer Protocol (SFTP) is designed as an extension for SSH-2 for
secure file transfer. In essence, it is a separate protocol layered over the Secure Shell protocol to
handle file transfers. SFTP encrypts both the username/password and the file data being transferred.
It uses the same port as the Secure Shell server, i.e. systems port no 22.

Port Forwarding (Tunneling) − It allows data from unsecured TCP/IP based applications to be
secured. After port forwarding has been set up, Secure Shell reroutes traffic from a program (usually
a client) and sends it across the encrypted tunnel to the program on the other side (usually a server).
Multiple applications can transmit data over a single multiplexed secure channel, eliminating the
need to open many ports on a firewall or router.

Network Security-II: Security at Network Layer: IPSec, System Security

IP SECURITY OVERVIEW
IPSec is an Internet Engineering Task Force (IETF) standard suite of protocols that provides data
authentication, integrity, and confidentiality as data is transferred between communication points
across IP networks. IPSec provides data security at the IP packet level. A packet is a data bundle that
is organized for transmission across a network, and it includes a header and payload (the data in the
packet).
IPSec SECURITY FEATURES:
IPSec is the most secure method commercially available for connecting network sites. IPSec was
designed to provide the following security features when transferring packets across networks:
Authentication: Verifies that the packet received is actually from the claimed sender.
Integrity: Ensures that the contents of the packet did not change in transit.
Confidentiality: Conceals the message content through encryption.

IPSec ELEMENTS:

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

IPSec contains the following elements:

Encapsulating Security Payload (ESP): Provides confidentiality, authentication and integrity.

Authentication Header (AH): Provides authentication and integrity.

Internet Key Exchange (IKE): Establish shared symmetric key. Provides key management and
Security Association(SA) management.

APPLICATIONS OF IPSec:
IPSec provides the capability to secure communications across a LAN, across private and public
WANs, and across the Internet.
Examples of its use include the following:
 Secure branch office connectivity over the Internet
 Secure remote access over the Internet
Establishing extranet and intranet connectivity with partners:
 IPSec can be used to secure communication with other organizations, ensuring
authentication and confidentiality and providing a key exchange mechanism.
Enhancing electronic commerce security:
 Even though some web and electronic commerce applications have built-in security
protocols, the use of IPSec enhances that security.

BENEFITS OF IPSEC:
 IPSec provides strong security within and across the LANs.
 Firewall uses IPSec to restrict all those incoming packets which are not using IP. Since firewall
is the only way to enter into an organization, restricted packets cannot enter.
 IPSec is below the transport layer (TCP, UDP) and so is transparent to applications.
 There is no need to change software on a user or server system when IPSec is implemented in
the firewall or router. Even if IPSec is implemented in end systems, upper-layer software, including
applications, is not affected. IPSec can be transparent to end users. IPSec can provide security for
individual users if needed.

IPSec Scenario:

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

IPSec Architecture:
Architecture covers general concepts of security requirements, definitions, and mechanisms defining
IPSec technology.

Figure: IPSec Architecture

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Encapsulating Security Payload (ESP): The ESP header is designed to provide a mix of security
services in IPv4 and IPv6. ESP may be applied alone, in combination with AH, or in a nested
fashion. It consists of an encapsulating header and trailer used to provide encryption or combined
encryption/authentication.CurrentspecificationisRFC4303

Authentication Header (AH): An extension header to provide message authentication. Current

specification is RFC 4302.

Encryption algorithms: Encryption algorithms encrypt data with a key. The ESP module in IPsec
uses encryption algorithms.

Authentication algorithms: Authentication algorithms produce an integrity check sum value or

digest that is based on the data and a key. The AH module uses authentication algorithms. The ESP
module can use authentication algorithms as well.

Domain of Interpretation (DOI): DOI is the identifier which support both AH and ESP protocols. It
contains values needed for documentation related to each other.

Key Management: It contains the document that describes how the keys are exchanged between
sender and receiver.

Transport and Tunnel Modes in IPSec

IPSec operates in two modes:
1) Tunnel Mode
2) Transport Mode

Tunnel Mode:

With tunnel mode, the entire original IP packet is protected by IPSec. This means IPSec wraps the
original packet, encrypts it, and adds a new IP header and sends it to the other side. Original IP
Header not visible to attacker (if it is using ESP).Attacker does not know which hosts are talking.
Tunnel mode is most commonly used between gateways, end-system to Gateways.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Figure: IPSec Tunnel mode

Transport Mode:
IPSec Transport mode is used for end-to-end communications, for example, for communication
between a client and a server or between a work station and a gateway (if the gateway is being
treated as a host).
When using the transport mode, only the IP payload is encrypted. AH or ESP provides protection for
the IP payload. The original IP header is not changed, so the passive attackers can see who is talking.

Figure: IPSec Transport Mode

AUTHENTICATION HEADER (AH)

The Authentication Header provides support for data integrity and authentication of IP packets. Data
integrity service insures that data inside IP packets is not altered during the transit.
The authentication feature enables an end system to authenticate the user or application and filter
traffic accordingly. It also prevents the address spoofing attacks.
AH is implemented in one way only i.e., Authentication along with Integrity.
AH provides authentication for as much of IP header as possible, but cannot all be protected by AH.
AH also includes an IPSec sequence number, which provides protection against replay attacks
because this number is also included in authenticated data and can be chec ked by the receiving
party. Data privacy is not provided by AH.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Figure: Authentication Header Format

1. Next Header: Identifies the type of header that immediately following the AH.
2. Payload Length: Length of Authentication Header in 32-bit words.
3. Reserved: For future use.
4. Security Parameters Index: Identifies a security association.
5. Sequence Number: A monotonically increasing counter value.
6. Authentication Data (variable): A variable-length field that contains the Integrity Check Value
(ICV), or MAC, for this packet.

Encapsulating Security Payload (ESP):

Security services can be provided between a pair of communicating hosts, between a pair of
communicatingsecuritygateways,orbetweenasecuritygatewayandahost.TheESPheaderis inserted after
the IP header and before the next layer protocol header (transport mode) or before an encapsulated IP
header (tunnel mode). ESP can be used to provide confidentiality, data origin authentication,
connectionless integrity, an anti-replay service (a form of partial sequence integrity), and (limited)
traffic flow confidentiality. The set of services provided depends on options selected at the time of
Security Association (SA) establishment and on the location of the implementation in a network
topology.

Figure: ESP Format

1. Security Parameters Index: Identifies a security association.
2. Sequence Number: A monotonically increasing counter value; this provides an anti-replay
function, as discussed for AH.
3. Payload Data: This is a transport-level segment (transport mode) or IP packet (tunnel mode) that
is protected by encryption.
4. Padding (0-255 bytes): Extra bits or spaces are added to the message in order to maintain
confidentiality
5. Pad Length: Indicates the number of pad bytes immediately preceding this field.
6. Next Header: means the next pay load or next data

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

7. Authentication Data (variable): contains the Integrity Check Value computed over the ESP
packet minus the Authentication Data field.

Security Associations (SAs)

An SA is a relationship between communicating devices that describes how they will use security
services to communicate securely.
If client wants to communicate with server, it has client Security Association, if Server wants to reply
to client, it has server Security association.
These SAs are one way communications.
If two parties need to communicate, they must determine which algorithms (RSA, 3DES, MD5,
SHA…) and session keys are used. SA used by IPSec to track all these parameters for each session.
You will need to configure SA parameters and monitor SAs on Cisco routers and the PIX Firewall.
• Separate pair of IPSec SAs are setup for AH and ESP transform.
• Each IPSec peer agrees to set up SAs consisting of policy parameters to be used during the
IPSec session.
• The SAs are unidirectional for IPSec so that peer1 will offer peer 2 a policy.
• If peer 2 accepts this policy, it will send that policy back to peer1. This establishes two one- way
SAs between the peers.
• Two-way communication consists of two SAs, one for each direction.
• Each SA consists of values such as destination address, a security parameter index (SPI), the
IPSec transforms used for that session, security keys, and additional attributes such as IPSec lifetime.

A security association is uniquely identified by three parameters:

• Security Parameters Index (SPI): A bit string assigned to this SA and having local
significance only. SPI is located in AH and ESP headers. SPI enables the receiving system under
which the packet is to process.
• IP Destination Address: It is the end point address of SA which can be end user system or a
network system.
• Security Protocol Identifier: security protocol identifier indicates whether the associations
is an AH or ESP.
All the SAs are maintained in Security Association Database (SAD)

SA Parameters:
Sequence Number Counter: A 32-bit value used to generate the Sequence Number field in AH or
ESP headers.

Sequence Counter Overflow: A flag indicating whether overflow of the Sequence Number Counter
should generate an auditable event and prevent further transmission of packets on this SA.
Anti-Replay Window: Avoid duplicate of packets

AH Information: Authentication algorithm, keys, key lifetimes, and related parameters being used
with AH.
ESP Information: Encryption and authentication algorithm, keys, initialization values, key lifetimes,

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

and related parameters being used with ESP (required for ESP implementations).
Life time of This Security Association: A time interval or byte count after which an SA must be
replaced with a new SA or terminated.

IPSec Protocol Mode: This parameter represents the type of mode used for IPSec implementation.
The mode may be a Tunnel or transport.

Security Policy (SP)

A Security Policy is a set of rules that define the type security applied to a packet when it is to be
sent or when it has arrived. It defines the network traffic at the IP layer.
IPSec protects your private network from internet attacks through end-to-end security.
IPSec policy is determined primarily by the interaction of two databases, the Security Association
Database (SAD) and the Security Policy Databases (SPD)
IPSec policies must be carefully designed, configures, coordinated and managed to ensure that IPSec
communication is successful.

Security Policy Database (SPD)

IPSec Policies are maintained in the Security Policy Database (SPD).

IPSec Policies defines which traffic to be protected, how it is to be protected and with whom to
protect it.
The sending host determines what policy is appropriate for the packet, depending on various
"Selectors" by checking in the Security Policy Database (SPD).
"Selectors" can include Source and Destination IP Addresses, Name (User ID: System Name),
Transport Layer Protocols (TCP or UDP) or Source and Destination Ports.
The Security Policy Database (SPD) indicates what the policy is for a particular packet. If the packet
requires IPSec processing, it will be it is passed to the IPSec module for the required processing.

Key Management of IPSec

The key management portion of IPSec involves the determination and distribution of secret keys
typical requirement is four keys for communication between two applications: transmit and receive
pairs for both AH and ESP.

Keys are managed by 2 ways:

• Manual: A system administrator manually configures each system with its own keys and
with the keys of other communicating systems. This is suitable for small, relatively static
environments.
• Automated: An automated system enables the on-demand creation of keys for SAs and
facilitates the use of keys in a large distributed system.

The default automated key management protocol for IPSec is referred to as ISAKMP/Oakley.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Key management protocols

1. Oakley Key Determination Protocol

2. Internet Security Association and Key Management Protocol (ISAKMP)

Oakley Key Determination Protocol:

• Oakley is a key exchange protocol based on the Diffie-Hellman algorithm but providing
added security.
• Oakley is generic in that it does not dictate specific formats.

The Diffie-Hellman algorithm has two attractive features:

1. Secret keys are created only when needed.

2. The exchange requires no preexisting infrastructure other than an agreement on the global
parameters.
3. It does not provide any information about the identities of the parties.
4. It is subject to a man-in-the-middle attack

It is computationally intensive. As a result, it is vulnerable to a clogging attack, in which an opponent

requests a high number of keys. Oakley is designed to retain the advantages of Diffie-Hellman while
countering its weaknesses.

Features of Oakley:

The Oakley algorithm is characterized by five important features:

• It employs a mechanism known as cookies to thwart clogging attacks.

• It enables the two parties to negotiate a group; this, in essence, specifies the global
parameters of the Diffie-Hellman key exchange.
• It uses nonces to ensure against replay attacks.
• It enables the exchange of Diffie-Hellman public key values.
• It authenticates the Diffie-Hellman exchange to thwart man-in-the-middle attacks.

Internet Security Association and Key Management Protocol (ISAKMP)

ISAKMP provides a framework for Internet key management and provides the specific protocol
support, including formats, for negotiation of security attributes.

ISAKMP Header Format:

An ISAKMP message consists of an ISAKMP header followed by one or more payloads. All of this
is carried in a transport protocol. The specification dictates that implementations must support the use
of UDP for the transport protocol.

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

It consists of the following fields:

1. Initiator Cookie (64bits): Cookie of entity that initiated SA establishment, SA notification, or SA
deletion.
2. Responder Cookie (64bits): The cookie of entity that is responding to an SA establishment
request, SA notification, or SA deletion. On the first message, the responder cookie is zero.
3. Next Payload (8bits): Indicates the type of the first payload in the message.
4. Major Version (4bits): Indicates major version of ISAKMP in use.
5. Minor Version (4bits): Indicates minor version in use.
6. Exchange Type (8bits): Indicates the type of exchange.
7. Flags (8bits): Indicates specific options set for this ISAKMP exchange.
8. Message ID (32bits): Unique ID for this message.
9. Length (32bits): Length of total message (header plus all payloads) in octets.

System Security

Malicious Software

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

1. Malicious programs which spread from machine to machine without the consent of the
owners/operators/users
Windows Automatic Update is (effectively) consensual

2. Many strains possible

 Viruses
 Worms
 Compromised Auto-updates
• No user action required, very dangerous

Trapdoors (Back doors)

• Secret entry point into a program

• Allows those who know access bypassing usual security procedures, e.g.,
authentications
• Have been commonly used by developers
• A threat when left in production programs allowing exploited by attackers
• Very hard to block in O/S
• Requires good s/w development & update

Logic Bomb

• One of oldest types of malicious software

• Code embedded in legitimate program
• Activated when specified conditions met
– E.g., presence/absence of some file
– Particular date/time
– Particular user
– Particular series of keystrokes
• When triggered typically damage system
– Modify/delete files/disks

Trojan horse

• Programs that appear to have one function but actually perform another.
• Modern Trojan Horse: resemble a program that the user wishes to run - usually
superficially attractive
– E.g., game, s/w upgrade etc
• When run performs some additional tasks
– Allows attacker to indirectly gain access they do not have directly
• Often used to propagate a virus/worm or install a backdoor or simply to destroy data

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Zombie

• Program which secretly takes over another networked computer

• Then uses it to indirectly launch attacks
• Often used to launch distributed denial of service (DDoS) attacks
• Exploits known flaws in network systems.

Worms

• Autonomous, active code that can replicate to remote hosts without any triggering
– Replicating but not infecting program
• Because they propagate autonomously, they can spread much more quickly than
viruses!
• Speed and general lack of user interaction make them the most significant threats

Virus
• Definition from RFC 1135: A virus is a piece of code that inserts itself into a host,
including operating systems, to propagate. It cannot run independently. It requires that
its host program be run to activate it.
• On execution
– Search for valid target files
• Usually executable files
• Often only infect uninfected files
– Insert a copy into targeted files
• When the target is executed, the virus starts running
• Only spread when contaminated files are moved from machine to machine.

Virus Operation

• virus phases:
– dormant – waiting on trigger event
– propagation – replicating to programs/disks
– triggering – by event to execute payload
– execution – of payload
• details usually machine/OS specific
– exploiting features/weaknesses

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Anatomy of a Virus

• Two primary components

– Propagation mechanism
– Payload
• Propagation
– Method by which the virus spreads itself.
– Old days: single PC, transferred to other hosts by ways of floppy diskettes.
– Nowadays: Internet.

Virus Detection

• Scanning
– Depend on prior knowledge of a virus
– Check programs before execution
– Need to be regularly updated
• Integrity Checking
– Read entire disk and record integrity data that acts as a signature for the files
and system sectors
– Use cryptographic computation technique instead of simple checksum
• Interception
– Monitoring for system-level routines that perform destructive acts
– Good for detecting logic bomb and Trojan horse
– Cannot depend entirely upon behavior monitors as they are easily bypassed.
• Combination of all three techniques can detect most viruses

Virus Recovery

• Extricate the virus from the infected file to leave the original behind
• Remove the redirection to the virus code
• Recover the file from backup
• Delete the files and move on with life

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Firewall
 A choke point of control and monitoring
 Interconnects networks with differing trust
 Imposes restrictions on network services
o only authorized traffic is allowed
 Auditing and controlling access
o can implement alarms for abnormal behavior
 Itself immune to penetration
 Provides perimeter defence

Classification of Firewall

Characterized by protocol level it controls in

 Packet filtering
 Circuit gateways
 Application gateways
Combination of above is dynamic packet filter

Firewalls – Packet Filters

 Simplest of components
 Uses transport-layer information only
o IP Source Address, Destination Address
o Protocol/Next Header (TCP, UDP, ICMP, etc)
o TCP or UDP source & destination ports
o TCP Flags (SYN, ACK, FIN, RST, PSH, etc)
o ICMP message type

Security & Performance of Packet Filters

 IP address spoofing
o Fake source address to be trusted

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

o Add filters on router to block

 Tiny fragment attacks
o Split TCP header info over several tiny packets
o Either discard or reassemble before check
 Degradation depends on number of rules applied at any point
 Order rules so that most common traffic is dealt with first
 Correctness is more important than speed

Firewalls – State-ful Packet Filters

 Traditional packet filters do not examine higher layer context

o ie matching return packets with outgoing flow
 Stateful packet filters address this need
 They examine each IP packet in context
o Keep track of client-server sessions
o Check each packet validly belongs to one
 Hence are better able to detect bogus packets out of context

Stateful Filtering

Firewall Gateways
 Firewall runs set of proxy programs
o Proxies filter incoming, outgoing packets

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

o All incoming traffic directed to firewall

o All outgoing traffic appears to come from firewall
 Policy embedded in proxy programs
 Two kinds of proxies
o Application-level gateways/proxies
 Tailored to http, ftp, smtp, etc.
o Circuit-level gateways/proxies
 Working on TCP level

Firewalls - Application Level Gateway (or Proxy)

Application-Level Filtering

 Has full access to protocol

o user requests service from proxy
o proxy validates request as legal
o then actions request and returns result to user
 Need separate proxies for each service
o E.g., SMTP (E-Mail)
o NNTP (Net news)
o DNS (Domain Name System)
o NTP (Network Time Protocol)
o custom services generally not supported

App-level Firewall Architecture

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Firewalls - Circuit Level Gateway

 Relays two TCP connections

 Imposes security by limiting which such connections are allowed
 Once created usually relays traffic without examining contents
 Typically used when trust internal users by allowing general outbound connections
 SOCKS commonly used for this

Bastion Host

 Highly secure host system

 Potentially exposed to "hostile" elements
 Hence is secured to withstand this
o Disable all non-required services; keep it simple
 Trusted to enforce trusted separation between network connections
 Runs circuit / application level gateways
o Install/modify services you want
 Or provides externally accessible services
Bastion/Screened Host Architecture

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Screened Subnet Using Two Routers

Dynamic Packet Filters

 Most commonly used and provides good administrators protection and full transparency.
 Network given full control over traffic.
 Captures semantics of a connection.

Firewalls Disadvantages

 Useless against attacks from the inside

o Attackers exists on inside

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

o Malicious code is executed on an internal machine

 Organizations with greater insider threat
o Banks and Military
 Protection must exist at each layer
o Assess risks of threats at every layer
 Cannot protect against transfer of all virus infected programs or files
o because of huge range of O/S & file types

IDS – Intruder Detection System

Intrusion

– A set of actions aimed to compromise the security goals, namely

• Integrity, confidentiality, or availability, of a computing and networking resource
Intrusion detection

– The process of identifying and responding to intrusion activities

Intrusion prevention

– Extension of ID with exercises of access control to protect computers from exploitation

Elements of Intrusion Detection

• Primary assumptions:
– System activities are observable
– Normal and intrusive activities have distinct evidence
• Components of intrusion detection systems:
– From an algorithmic perspective:
• Features - capture intrusion evidences
• Models - piece evidences together
– From a system architecture perspective:
• Various components: audit data processor, knowledge base, decision engine,
alarm generation and responses
Components of Intrusion Detection System

 Audit Data Preprocessor

 Detection Models
 Detection Engine
 Decision Table

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Intrusion Detection Approaches

• Modeling
– Features: evidences extracted from audit data
– Analysis approach: piecing the evidences together
• Misuse detection (a.k.a. signature-based)
• Anomaly detection (a.k.a. statistical-based)
• Deployment: Network-based or Host-based
• Development and maintenance
– Hand-coding of “expert knowledge”
– Learning based on audit data

Host-Based IDSs

• Using OS auditing mechanisms

– E.G., BSM on Solaris: logs all direct or indirect events generated by a user
– strace for system calls made by a program
• Monitoring user activities
– E.G., Analyze shell commands
• Monitoring executions of system programs
– E.G., Analyze system calls made by sendmail

Network IDSs

• Deploying sensors at strategic locations

– E.G., Packet sniffing via tcpdump at routers
• Inspecting network traffic
– Watch for violations of protocols and unusual connection patterns
• Monitoring user activities
– Look into the data portions of the packets for malicious command sequences
• May be easily defeated by encryption
– Data portions and some header information can be encrypted

Suthoju Girija Rani, Assistant Professor, CSE.

Cryptography & Network Security CSE

Requirements of Network IDS

• High-speed, large volume monitoring

– No packet filter drops
• Real-time notification
• Mechanism separate from policy
• Extensible & Broad detection coverage
• Economy in resource usage
• Resilience to attacks upon the IDS itself

Firewall Versus Network IDS

• Firewall
– Active filtering
– Fail-close
• Network IDS
– Passive monitoring
– Fail-open

Suthoju Girija Rani, Assistant Professor, CSE.