Chapter 5

Architecture and Implementation

Based on the place where data source are collected and analyzed, the IDS can be
classified into centralized, distributed and agent based. In this Chapter, we discuss
each category in terms of its architecture and implementation.

5.1 Centralized

The first generation of IDSs is generally implemented inside the mainframe com-
puter systems that they monitor and protect. These host-based IDSs run on the target
system in order to monitor and analyze the operating system and host activities and
to detect malicious activities. Due to the overhead caused by IDSs on the target
system, the next generation of IDSs is proposed in which the intrusion monitoring,
analysis and detection are moved from the target system to a separate system. Most
of current IDSs are centralized systems. With a centralized architecture, all of the
monitoring, detection, and response activities are controlled directly by a central
console. Figure 5.1 illustrates a generic centralized IDS architecture.

5.2 Distributed

Different with the centralized IDS architecture , the partially distributed (i.e. hierar-
chical) architecture is proposed so that data collection is implemented locally in each
subnet and is then reported to one or more central locations. Figure 5.2 illustrates
a typical hierarchical IDS architecture, in which a subnet IDS console collects re-
ports from local sensors and then sends reports to the higher level IDS console (e.g.,
enterprise-level IDS console). This higher level IDS consol might send all reported
information to another higher level IDS console that manages the detection and re-
sponse among a set of cooperating networks. Recent literatures have proposed some
prototypes and frameworks for hierarchical IDSs, and the major techniques applied

A.A. Ghorbani et al., Network Intrusion Detection and Prevention: Concepts and Techniques, 115
Advances in Information Security 47, DOI 10.1007/978-0-387-88771-5_5,
© Springer Science + Business Media, LLC 2010
116 5 Architecture and Implementation

Fig. 5.1 Centralized IDS Architecture

are based on the agent technology. Agent based approach is used for hierarchical
IDSs, they are also utilized for implementing fully distributed IDSs where data is
collected and analyzed at a number of locations which is directly proportional to the
number of monitored components [1]. Figure 5.3 shows a fully-distributed architec-
ture.

5.2.1 Intelligent Agents

Traditional approaches to intrusion detection are centralized and have two major
limitations [13], (1) existing commercial solutions to network intrusions cannot
cover all possible attacks on the network accurately (i.e., they drop packets, but
generate a huge number of false alarms) and (2) existing approaches are unable to
respond to attacks in a timely manner. As a result, a distributed intelligent agent-
based system is proposed to overcome these shortcomings of conventional systems.
Instead of applying an individual IDS to defend the network, agents offer a new
approach for the implementation of IDSs in which several independent and intelli-
gent processes cooperate in securing the network. Such an agent-based IDS frame-
work has many advantages, consisting of the distribution of the computation cost,
the reduction in the amount of information sent over the network, the platform inde-
5.2 Distributed 117

Fig. 5.2 Hierarchical IDS Architecture

Fig. 5.3 Distributed IDS Architecture

118 5 Architecture and Implementation

pendence, the asynchronous operation, and the ease of updating [6]. Some other
benefits using the agent-based approach are also mentioned in [18] and include
efficiency, fault tolerance, extensibility, scalability, and resilience to degradation.
Furthermore, the application of intelligent agents allows the complex IDS to be im-
plemented in a highly modular manner and provides a possibility for the IDS to do
an active defense instead of reporting intrusions passively.
In an agent-based system, the individual agents are designed to manage a particu-
lar task and work together to fulfill the requirements of the whole system. The main
drawbacks of agent systems include the overhead of a large number of processes
and the lack of viable research in understanding and addressing agents’ potential
security problems. In this section, we discuss some typical examples regarding the
agent-based intrusion detection.

5.2.1.1 Autonomous Agents for Intrusion Detection (AAFID)

AAFID is a distributed IDS developed by the Center for Education and Research
in Information Assurance and Security (CERIAS) at the Purdue University [17].
The agents in AAFID are organized in a hierarchical fashion for data collection and
analysis, and there are four components included in the system architecture, namely
agents, filters, transceivers, and monitors.
Filters provide a subscription-based service to agents with two main functions,
namely data selection and data abstraction. Each data source has only one filter that
can be subscribed by multiple agents. When an agent starts a subscription request to
a filter, it specifies which records it wants based on some criteria and then the filter
replies the request with records satisfying the criteria to the agent (i.e. function of
data selection). On the other hand, filters implement all the system-dependent mech-
anisms to obtain the data requested by the agents (i.e. function of data abstraction).
As a result, the same agent can be operated under different architectures by simply
connecting to the appropriate filter.
A transceiver in AAFID receives findings reported by agents. Agents do not com-
municate directly with each other in the AAFID architecture and their operations are
monitored by the transceivers on host entities. The transceiver has the ability to start,
stop or send configuration commands to agents and can also perform data reduction
on the data received from different agents.
The transceivers report their results to one or more monitors. Since monitors
have access to network-wide data, they are able to perform higher-level intrusion
detection and correlation that involves several hosts. Monitors can also be organized
in a hierarchical fashion so that one monitor may in turn report to the other higher-
level monitor. In case an monitor is down or fails to do operations, the transceiver
can send its report to more than one monitor, thus providing the redundancy and
resistance to the failure of one of the monitors.
The proposed AAFID system can be distributed over any number of hosts in a
network. Each host contains a large number of agents that monitor important events
occurring in the host. The monitor agent can be a very simple program to monitor a
5.2 Distributed 119

specific event, e.g. counting the number of telnet connections within last five min-
utes, or a complex software system, e.g. an instance of IDIOT [3] looking for a set
of local intrusion patterns.

5.2.1.2 Multi-agents System-based Network Security Management

Architecture

Boudaoud et al. apply Belief- Desire-Intention (BDI) agents for intrusion detection
and propose an architecture called MANSMA (Multi-Agents system-based Network
Security Management Architecture) consisting of two layers, namely the Manager
Layer and the Local Layer. The Manager Layer is used to manage the global security
of a large network; and the Local Layer is to manage the security of a domain. There
are three types of agents identified in the Manager Layer, namely Security Policy
Manager Agent (SPMA), Extranet Manager Agent (EMA), and Intranet Manager
Agent (IMA). The SPMA maintains the global security policy that is determined
by a human administrator. The EMA takes the control of IMAs and manages the
distributed Extranet. Each IMA manages the security of a local network and is able
to control specified agents. The security of a domain is managed in the Local Layer,
where three types of Local Agents (LAs) are defined including Extranet LA, In-
tranet LA, and Internet LA. The main functions of LAs contain monitoring specified
activities and sending report to the Manager Agents.
In [2], Boudaoud et al. also define three functions for each agent, namely Event
Filtering, Interaction and Deliberation. Event filtering function filters detected secu-
rity events according to the event class specified in the detection goal of the agent.
The detection goal for each agent determines a set of event classes to be observed.
Interaction function allows agents to communicate and exchange their analysis and
knowledge. Deliberation function determines the agent’s capability to built knowl-
edge and experience and to reason according to its mental attitudes. According to
Boudaoud et al., agents on each layer communicate and exchange knowledge and
analysis results for detecting intrusive activities. It is still unclear on the proposed
architecture regarding the protocol applied for supporting the analysis, communi-
cation, and cooperation among different agents. Even though it was claimed by the
authors that the BDI solution can be used to model a security management system
and corresponding agents have the deliberation function to built knowledge and ex-
perience in a rational way and to reason and extrapolate according to their mental
attitudes, more details about the system’s design is missed in the paper. Moreover,
the description about the implementation is very brief and the illustrated case study
is more like an adhoc detection instead of a systematic approach.

5.2.1.3 Hummingbird

In [4], Frincke et al. described a distributed IDS , called Hummingbird, in which a

set of Hummer agents are deployed on a single host or a set of hosts for detecting
120 5 Architecture and Implementation

intrusions. Hummers in the system communicate with each other through a manager,
a subordinate, and peer relationships. During the communication, managers transmit
commands to subordinates. Such commands include gather/stop for data gathering
or forward/stop for data forwarding. Peers send requests to other peers for gathering,
forwarding or receiving data, and other peers then decide whether to accept or reject
such requests.
The Hummingbird system allows a system administrator to monitor security
threats on multiple computers from one central console. The main objective of Hum-
mingbird is to gather data about possible security problems and then re-organize
them into a standard format. Different with most security tools, Hummingbird com-
piles data collected from multiple workstations on different networks through run-
ning a local hummer on each workstation. As a result, system administrators can
react more quickly to security threats.
The biggest advantage of Hummingbird system is that it can share data with other
sites and at the same time does not compromise the security and confidentiality of
the system. Moreover, individual hummer agents in the Hummingbird system can
be used to generate and distribute misuse reports, thus maximizing the system’s
modularity.
The architecture of Hummingbird system mainly consists of three parts, namely
Message Distribution Unit (MDU), Data Distribution Unit (DDU) and Data Col-
lection Unit (DCU). MDU communicates with other hummers; DDU decides which
data should be sent to other hummers; and DCU uses data collection modules to col-
lect data. The three components communicate with each other through a local hum-
mer network that is implemented based on sockets. Frincke et al. deployed the sys-
tem as a testbed for investigating and studying issues regarding data sharing among
different sites, such as the reliability of intrusion alerts and misuse data, the safety
of data sharing and the data collection associated with a quantifiable contribution to
intruder identification.

5.2.1.4 Multi-agent-based IDS

In [5], Hegazy et al. propose a multi-agent IDS where they classify agents into four
categories: (1) Simple Reflex Agents, connecting with networks and being able to
collect packets moving around, (2) Analysis Agents, requesting the buffer (i.e., logs)
from the Simple Reflex Agents (i.e. sniffing agents) and building a list of suspi-
cious packets, (3) Goal-based Agents, requesting the list of suspicious packets from
their complementary analysis agents for making an intrusion decision and taking
necessary actions, and (4) Utility-based Agents, mapping the percept states into a
set of numbers that measure how closely the goals are achieved. The simulation
results show that the system can detect the X-mass tree attack. Although Hegazy
et al. present a lot of advantages of multi-agent technology over traditional object-
oriented programming, some important advantages of multi-agent systems, e.g. co-
operation in a distributed domain and their intelligent and flexible behavior, have
5.2 Distributed 121

not been discussed in the paper. Moreover, the utility-based agents are missed to be
described in the paper.

5.2.1.5 Adaptive Hierarchical Agent-based Intrusion Detection System

An Adaptive Hierarchical Agent-based Intrusion Detection System , called AHA!

IDS, is proposed by Ragsdale et al. in [14], which is based on a fully distributed,
multi-agent framework, consisting of 4 major components: (1) Director Agents, be-
ing responsible for detecting intrusive behavior, (2) Surrogate Agents, taking and
covering the responsibilities of Director Agent when it fails in some cases, (3) Man-
ager Agents, being responsible for detecting intrusive activities on a subset of sys-
tems for which a Director is responsible, and (4) Tool Agents, employed by a Man-
ager agent to detect intrusive activity. Three types of detection adaption are provided
by the AHA!IDS framework, namely adjusting the amount of system resources de-
voted to intrusion detection according to perceived degree of threat; invoking dy-
namically new combinations of low-level detection agents in response to changing
circumstances; and, adjusting the confidence metric associated to the low-level Tool
agents.

5.2.1.6 Fuzzy Adaptive Survivability Tools (FAST)

In [15, 16], Shajari and Ghorbani proposed an intelligent multi-agent based intru-
sion detection system, called Fuzzy Adaptive Survivability Tools (FAST) , in order
to protect a network against the large-scale intrusions. The FAST system is based on
an automated detection model and a response approach for survivability, in which
different intelligent agents are used to identify normal and abnormal patterns auto-
matically and adaptively. The anomalous network variables are identified and then
are used for detecting the threat degree of known attacks and events of interest.
Moreover, the FAST system is able to make decisions about events that meet the
predefined criteria and site-specific policies. In FAST, fuzzy logic is used to identify
the degree of suspicion of each attack and to deal with the uncertainties of response.
There are four different types of agents implemented in the FAST, namely HCI,
Monitor, Detection, and Decision agents. The HCI agent provides an appropriate
user interface for the control of a human operator. The Monitor agent identifies and
detects anomalous network activities represented by different variables. The Detec-
tion agent inspects each flow in order to find the attack sign appeared on the network.
The Decision agent, upon receiving an attack alert will initiate a task that involves
selecting and executing a predefined plan that is both relevant and applicable to the
event in the current context.
A general implementation architecture for the FAST system is illustrated in Fig-
ure 5.4. There are three basic components of the system, namely Sensor (S-box),
Manager (MS-box) and Console Software. MS-box consists of a management box
(M-box) along with its dedicated sensor.
122 5 Architecture and Implementation

Local Alerts and

rd Global Response
3 Party Device Responses to
from Higher M
Higher M

Network Status, Graphic

Local Alerts and
Features, Local Alerts and
Response View Coord inates
Responses
Network
Data
Sensor Box (S-Box) Manager Box (M-Box) Console Software

Global Response,
Configuration Read
Configuration Updates
and Update

Local Response Local Alerts from

To Lower Ms Lower Ms

Configuration
Read and Update

Local Alerts and

Responses

Fig. 5.4 Overall Design of the FAST System

A sensor operates at the lowest level of the system hierarchy. The sensor is re-
sponsible for monitoring network traffic, detecting malicious activities and provid-
ing appropriate local responses. Each sensor has two interfaces to the local network.
The first interface connects to the mirror port of the local switch and is used for
collecting network data in promiscuous mode. The second interface connects to a
regular network port and is used for communication with a higher level MS-box,
console, and third party devices.
The management module is responsible for managing S-boxes and MS-boxes at
a lower level of hierarchy in the network. This module includes an S-box module
within itself. Therefore, MS-box is capable of performing jobs related to the S-box
as well as high level management of the sensors at lower level.
The management tasks of this box include correlation of the data received from
the sensors as well as planning a global response according to the current condition
of the network. Each MS-box will communicate with other MS-boxes at a higher
hierarchical level of the network. The management modules at the higher level will
control and respond to their corresponding lower level management modules in ad-
dition to the sensors that are connected to them. Management modules monitor the
network and they apply a global response to their lower level sensors/managers
considering the correlated information received from them and the global response
received from the higher level management.
The console is the user interface for the FAST. The console software will run on a
separate computer and is designed to communicate with all management and sensor
modules in the network. Using the console software, the system administrator can
update the configuration of S- and MS-boxes. The console software also displays
different graphical and text information regarding the attacks and current status of
the network.
5.2 Distributed 123

5.2.2 Mobile Agents

Mobile agents have been commonly used to design and implement distributed appli-
cations in a dynamic environment. Recent reports show that they can be applied for
the purpose of intrusion detection. Typical examples include [9] and [10]. In reality,
it is not an easy task to achieve an effective detection for network intrusions when
applying mobile agents. In [19], Vigna discusses the reasons why mobile agents
have not been well received by the current intrusion detection research community.
It was claimed in the paper that mobile agents are: (1) expensive, (2) difficult to
develop, test and debug, (3) difficult to authenticate and control, and (4) vulnerable
to a number of attacks coming from malicious executing environments. Moreover,
they do not have a ubiquitous infrastructure and a shared language/ontology, and
might be easily exploited by worm attacks. In this section, we introduce two typical
mobile-agent based IDSs.

5.2.2.1 Intrusion Detection Agent system (IDA)

In [11], Asaka et al. propose and implement an intrusion detection prototype system
based on mobile agents, called Intrusion Detection Agent system (IDA) . The agents
in IDA collect the information related to the intrusion along the intrusion-route and
then apply them to make decisions about whether an intrusion has occurred. The
IDA system provides a set of functions that enable efficient information retrieval
and also make it possible to detect compromised intermediate hosts.
Instead of detecting precisely all intrusions, the IDA system focuses on detecting
intrusions efficiently. Therefore, the IDA monitors and inspects events that are only
related to intrusions and these suspicious events are called Marks Left by Suspected
Intruder (MLSI). When an MLSI gets detected, the IDA will collect and analyze
corresponding information, and then make an intrusion decision. One advantage to
use mobile agents in IDA is that they can autonomously migrate to target systems
to collect information related to intrusions, thus eliminating the need to transfer
system logs to the server. In practice, the IDA system is composed of 6 compo-
nents, namely manager, sensors, bulletin boards, message boards, tracing agents,
and information-gathering agents. The IDA manager resides on each network seg-
ment. The sensors are deployed on each target system and are used to monitor sys-
tem logs for inspecting MLSIs. Once a MLSI is found by a sensor, it will be reported
to the manager agent. The intrusion-route tracing agent traces the path of an intru-
sion and identifies the original point of a real attacker. During the intrusion route
tracing period any intermediate node that is compromised can be detected.
In the IDA system, the manager, sensors, and tracing agents cooperate in an ef-
fective way. In particular, the sensor detects an MLSI and reports it to the manager
and then the manager launches a tracing agent to the target system. The tracing
agent migrates autonomously from machine to machine and traces the intrusion in-
dependently without the involvement of the manager. Although a tracing agent can
124 5 Architecture and Implementation

migrate to any system in which IDA is installed, it cannot make any judgment about
intrusions.
The mobile information-gathering agent in IDA collects information related to
MLSIs from a target system. Each time a tracing agent is launched into a target
system, it activates an information-gathering agent deployed in the system. Next,
the information gathering agent collects information according to the MLSI type and
then reports the results to the manager. Same with the tracing agent, the information
gathering agent is not able to decide whether an intrusion has occurred.
The bulletin board and the message board are built for the purpose of informa-
tion exchange between the tracing agents and information-gathering agents. The
message board is deployed on each target system and is used by tracing agents for
exchanging information. The bulletin board is installed on the manager’s computer
and is used not only for recording information collected by information-gathering
agents on target systems, but also for integrating the information gathered about
tracing routes. In IDA, the final intrusion decision is made by the manager through
analyzing information gathered by the information-gathering agents. The manager
agent has an interface between administrators and the system, in which it manages
the mobile agents and bulletin boards, and accumulates and weighs the information
entered by the mobile agents on the bulletin board. In case the weights exceed a
predefined threshold, an intrusion alert will be reported.

5.2.2.2 Mobile Agents for Intrusion Detection (MAIDS)

In [7], Helmer et al. propose and implement an intrusion detection system based
on distributed intelligent mobile agents. They called the system MAIDS in which
data mining techniques are performed to provide global and temporal views for the
entire network system. The so-called lightweight agents in MAIDS can complete
their essential tasks with minimal code (cost) and can be updated or upgraded dy-
namically due to their smaller size. In MAIDS, data gathering agents parse system
logs and activity data into a common format. The low level agents classify recent
activities and then send data and current classification states to other peers or higher
level agents. The higher level agents implement data mining based on the entire
knowledge base and data sources on the system. Distributed data cleaning agents
process data collected from log files, networks, and system activities. The mobile
agents are above this layer. They focus on system calls, TCP connections, and lo-
gins. These lower-level agents form a rough opinion of intrusions and can travel to
each of their associated data cleaning agents, gather recent information, and clas-
sify the data to determine whether suspicious activity is occurring. Above the layer
of mobile agents is a mediation component that is connected to database and data
mining and data fusion agents. Intelligent agents on this level maintain the data
warehouse by combining knowledge and data from the lower layer of agents and
then applying data mining algorithms to them. As a result, they offer a potential to
discover associations among suspicious events that occur together with some fre-
quency and attack patterns as well. An interface agent is located on top of the whole
 5.3 Cooperative Intrusion Detection 125

structure and is an analysis console in which system status reported from the low
level agents are shown.

5.3 Cooperative Intrusion Detection

Distributed IDSs rely on information exchange and sharing among different sites
where there is not a common administrator. As a result, an effective cooperation is
required in order to detect and respond to security incidents. In [12], McConnell et
al. propose some basic principles of information sharing among networks without
any central administration. Such an information sharing is very helpful to identify
and counter large-scale Internet attacks. Below are brief descriptions about these
basic principles:

1. Local control over policy decisions by each cooperating network is important

since most likely the sites do not trust each other.
2. The local network collects the information to be used in identifying the policy
violations and decides whether to provide the information to other parties or not.
3. The data authenticity and integrity on different domains must be proved since the
source can be compromised and may submit misleading data.
4. Hosts and networks that identify some policy violations are not responsible for
the policy enforcement. Instead, the policy enforcement remains a local decision.
5. An authentication mechanism is necessary among cooperating networks.
6. A hierarchical architecture of cooperative entities is necessary. The high level
manager has higher authority than low level subordinates. Information sharing
on the hierarchy can be vertical (i.e. a manager in high level and its subordinate
in low level) or horizontal (i.e. between two subordinates).
7. Data collection should be redundant so that the system can keep operating when
one data collector is compromised or becomes unavailable.
8. Data reduction is necessary when collecting data in order to avoid high volume
data exchange among cooperating partners.
9. Data sanitization is required to avoid possible security risks to the transmitting
network caused by the sharing of critical host and network specific attributes.
10. In a cooperating framework with a large number of cooperating networks, the
data volume is huge and as a result, the data visualization tools are required for
the human system/network administrator to analyze the data in a timely manner.

In [8], Huang et al. address the problem of large-scale distributed intrusion detec-
tion and propose a cooperative IDS based on the goal-tree representation of attack
strategies. A high level information exchange and work coordination mechanism is
implemented among different IDSs. The basic idea behind the system is that the co-
operation of remote IDSs is driven by a common goal for identifying an intruder’s
attack strategies and intentions. Intrusion intentions are seen as high-level platform-
independent attack strategies and the IDSs can recognize attacks at the strategic level
through the intention analysis. In the paper, the goal-tree is presented to formalize
126 5 Architecture and Implementation

intrusion intentions, in which the root node represents the final target of an intrusion
and the lower level nodes represent ordered sub-goals. Such as augmented goal-tree
includes three basic constructs namely OR, AND, and ordered AND. Global agent
performs intention analysis and local IDSs search for more evidences that match the
sub-goals of the intention and trends predicated by a global agent. Although the idea
in the paper is interesting the authors do not provide any formalization, architecture,
implementation or evaluation to validate their proposal. Moreover, some important
issues such as the trust relationship among remote IDSs and the architecture and
location of the global IDS agent, are missed in the paper.
The cooperative strategy for intrusion detection is also used to discover the con-
nection chain of an intrusion [20]. In practice, network based attackers seldom attack
directly from their own computers. Instead, they prefer to start the attacks through
a set of intermediate stepping stones to hide their identity and origin. Therefore, in
order to identify real attackers behind stepping stones, IDSs must be able to trace
through the stepping stones and reconstruct the correct intrusion connection chain
[20].
In the cooperative IDS, the secure communication protocol between agents is
very important since the whole IDS might be simply broken down by a compro-
mised agent. In [21], Xue et al. propose a multi-agent system for distributed in-
trusion detection in which different types of agents communicate with each other
through a public-key encryption algorithm used to encrypt all the communication
and passwords in the system.

References

1. S. Axelsson, Intrusion detection systems: A survey and taxonomy, Tech. Report 99-15,
Chalmers University of Technology, Department of Computer Engineering, 2000.
2. Karima Boudaoud and Zahia Guessoum, A multi-agents system for network security manage-
ment, Telecommunication Network Intelligence, IFIP TC6 WG6.7 Sixth International Confer-
ence on Intelligence in Networks (SMARTNET 2000) (Vienna, Austria) (Harmen R. van As,
ed.), IFIP Conference Proceedings, vol. 178, Kluwer, September 2000, pp. 172–189.
3. M. Crosbie, B. Dole, T. Ellis, I. Krsul, and E. Spafford, Idiot - users guide, technical report,
Tech. Report TR-96-050, Purdue University, COAST Laboratory, September 1996.
4. D. Frincke, D. Tobin, J. McConnell, J. Marconi, and D. Polla, A framework for cooperative in-
trusion detection, Proceedings of the 21st National Information Systems Security Conference
(Arlington, VA), October 1998, pp. 361–373.
5. I.M. Hegazy, T. Al-Arif, Z.T. Fayed, and H.M. Faheem, A multi-agent based system for intru-
sion detection, IEEE Potentials 22 (2003), no. 4, 28–31.
6. G. Helmer, Intelligent multi-agent system for intrusion detection and countermeasures, Ph.D.
thesis, Iowa State University, Computer Science Department, Ames, IA, 2000.
7. G. Helmer, J.S.K. Wong, V. Honavar, L. Miller, and Y. Wang, Lightweight agents for intrusion
detection, The Journal of Systems & Software 67 (2003), no. 2, 109–122.
8. M.Y. Huang, R.J. Jasper, and T.M. Wicks, Large scale distributed intrusion detection frame-
work based on attack strategy analysis, COMPUT. NETWORKS 31 (1999), no. 23, 2465–
2475.