Implementation Analysis of Different Learning Models for Intrusion

Detection
Dissertation submitted in fulfillment of the requirements for the Degree of
BACHELOR OF TECHNOLOGY
in
COMPUTER SCIENCE AND ENGINEERING
By
SRI RAMA MANIKYA RAJESH KORUPROLU
DITHIN VIJAY KUMAR REDDY SATHI
DWARAMPUDI MANIKANTA SURESH KRISHNA REDDY
VENKATA SAI KISHOR PILLI
SHAIK MOHAMMAD YASAR

School of Computer Science and Engineering

Lovely Professional University
Phagwara, Punjab (India)
1. Abstract

The rapid evolution of cyber threats necessitates robust and adaptive Intrusion Detection
Systems (IDS) capable of identifying and mitigating both known and emerging attacks.
This project aims to develop a hybrid IDS leveraging the strengths of machine learning
techniques to enhance detection accuracy, reduce false positive rates, and address the
challenges posed by zero-day attacks. The hybrid model integrates autoencoders for
anomaly detection and decision trees for traffic classification, combining unsupervised
and supervised learning approaches.

The project utilizes two datasets: CICIDS-2017, representing modern, realistic network
traffic with diverse attack scenarios, and NSL-KDD, a benchmark dataset offering a
simplified yet comprehensive view of traditional network behaviors. Data preprocessing
steps included feature encoding, scaling, and addressing class imbalances to ensure
compatibility with machine learning models.

Key methodologies involve training baseline models such as Random Forest, SVM, and
Neural Networks, alongside the proposed hybrid model. Performance was evaluated
using metrics like accuracy, precision, recall, and F1-score, with results indicating the
hybrid model’s superiority in detecting anomalies and minimizing false positives.

Findings from this study highlight the importance of dataset selection in IDS
development, showcasing how CICIDS-2017's complexity and NSL-KDD's simplicity
complement each other. The conclusions underscore the hybrid model’s potential for real-
world deployment, offering a scalable, adaptable, and accurate solution for evolving
cybersecurity challenges.
2. Introduction
The exponential growth of digital networks has brought increased vulnerabilities, making
Intrusion Detection Systems (IDS) a critical component of cybersecurity. IDS monitors
and analyzes network traffic to detect unauthorized activities, threats, or policy
violations, ensuring the safety and integrity of sensitive data and systems.
2.1 Overview of Intrusion Detection Systems (IDS)
Intrusion Detection Systems (IDS) are critical in safeguarding digital networks against
unauthorized access, data breaches, and malicious attacks. They operate by monitoring,
analyzing, and responding to suspicious activities across a network or within a specific
system. IDS can be broadly classified into:
 Network-based IDS (NIDS): These systems monitor traffic across the entire
network, examining data packets for anomalies or malicious patterns.
 Host-based IDS (HIDS): These systems focus on monitoring individual devices,
analyzing activities like file changes, log entries, or system calls.
Both types aim to detect threats early and mitigate their impact, making them
indispensable for cybersecurity in environments ranging from small organizations to
large enterprises.

2.2 Signature-Based vs. Anomaly-Based Detection

Intrusion detection methodologies are primarily divided into two categories:
2.2.1 Signature-Based Detection
Signature-based IDS identifies threats by matching observed patterns with a predefined
database of known attack signatures. It operates on the principle that malicious activities
often leave unique traces or "signatures."
Advantages:
 High detection accuracy for previously identified threats.
 Low false positive rates due to precise pattern matching.
 Simplicity in implementation and understanding.
Disadvantages:
 Limited to known attacks; cannot detect novel or zero-day threats.
 Requires frequent updates to the signature database, leading to maintenance
overhead.
Examples: Snort and Suricata, which are widely used in industry for their reliability in
detecting well-documented attacks.
2.2.2 Anomaly-Based Detection
Anomaly-based IDS identifies threats by monitoring deviations from established normal
behavior. It uses statistical models, heuristics, or machine learning to create a baseline of
normal network traffic. Any deviation from this baseline is flagged as suspicious.
Advantages:
 Capable of detecting unknown threats, including zero-day attacks.
 Adaptability to evolving attack patterns.
Disadvantages:
 Higher false positive rates due to the challenge of accurately defining "normal"
behavior.
 Computationally intensive, requiring advanced algorithms and resources.
Examples: Systems that use machine learning models to detect anomalies in real-time
network traffic.
2.3 Challenges in Current IDS Systems and the Role of Machine Learning
Despite their importance, traditional IDS approaches face significant challenges:
Detection of Unknown Threats:
 Signature-based systems fail to recognize new attack types without corresponding
signatures.
 Anomaly-based systems struggle to differentiate between benign anomalies and
genuine threats.
High False Positive Rates:
 Legitimate variations in network behavior often trigger false alarms in anomaly-
based systems.
 This undermines user trust and leads to alert fatigue among security analysts.
Scalability Issues:
 The growing volume and complexity of network data challenge the scalability of
traditional IDS.
 Handling high-dimensional data effectively requires advanced computational
methods.
Adaptability:
  Cyber threats constantly evolve, rendering static detection systems obsolete.
 IDS must dynamically learn and adapt to new patterns in traffic and attack
methods.
Role of Machine Learning in IDS Development:
 Machine learning and deep learning have emerged as transformative technologies
for overcoming these limitations.
 Pattern Recognition: Algorithms like Random Forest and Neural Networks can
identify complex patterns in network traffic.
Adaptability: Models continuously learn from historical data to adapt to new attack
scenarios.
Hybrid Approaches: Combining unsupervised methods (e.g., autoencoders) with
supervised classification (e.g., decision trees) enhances detection accuracy while
minimizing false positives.
2.4 Importance of Selecting Robust Datasets for Effective IDS Development
The efficacy of an IDS largely depends on the dataset used for its development and
evaluation. A robust dataset ensures the model is trained on diverse, real-world scenarios,
enabling it to generalize effectively.
Characteristics of a Good Dataset:
 Diversity of Attack Types: The dataset should include various attack scenarios,
such as denial-of-service (DoS), brute force, phishing, and web attacks.
 Realism: It should reflect realistic network traffic patterns for practical relevance.
 Balanced Representation: Equal emphasis on normal and malicious traffic
prevents model bias.
 Labeled Data: Properly annotated datasets simplify training and evaluation.
2.5 Datasets Used in This Project:
CICIDS-2017:
 A modern dataset simulating realistic network traffic, including a wide range of
attack scenarios.
 Suitable for testing IDS on real-world, dynamic environments.
NSL-KDD:
 An improved version of the KDD Cup 1999 dataset, addressing redundancy and
skewed class distributions.
 Useful as a benchmark for foundational IDS algorithms due to its simplicity and
ease of preprocessing.
Benefits of Using Both Datasets:
 CICIDS-2017 helps evaluate the IDS's ability to handle diverse, complex, and
modern threats.
 NSL-KDD provides a controlled setup for benchmarking and testing the system's
basic detection capabilities.
 Combining insights from both datasets enables comprehensive model evaluation,
ensuring scalability, adaptability, and effectiveness in real-world applications.
2.6 Objective of the Project
The main purpose of this research is to develop and create a hybrid Intrusion Detection
System (with the combination of autoencoders with ensemble decision tree ) for detection
of measurable network based attacks which include known and unknown classes. Project
aims are:
The goal of this project is to develop a Hybrid Intrusion Detection System (IDS) that
combines Autoencoders for anomaly detection and Decision Trees for traffic
classification. This hybrid approach leverages the strengths of both methods, enhancing
the overall performance and effectiveness of the IDS in detecting network intrusions,
especially in complex and evolving threat landscapes.
2.6.1 Autoencoders for Anomaly Detection:
Autoencoders are a type of unsupervised neural network that learns to compress (encode)
and reconstruct (decode) the input data. In the context of IDS, autoencoders will be
trained to learn the normal patterns of network traffic and detect anomalies (i.e.,
deviations from normal behavior). This approach is particularly useful for detecting zero-
day attacks, which may not be part of any known attack signature.
 Encoder: The encoder part of the autoencoder network will reduce the
dimensionality of the input features (network traffic data), creating a compressed
representation that captures the essential patterns of normal traffic behavior.
 Decoder: The decoder will attempt to reconstruct the original input data from this
compressed representation. During training, the autoencoder learns to minimize
the difference between the input and the reconstructed output (often using mean
squared error as a loss function).
 Anomaly Detection: After training, any significant deviation between the original
input and the reconstructed output can be flagged as an anomaly, suggesting
potential intrusions. The larger the reconstruction error, the more likely the data
point is anomalous.
Advantages:
 Effective at identifying previously unseen or zero-day attacks.
  No need for labeled data during training, making it suitable for real-time threat
detection.
2.6.2 Decision Trees for Traffic Classification:
Decision Trees are a type of supervised learning model used for classification tasks. In
this hybrid IDS model, decision trees will be trained to classify network traffic as either
benign or malicious based on features extracted from the data. Unlike autoencoders,
decision trees require labeled data for training.
 Classification: Decision trees will use features such as packet size, time intervals,
flags, source and destination IP addresses, etc., to classify traffic into different
categories (benign or malicious).
 Tree Structure: The decision tree is built by recursively splitting the dataset based
on feature values, optimizing the splits based on a measure like Gini impurity or
Information Gain.
 Interpretability: Decision trees are easy to interpret, making them useful for
explaining why a particular traffic pattern was classified as an attack or benign.
Advantages:
 Clear decision-making process, making the model interpretable.
 Can handle both categorical and continuous features.
2.6.3 Integration of Autoencoder and Decision Tree:
The core of the hybrid IDS lies in the integration of these two models:
 Anomaly Detection + Classification: First, the autoencoder is used to detect
anomalies in the network traffic, identifying potential threats that deviate from
normal patterns. The decision tree classifier is then used to verify whether these
anomalies are truly malicious or benign by analyzing additional features of the
network traffic.
 Hybrid Decision Logic: The hybrid system combines the results from both
models. For example, an anomaly detected by the autoencoder could be passed to
the decision tree for final classification. If the decision tree classifies the traffic as
malicious, an alert is raised. If the anomaly is deemed benign, the traffic is
allowed through without raising an alert.
This two-step approach significantly reduces false positives and increases the robustness
of the IDS system by ensuring that both new and known attack patterns are detected
effectively.
2.6.4 System Workflow:
  Step 1: Data Preprocessing: The raw network traffic data from the datasets
(CICIDS-2017 and NSL-KDD) will undergo preprocessing, including
normalization, scaling, and encoding.
 Step 2: Anomaly Detection (Autoencoder): The preprocessed data will first be
passed through the autoencoder to identify anomalous patterns.
 Step 3: Classification (Decision Tree): The anomalous data points will be
classified as either benign or malicious using the decision tree model.
 Step 4: Evaluation: The performance of the hybrid IDS will be evaluated using
metrics such as accuracy, precision, recall, and F1-score.
2.6.5 Advantages of the Hybrid Model:
 Adaptability: The hybrid model can adapt to both known and unknown threats by
combining the strengths of both anomaly detection (for zero-day threats) and
classification (for known attack patterns).
 Reduced False Positives: The combination of autoencoders and decision trees
helps reduce the occurrence of false alarms, which are a common issue with
standalone anomaly detection systems.
 Improved Detection Rates: By using the best of both worlds, the hybrid model
improves the accuracy of intrusion detection, especially in complex environments
where both known and unknown threats coexist.
2.7 Reduce False Positives
One of the primary challenges in Intrusion Detection Systems (IDS) is minimizing false
positives—legitimate network traffic incorrectly classified as malicious. False positives
lead to unnecessary alerts, system downtimes, and wasted resources, which can
significantly impair the efficiency and credibility of an IDS.
In this project, the hybrid IDS model developed with autoencoders and decision trees is
designed with a specific focus on reducing false positives while maintaining a high
detection rate for intrusions. Here’s how the hybrid model addresses this challenge:
2.7.1 Autoencoders for Anomaly Detection:
 Role in False Positive Reduction: Autoencoders are trained to recognize patterns
of normal network behavior. Once trained, they can identify outliers or anomalies
that deviate significantly from the normal patterns. Since the autoencoder is
unsupervised, it doesn't require labeled data, making it ideal for detecting
previously unknown threats.
 Minimizing False Positives:
Reconstruction Error Thresholding: Autoencoders detect anomalies based on the
reconstruction error—the difference between the input and the reconstructed data.
By setting a sensitivity threshold for reconstruction error, the system can filter out
 benign data with low reconstruction errors, which helps in preventing false
alarms.
 Anomaly Scoring: Instead of labeling every anomaly as an attack, the
reconstruction error can be used to score the severity of the anomaly. Low scoring
anomalies (benign) are ignored, whereas high scoring anomalies (indicating
potential threats) are further processed by the decision tree classifier.
2.7.2 Decision Trees for Classification:
 Role in False Positive Reduction: Decision trees add a layer of classification to
the detection process. Once an anomaly is detected by the autoencoder, the
decision tree classifies the anomaly as either benign or malicious based on
additional features like source IP, packet size, protocol type, etc. This two-tier
approach ensures that not every anomaly detected by the autoencoder is flagged
as an attack.

 Minimizing False Positives:

o Feature Selection: Decision trees inherently perform feature selection
during training, which means they will prioritize the most relevant features
for classification. This helps reduce false positives by focusing on the
characteristics that are truly indicative of malicious activity.
o Pruning: Decision trees are prone to overfitting, especially when trained
on noisy data. To counter this, pruning techniques are used to trim
unnecessary branches of the tree, ensuring that the decision-making
process is based on the most important factors, further reducing the
likelihood of misclassification.
2.7.3 Hybrid System Design to Minimize False Positives:
 Combining Anomaly Detection and Classification: The hybrid model’s use of
both anomaly detection (autoencoder) and traffic classification (decision tree)
provides a more reliable detection mechanism compared to using either model
independently. If an anomaly is detected by the autoencoder but classified as
benign by the decision tree, it will not raise an alert. This two-stage filtering
process significantly reduces false positives because the system needs to detect an
anomaly and classify it as malicious before triggering an alert.
 Thresholding and Decision Fusion: The combination of the two models allows for
more precise control over false positives. For example:
o Autoencoder Threshold: If the reconstruction error is below a certain
threshold, the data is not flagged as an anomaly, reducing unnecessary
checks by the decision tree.
 o Decision Tree Confidence: Decision trees can provide a confidence score
for classifications. If the classification confidence is below a set threshold,
the system can opt not to issue an alert, further minimizing the chance of a
false positive.
 Feedback Loop for Adaptability: As the hybrid IDS system is exposed to more
data, it can adapt by adjusting thresholds and model parameters, gradually
improving its performance in terms of reducing false positives while maintaining
high detection rates.

2.7.4 Evaluation and Metrics:

 Precision and Recall: The key performance metrics to assess false positives
include precision (the proportion of true positive alerts out of all alerts raised) and
false positive rate (FPR). A low FPR directly correlates with fewer false positives.
 F1-Score: The hybrid IDS will be evaluated using the F1-score, which balances
precision and recall. A high F1-score indicates that the system has an effective
balance between detecting intrusions and minimizing false alarms.
 Comparison with Baseline Models: By comparing the hybrid model's false
positive rate against traditional machine learning models like Random Forest
(RF), Support Vector Machines (SVM), and Neural Networks, the effectiveness of
the hybrid approach in reducing false positives will be clearly demonstrated.
2.7.5 Real-Time Monitoring and Alerts:
 Dynamic Adjustment: In real-time deployment scenarios, the hybrid IDS can
dynamically adjust detection thresholds based on the volume and nature of
network traffic. During periods of high traffic, the system can become more
conservative, reducing the sensitivity to avoid unnecessary alerts (thus reducing
false positives). Conversely, during lower traffic periods or when new attack
patterns are detected, the system can become more aggressive in detecting
anomalies.
 Alert Prioritization: False positives can overwhelm security teams and lead to
alert fatigue. To address this, the hybrid IDS will prioritize alerts based on the
severity of the detected anomalies and classification confidence, ensuring that the
most likely threats are flagged for immediate investigation.

2.8 Adapt to Zero-Day Threats
Zero-day threats are security vulnerabilities that are unknown to the public or the
software vendor. These threats are often exploited by attackers before the vulnerability is
identified and patched, making them particularly dangerous and difficult to detect using
traditional signature-based Intrusion Detection Systems (IDS).

The hybrid IDS model developed with autoencoders and decision trees is specifically
designed to adapt to zero-day threats, combining the power of anomaly detection with
traffic classification. Here's how the hybrid IDS can effectively handle zero-day threats:

2.8.1 Anomaly Detection with Autoencoders:

Autoencoders, as unsupervised learning models, are ideal for detecting zero-day threats
because they do not rely on pre-defined attack signatures. Instead, they learn to recognize
normal behavior patterns in network traffic, which allows them to identify deviations that
could indicate potential zero-day attacks.
 Unsupervised Learning: Since zero-day attacks are new and do not have known
signatures, autoencoders do not require labeled data for training. This
unsupervised learning capability is crucial for detecting attacks that have never
been seen before.
 Normal Behavior Modeling: During the training phase, the autoencoder learns the
normal traffic distribution by reconstructing network packets, flow data, or other
features of the network. When an attack occurs, the traffic will deviate from the
normal behavior, causing the reconstruction error to increase. These anomalies in
the reconstructed output are flagged as suspicious, potentially indicating a zero-
day attack.
 Dynamic Thresholding for Anomaly Scoring: Autoencoders can be designed with
dynamic thresholding based on the volume and variability of traffic. If a sudden,
unexpected pattern arises (which could be a zero-day attack), the autoencoder will
flag it as anomalous, enabling the system to quickly identify previously unknown
attack vectors.

2.8.2 Feature Diversity with Decision Trees:

Decision trees, as a supervised learning approach, help classify the detected anomalies by
evaluating multiple features (e.g., packet size, source/destination IPs, protocol type). For
zero-day threats, decision trees use features that are independent of attack signatures to
make classifications.
 Adaptable to New Features: While decision trees rely on labeled data for training,
they can handle a variety of features, including traffic metadata (e.g., source IP,
 destination port, protocol type). This allows the decision tree to adapt to new
attack patterns that may be recognized by anomalous patterns but not yet by
known signatures.
 Classification of Anomalies: When the autoencoder detects an anomaly, the
decision tree steps in to classify whether that anomaly represents a legitimate
zero-day threat or is benign. The decision tree can be retrained over time with new
data to improve its ability to classify increasingly sophisticated zero-day attacks.

2.8.3 Two-Stage Detection Process:

The hybrid system’s two-stage detection process is especially useful for handling zero-
day threats:
 Anomaly Detection (Autoencoder): The system detects potential zero-day threats
by identifying deviations from normal behavior in the network traffic. Zero-day
attacks, being novel, will cause reconstruction errors in the autoencoder since the
traffic doesn’t match any known patterns.
 Classification (Decision Tree): The anomalous data is then passed to the decision
tree, which classifies it based on a set of relevant features. The tree helps in
verifying whether the anomaly is benign or malicious, based on its learned
patterns. If the anomaly is classified as malicious, the system raises an alert for
potential zero-day activity.
This dual approach ensures that even if an attack is not part of the signature database, it is
still detected based on its anomalous behavior.

2.8.4 Zero-Day Attack Detection Strategy:

 Early Detection: The hybrid model can detect zero-day attacks in real-time by
leveraging anomaly scores and classification confidence. By detecting abnormal
traffic patterns early, it can issue alerts even before the signature-based systems
have been updated.
 Behavioral Analysis: Zero-day attacks are often identified by their behavioral
traits rather than by specific signatures. The autoencoder identifies these
behavioral anomalies, while the decision tree helps categorize them into relevant
classes based on their severity.
 Adaptive Learning: Zero-day threats evolve over time. The hybrid system can be
designed to continuously learn from new data and anomalies, allowing it to adapt
to new attack techniques as they emerge. For instance, as new types of zero-day
attacks are identified, the decision tree can be retrained with updated features, and
 the autoencoder can adapt its anomaly thresholds to accommodate new patterns of
normal traffic behavior.
2.8.5 Evaluation of Zero-Day Threat Detection:
 Test with Novel Datasets: To evaluate the ability of the hybrid IDS to detect zero-
day threats, it should be tested on novel datasets that simulate new and unseen
attack scenarios. The CICIDS-2017 dataset contains many modern attack
scenarios, including zero-day attacks, while the NSL-KDD dataset, though older,
can also be used for benchmarking.
 Performance Metrics for Zero-Day Detection:
o True Positive Rate (TPR) or Recall: Measures how effectively the hybrid IDS
detects actual zero-day attacks.
o False Positive Rate (FPR): Evaluates how many legitimate traffic samples are
incorrectly flagged as zero-day attacks.
o F1-Score: Balances precision and recall, providing an overall measure of the
model's ability to detect zero-day threats without generating excessive false
alarms.
2.8.6 Continuous Adaptation and Feedback Loop:
 Incremental Learning: Zero-day threats often evolve over time, meaning a model
trained on old data may miss new attack patterns. To combat this, the hybrid IDS
can be set up to incrementally learn from newly labeled data or new traffic
patterns, ensuring the system can adapt as attackers change their tactics.

 Human-in-the-Loop: As part of the feedback loop, cybersecurity experts can

manually review flagged zero-day attacks and provide feedback. This feedback
can then be used to retrain the model, allowing it to adapt more quickly to new
attack techniques.
2.9 Evaluate the Hybrid System’s Performance
Evaluating the performance of the hybrid Intrusion Detection System (IDS) that
combines autoencoders for anomaly detection and decision trees for traffic classification
is essential to assess its effectiveness in detecting both known and unknown threats,
particularly zero-day attacks. The evaluation should focus on the model’s ability to detect
anomalies accurately, minimize false positives, and perform well on the datasets used
(CICIDS-2017 and NSL-KDD). Here’s a structured approach to evaluating the
performance of the hybrid IDS:
2.9.1 Evaluation Criteria and Performance Metrics
To comprehensively evaluate the hybrid IDS, several performance metrics are necessary
to gauge its effectiveness in different aspects of intrusion detection.
2.9.1.1 Accuracy:
o Definition: Accuracy measures the proportion of correctly classified instances
(both true positives and true negatives) out of the total number of instances.
o Formula:
Accuracy = True Positives (TP)
+True Negatives (TN)/Total Instances (TP + TN + FP + FN)
o Significance: This metric provides an overall measure of how well the hybrid IDS
performs but may be misleading if the dataset is imbalanced.
2.9.1.2 Precision:
o Definition: Precision measures the proportion of true positives among all
instances that the model labeled as positive (i.e., the proportion of correctly
identified attacks among all detected attacks).
o Formula:
Precision = True Positives (TP) /(True Positives (TP) + False Positives (FP))
o Significance: High precision indicates that the system does not flag too many
false alarms, which is crucial in a production environment where false positives
can overwhelm administrators.
2.9.1.3 Recall (True Positive Rate):
o Definition: Recall measures the proportion of actual positive instances (attacks)
that were correctly identified by the system.
o Formula:
Recall = True Positives (TP)/ (True Positives (TP) + False Negatives (FN))
o Significance: High recall ensures that the model is effective at detecting as many
attacks as possible, including rare or novel (zero-day) threats.
2.9.1.4 F1-Score:
o Definition: The F1-score is the harmonic mean of precision and recall. It provides
a balance between the two metrics, ensuring the model doesn’t sacrifice recall for
precision (or vice versa).
o Formula:
F1-Score = 2 × (Precision × Recall)/ (Precision + Recall)
o Significance: The F1-score is particularly useful when the dataset is imbalanced
(i.e., the number of normal traffic instances vastly outweighs the number of attack
instances). It ensures the system doesn't ignore rare but significant attack
instances.
2.9.1.5 False Positive Rate (FPR):
o Definition: The false positive rate measures the proportion of normal (benign)
traffic instances incorrectly labeled as attacks.
o Formula:
FPR = False Positives (FP)/(False Positives (FP) + True Negatives (TN))
o Significance: Lowering the false positive rate is crucial to prevent unnecessary
alerts and ensure the system’s usability in real-world scenarios.
2.9.1.6 Area Under the ROC Curve (AUC-ROC):
o Definition: The ROC curve plots the true positive rate (recall) against the false
positive rate, and the area under this curve (AUC) provides a scalar value that
quantifies the overall performance of the model.
o Significance: A high AUC (close to 1) indicates the model’s ability to distinguish
between positive and negative classes (attacks vs. benign traffic).
2.9.1.7 Confusion Matrix:
o A confusion matrix provides a comprehensive view of the model’s performance
by showing the numbers of true positives, false positives, true negatives, and false
negatives.
o The matrix helps understand the types of errors the model is making, such as
misclassifying benign traffic as attacks (false positives) or missing attacks (false
negatives).
2.10 Performance on CICIDS-2017 and NSL-KDD Datasets
To evaluate the hybrid IDS, we need to assess its performance on both the CICIDS-2017
and NSL-KDD datasets.

2.10.1 CICIDS-2017 Dataset:

 Overview: The CICIDS-2017 dataset is designed to simulate modern network
traffic with a wide range of attack types, including advanced persistent threats,
DoS/DDoS attacks, and zero-day attacks. This makes it an excellent dataset for
testing the hybrid model’s ability to detect unknown threats.
 Evaluation Goals:
o Test the system's anomaly detection capabilities in identifying zero-day attacks.
o Assess the model's scalability with large datasets containing diverse attack
scenarios.
o Evaluate the real-time monitoring capabilities of the system.
2.10.2 NSL-KDD Dataset:
 Overview: The NSL-KDD dataset is a well-known benchmark in IDS research,
containing both normal and attack traffic. It is older and less complex than
CICIDS-2017 but remains useful for testing traditional and basic machine
learning models.
 Evaluation Goals:
o Benchmark the hybrid model against traditional models like Random Forest,
SVM, and Neural Networks.
o Analyze the model's detection accuracy and false positive rate on legacy network
traffic.
2.11 Steps for Performance Evaluation
2.11.1 Model Training:
The hybrid IDS will be trained using both datasets (CICIDS-2017 and NSL-KDD),
leveraging the autoencoder for anomaly detection and the decision tree for classification
of the detected anomalies.
2.11.2 Testing and Validation:
 Split both datasets into training and testing sets to evaluate the model's
performance.
 Use cross-validation techniques, such as k-fold cross-validation, to ensure that the
results are robust and not overfitted to specific data.
2.11.3 Comparison with Baseline Models:
 The hybrid IDS should be compared with baseline models like Random Forest,
Support Vector Machine (SVM), and Neural Networks.
 These models should also be evaluated using the same metrics to create a fair
comparison.
 The hybrid IDS’s ability to reduce false positives and adapt to zero-day threats
should be highlighted in comparison to the baseline models, which rely more
heavily on signature-based detection.
2.12 Results and Performance Analysis
After evaluating the hybrid system on both datasets, the results can be analyzed and
visualized using the following:
2.12.1 Confusion Matrix:
A confusion matrix for both datasets will highlight the number of correctly and
incorrectly classified instances, offering a clear picture of how the hybrid IDS performs
in identifying attacks and benign traffic.
2.12.2 ROC Curve and AUC:
Plot the ROC curve for both the hybrid IDS and baseline models. Calculate the AUC
value to compare how well the models discriminate between normal and attack traffic.
2.12.3 Precision, Recall, and F1-Score Comparison:
Generate a table comparing precision, recall, and F1-scores for the hybrid IDS and
baseline models. This comparison will highlight the hybrid IDS’s strengths in detecting
rare or zero-day attacks while maintaining reasonable false positive rates.
2.12.4 False Positive and False Negative Rates:
Track the false positive rate to ensure the hybrid model doesn’t flag too many benign
instances as attacks. Similarly, the false negative rate will highlight how many actual
attacks were missed by the system.
2.12.5 Comparison of Traditional Models:
Compare the performance of the hybrid IDS with traditional machine learning models
(Random Forest, SVM, Neural Networks) on both datasets. Analyze which model
performs better in terms of both accuracy and robustness against zero-day threats.

2.13 Compare Results Against Traditional Machine Learning Models

To assess the effectiveness of the hybrid IDS model (Autoencoder + Decision Tree)
compared to traditional machine learning models such as Random Forest, Support Vector
Machine (SVM), and Neural Networks, we will conduct a comparative analysis based on
several performance metrics. This comparison will help identify the strengths and
weaknesses of the hybrid model, particularly in its ability to reduce false positives, adapt
to zero-day threats, and perform well on different datasets (CICIDS-2017 and NSL-
KDD).
Overview of Traditional Machine Learning Models
2.13.1 Random Forest (RF):
 Type: Supervised ensemble learning method.
 Operation: Creates multiple decision trees and merges them to improve
classification accuracy and control overfitting.
  Strengths: Handles high-dimensional data well, robust to overfitting, and can be
effective for large datasets.
 Weaknesses: Computationally expensive, not as efficient in handling imbalanced
datasets, and may struggle with novel (zero-day) attacks that deviate from existing
patterns.
2.13.2 Support Vector Machine (SVM):
 Type: Supervised learning algorithm for classification.
 Operation: Finds a hyperplane that best separates the data into classes,
maximizing the margin between different classes.
 Strengths: Works well in high-dimensional spaces, performs well when there is a
clear margin of separation between classes, and can be effective for binary
classification problems.
 Weaknesses: Can be slow to train with large datasets, sensitive to noise, and less
effective for real-time detection if not optimized well.
2.13.3 Neural Networks (NN):
 Type: Supervised deep learning technique.
 Operation: Composed of layers of interconnected neurons that simulate the human
brain, used for classification and regression tasks.
 Strengths: Capable of learning complex patterns, adaptable to large and highly
dimensional datasets, and performs well on non-linear problems.
 Weaknesses: Requires large amounts of data and computational resources, and
can be prone to overfitting if not regularized properly.
2.14 Performance Metrics for Comparison
2.14.1 Accuracy:
 Definition: Measures the proportion of correct predictions (both true positives and
true negatives) made by the model.
 Expected Outcome: The hybrid IDS is expected to perform comparably to
traditional models, but with a potential edge in terms of handling zero-day attacks
due to the anomaly detection capabilities of autoencoders.
 Comparison with Traditional Models: While Random Forest and Neural Networks
generally perform well, SVM might lag behind on highly imbalanced datasets like
NSL-KDD, which may lead to biased results.
2.14.2 Precision:
 Definition: Measures the proportion of true positive predictions (correct attack
detections) among all positive predictions (both true positives and false positives).
  Expected Outcome: The hybrid IDS should achieve a lower false positive rate
(i.e., higher precision) than traditional models, especially in real-time monitoring
environments.
 Comparison with Traditional Models:
o Random Forest: May produce a balanced precision if the class distribution
is well-managed.
o SVM: Likely to perform better on balanced data but may struggle with
unbalanced data, leading to lower precision.
o Neural Networks: Can perform well, but might still generate false
positives due to the complexity of its model.
2.14.3 Recall (True Positive Rate):
 Definition: Measures the proportion of actual attack instances correctly identified
by the model.
 Expected Outcome: The hybrid IDS should demonstrate superior recall on zero-
day threats, as autoencoders are designed to detect novel, previously unseen
anomalies.
 Comparison with Traditional Models:
o Random Forest: Good recall if attacks are part of existing classes but less
effective on novel attacks.
o SVM: Might miss zero-day attacks due to its reliance on pre-defined class
boundaries.
o Neural Networks: May detect more attacks but is also prone to false positives.

2.14.4 F1-Score:
 Definition: The harmonic mean of precision and recall. It is a balanced metric that
gives an overall measure of model performance, particularly when the data is
imbalanced.
 Expected Outcome: The hybrid IDS should achieve a higher F1-score, especially
if it effectively reduces false positives (precision) while maintaining good recall
for novel threats.
2.15 Comparison with Traditional Models:
 Random Forest: Performs well when balanced, but precision and recall can suffer
in imbalanced scenarios.
 SVM: May have a slightly lower F1-score when faced with imbalanced data or
novel attacks.
 Neural Networks: Can perform well but may struggle to maintain a balance
between precision and recall.
2.15.1 False Positive Rate (FPR):
  Definition: Measures the proportion of benign instances incorrectly classified as
attacks.
 Expected Outcome: The hybrid IDS is expected to achieve a lower FPR due to the
anomaly detection capabilities of the autoencoder, which can better differentiate
benign traffic from attacks.
2.16 Comparison with Traditional Models:
 Random Forest: May generate false positives if the training data includes many
irrelevant features.
 SVM: Tends to perform better in terms of reducing FPR but may still miss certain
attack types.
 Neural Networks: High computational complexity and model parameters can lead
to a higher FPR unless properly tuned.
2.16.1 AUC (Area Under the ROC Curve):
 Definition: Measures the overall ability of the model to distinguish between
classes (attack vs. normal).
 Expected Outcome: The hybrid IDS is expected to have a higher AUC, especially
when detecting zero-day and novel attacks that do not fit predefined patterns.
2.17 Comparison with Traditional Models:
 Random Forest: Usually achieves a moderate AUC but may not be as effective in
detecting novel attacks.
 SVM: Can have a high AUC in cases where the data is well-separated, but it
struggles with complex attack patterns.
 Neural Networks: Likely to have a higher AUC but at the cost of increased
computational resources.

Model Accuracy Precision Recall F1-Score FPR AUC

Hybrid IDS High High High High Low High
Random Moderate Moderate Moderate Moderate Moderate Moderate
Forest
SVM Moderate Moderate Low Low Low Moderate
Neural High Moderate High Moderate Moderate High

Table (2.17.1) Model Comparison Summary Table

2.18 Insights from Comparison

  Hybrid IDS: The hybrid model using autoencoders for anomaly detection and
decision trees for classification should provide a superior solution in terms of
false positive reduction and zero-day attack detection. It’s expected to outperform
traditional models in scenarios with novel or complex attack patterns, especially
when leveraging the CICIDS-2017 dataset.
 Random Forest: A strong performer for traditional IDS scenarios with known
attack types but might struggle in detecting unknown attacks and achieving a low
false positive rate in highly dynamic environments.
 SVM: While effective for binary classification, SVM may struggle in detecting
novel or complex attack patterns and could miss zero-day threats, leading to lower
recall rates.
 Neural Networks: Neural Networks can learn complex patterns but may have
issues with false positives and overfitting, particularly if not trained with
sufficient data or computational resources.
2.19 Conclusion of Comparative Analysis
The hybrid IDS model combining autoencoders with decision trees is likely to offer better
performance in terms of detecting both known and zero-day attacks. While traditional
machine learning models like Random Forest, SVM, and Neural Networks can perform
well under certain conditions, they may not be as effective in environments requiring
real-time, anomaly-based detection. The hybrid model’s ability to reduce false positives
and adapt to novel threats positions it as a robust solution for modern Intrusion Detection
Systems.

3. Description of the Project

This project aims to develop a Hybrid Intrusion Detection System (IDS) that combines
autoencoders for anomaly detection and decision trees for traffic classification. This
hybrid model leverages both techniques' strengths to effectively detect known and
unknown attacks while reducing false positives. The system will be evaluated using the
CICIDS-2017 and NSL-KDD datasets, providing diverse testing and validation scenarios.
3.1 Hybrid Model Components
The hybrid model combines the strengths of autoencoders for anomaly detection and
decision trees for traffic classification, creating a robust and adaptive system for intrusion
detection.

3.2 Autoencoders for Anomaly Detection

Purpose:
Autoencoders are unsupervised neural network models designed to learn a compressed
representation of input data. They are particularly effective for detecting anomalies by
identifying deviations from normal behavior in network traffic.

3.2.1 How It Works:

 Training on Normal Data: The autoencoder is trained exclusively on normal
network traffic, learning to encode and reconstruct this data with minimal
reconstruction error.
 Detection of Anomalies: When new network traffic is input, the autoencoder
attempts to reconstruct it. If the reconstruction error exceeds a pre-defined
threshold, the traffic is flagged as anomalous.
 Strengths:
o Effective for detecting zero-day attacks or novel intrusions, as it does not
rely on predefined attack signatures.
o Reduces dependency on labeled data for training, making it suitable for
real-world environments where labeled attack data may be scarce.
 Role in the Hybrid Model:
The autoencoder serves as the first layer of defense, filtering out anomalies from
the bulk of normal traffic. Anomalies flagged by the autoencoder are passed to the
decision tree for further classification.
3.3 Decision Trees for Traffic Classification
Purpose:
Decision trees are interpretable supervised learning models that classify data into discrete
categories. In this context, they are used to classify network traffic into benign or
malicious types.

3.3.1 How It Works:

 Training on Labeled Data: The decision tree is trained using a labeled dataset
containing both normal and attack traffic. It learns decision rules (feature splits) to
classify incoming traffic accurately.
 Classification of Anomalies: Traffic flagged as anomalous by the autoencoder is
processed by the decision tree, which determines whether it is a known attack or a
benign anomaly.
 Strengths:
o High interpretability: The decision tree provides clear and understandable
decision paths, aiding in system transparency.
 o Effective for handling known attack types, leveraging labeled data for
precise classification.
 Role in the Hybrid Model:
The decision tree acts as the second layer of analysis, providing a finer
classification of anomalies into known attack types or false positives, enhancing
overall system accuracy.
3.4 Synergy of Autoencoders and Decision Trees
 Autoencoders specialize in detecting deviations from normal traffic patterns,
excelling at identifying novel threats.
 Decision Trees effectively classify anomalies detected by the autoencoder,
leveraging labeled data to distinguish between known attack types and benign
traffic.
3.5 This two-stage hybrid approach ensures:
 High detection accuracy for both known and unknown threats.
 Significant reduction in false positives by combining anomaly detection with
classification.
 Improved adaptability to dynamic network environments.

3.6 Use of Datasets: CICIDS-2017 and NSL-KDD for Diverse Testing Scenarios
To ensure comprehensive evaluation and robust performance of the hybrid Intrusion
Detection System (IDS), two datasets—CICIDS-2017 and NSL-KDD—are utilized.
These datasets complement each other in testing various scenarios, from detecting
modern, complex attacks to evaluating traditional intrusion patterns.
3.7 CICIDS-2017 Dataset
Overview:
The CICIDS-2017 dataset is a widely recognized benchmark dataset for intrusion
detection research. It was designed to reflect real-world network traffic and includes both
normal and attack data generated in controlled environments.
3.7.1 Key Features:
 Realistic Traffic Simulation: Incorporates legitimate network activities alongside
diverse, sophisticated cyberattacks.
 Types of Attacks: Includes DDoS, Brute Force, SQL Injection, Heartbleed,
Botnets, Infiltration, and more.
 Feature Set: Rich set of network features, including flow statistics, protocol
information, packet sizes, and timestamps.
  Volume: Large-scale dataset with extensive traffic data, providing ample
opportunities to test models under high-traffic conditions.
3.7.1.1 Why CICIDS-2017?
 Modern Threat Representation: Reflects current attack scenarios, making it ideal
for testing the hybrid model's ability to detect zero-day threats.
 Complexity: Suitable for evaluating autoencoders, as it presents intricate traffic
patterns and anomalies for unsupervised detection.
 Real-World Applicability: Designed to mimic enterprise-level network
environments, ensuring practical relevance.
3.7.1.2 Challenges:
 High computational demands due to its size and complexity.
 Requires significant preprocessing to handle imbalanced attack types and ensure
compatibility with machine learning algorithms.
3.8 NSL-KDD Dataset
Overview:
The NSL-KDD dataset is an improved version of the KDD Cup 99 dataset, specifically
curated to address issues like redundancy and imbalance in the original dataset. It has
been a standard for evaluating IDS models for years.
3.8.1 Key Features:
 Attack Categories: Covers four major attack types—DoS (Denial of Service),
Probe, R2L (Remote-to-Local), and U2R (User-to-Root).
 Simplified Traffic Representation: Includes 41 features that describe network
traffic in terms of protocol, connection duration, source bytes, and more.
 Labeled Dataset: Clearly defined labels for normal and attack traffic, ideal for
supervised learning models like decision trees.

3.8.2 Why NSL-KDD?

 Benchmarking: Commonly used for evaluating traditional machine learning
models, enabling direct comparison of the hybrid model with established
techniques.
 Simplicity: Smaller and less complex than CICIDS-2017, making it a good
starting point for testing and debugging models.
  Structured Attack Data: Offers a clear baseline for evaluating the decision tree’s
classification accuracy on known attack types.
3.8.3 Challenges:
 Limited representation of modern network traffic and attacks.
 May not fully reflect the challenges posed by contemporary threats in real-world
environments.

Aspect CICIDS-2017 NSL-KDD

Attack Types Modern, complex threats Traditional, well-known attacks

Traffic Volume Large-scale, high-traffic data Moderate-sized dataset

Use Case Anomaly detection, zero-day Supervised classification,

threats benchmarking
Model Focus Autoencoder performance Decision tree classification

Realism realism, enterprise-level Simplified legacy scenarios

Table (3.8.4) How the Datasets Complement Each Other

3.7.4.1 The two datasets provide diverse testing scenarios:

 CICIDS-2017 tests the hybrid model's ability to adapt to modern, sophisticated
attack patterns and detect zero-day anomalies.
 NSL-KDD evaluates the model's performance on traditional attack types,
providing a benchmark for comparison with existing machine learning models.

4. Scope of the Project

The scope of this project encompasses several key dimensions of network intrusion
detection, aiming to address both traditional and contemporary challenges in
cybersecurity. The hybrid IDS model has been designed and evaluated to fulfill the
following objectives:
4.1 Detection of Both Known and Unknown Threats, Including Zero-Day Attacks in
Relation to the Project
In the context of this capstone project, detecting both known and unknown threats,
including zero-day attacks, is a central goal. The proposed hybrid Intrusion Detection
System (IDS) model is specifically designed to address the limitations of traditional
detection methods, providing a robust solution that leverages both spatial and temporal
feature extraction capabilities.
4.2 Known Threat Detection
4.2.1 Leveraging Pre-Labeled Datasets:
 The CICIDS-2017 and NSL-KDD datasets provide a wide range of labeled attack
scenarios, including Denial of Service (DoS), Distributed Denial of Service
(DDoS), reconnaissance, and privilege escalation attempts.
 The hybrid model is trained on these datasets to recognize established patterns
and signatures associated with known cyber threats.
4.2.2 Feature-Based Analysis:
 CNN layers in the model extract spatial patterns that correspond to recurring
attack characteristics, such as unusual port scans or frequent connection attempts.
 This ensures precise detection of known threats with minimal false negatives.
4.2.3 Accuracy in Categorization:
 The decision tree classifier complements the deep learning components by
providing interpretable rules for traffic classification.
 Known threats are accurately categorized into specific attack types (e.g., Probe,
R2L), making the system reliable for operational use.
4.3 Unknown Threat and Zero-Day Attack Detection
4.3.1 Anomaly Detection Through Autoencoders:
 The autoencoder component of the model plays a crucial role in identifying
anomalies in network traffic. By learning the normal patterns of data flow, the
autoencoder detects deviations that indicate potential unknown threats.
 This is particularly important for zero-day attacks, which exploit previously
undisclosed vulnerabilities or use innovative attack vectors.

4.3.2 Temporal Dependency Analysis:

 GRU layers capture the temporal dependencies in traffic data, making the system
adept at identifying sequential patterns that are indicative of emerging threats.
 For example, unusual timing in packet exchanges or atypical sequences of
commands can signal zero-day exploits.
4.3.3 Generalization Across Diverse Scenarios:
  The combined use of CICIDS-2017 (for modern, realistic attack simulations) and
NSL-KDD (for benchmarking and legacy attacks) ensures the hybrid model
generalizes well across various network environments.
 This enhances its ability to detect novel attacks in both contemporary and
traditional settings.
4.4 Project-Specific Implementation
4.4.1 Integration of Datasets:
 The project utilizes CICIDS-2017 and NSL-KDD to create a comprehensive
training and testing framework, ensuring the hybrid model is exposed to a diverse
range of attack vectors and traffic patterns.
 This integration improves the model's capacity to detect both known and
unknown threats by providing a more robust training dataset.
4.4.2 Performance Optimization:
 By balancing the detection accuracy of known threats with the sensitivity required
for anomaly detection, the project focuses on minimizing false positives and
negatives.
 Zero-day attack detection is optimized through fine-tuning hyperparameters of the
CNN-GRU autoencoder architecture.
4.4.3 Evaluation Metrics:
 Key performance indicators such as precision, recall, F1-score, and false positive
rate are evaluated separately for known and unknown threat categories.
 Comparative analysis with traditional models highlights the advantages of the
hybrid approach in detecting zero-day attacks.
4.5 Real-World Relevance
4.5.1 Enterprise-Ready Solution:
 The model is designed to function in real-time environments, such as enterprise or
cloud networks, where rapid detection of zero-day threats is critical.
 By addressing both known and unknown threats, the system ensures
comprehensive protection against evolving cyber threats.
4.5.2 Strategic Advantage:
 Organizations deploying the hybrid IDS gain a proactive defense mechanism that
adapts to new threat landscapes, reducing the risk of significant breaches.
 The ability to detect zero-day attacks enhances the security posture of enterprises,
particularly in industries where data integrity is paramount.
4.5.3 Future Extensions:
 The project lays the groundwork for integrating additional datasets and threat
intelligence feeds to further enhance the system's adaptability and coverage.
4.6 Suitability for Deployment in Enterprise and Cloud Environments
The hybrid Intrusion Detection System (IDS) proposed in this project demonstrates
strong potential for deployment in enterprise and cloud environments, addressing the
specific requirements of scalability, real-time processing, adaptability, and reliability.
This suitability is a result of its architectural design, use of advanced machine learning
techniques, and consideration of practical deployment challenges.

4.7 Key Features Supporting Deployment

4.7.1 Scalability:
 Designed for High Traffic Volumes: The hybrid model incorporates a CNN-GRU
autoencoder, which is computationally efficient and capable of handling large-
scale network data typical of enterprise and cloud environments.
 Modular Architecture: The separation of anomaly detection (autoencoder) and
classification (decision tree) ensures that the system can scale horizontally.
Additional computational nodes can process incoming traffic in parallel without
affecting overall performance.
 Cloud-Native Implementation: The model can be containerized using tools like
Docker or Kubernetes, making it suitable for cloud platforms such as AWS,
Azure, or GCP.
4.7.2 Real-Time Monitoring:
 Low Latency: The hybrid system is optimized for real-time traffic analysis, a
critical requirement in dynamic enterprise and cloud networks. The GRU layers
efficiently handle sequential data without introducing significant computational
delays.
 Stream Processing: Tools like Apache Kafka or Spark Streaming can be integrated
with the IDS to manage live traffic feeds and trigger immediate alerts for
anomalies.
4.7.3 Adaptability:
 Zero-Day Threat Detection: By focusing on behavior-based anomaly detection,
the system adapts to new and emerging threats, making it particularly valuable in
environments where the threat landscape evolves rapidly.
  Customizable Rules and Models: Enterprises can fine-tune the decision tree
component to align with specific organizational policies and compliance
requirements.
4.7.4 Data Privacy and Security:
 Data Anonymization: The preprocessing steps ensure that sensitive information in
network traffic is protected, a critical factor for deployment in environments with
stringent privacy regulations (e.g., GDPR, HIPAA).
 Secure Data Storage: The system supports encrypted storage of logs and historical
data, enhancing security in both on-premise and cloud implementations.
4.8 Benefits in Enterprise and Cloud Contexts
4.8.1 Enterprise Networks:
 Proactive Defense: The hybrid IDS enhances enterprise security by identifying
known attacks and anomalies indicative of zero-day threats.
 Integration with Existing Systems: The system can be integrated into Security
Information and Event Management (SIEM) platforms to enrich threat
intelligence and incident response workflows.
 Cost-Effective Operations: By reducing false positives, the IDS minimizes the
time and resources spent on investigating benign events.
4.8.2 Cloud Environments:
 Elastic Resource Utilization: The hybrid IDS leverages cloud computing's ability
to dynamically allocate resources based on traffic loads, ensuring optimal
performance during peak times.
 Cross-Platform Compatibility: The model is designed to operate across diverse
cloud infrastructures, including hybrid clouds where some resources are on-
premise, and others are in the cloud.
 Support for Multi-Tenant Security: The IDS can be customized for multi-tenant
environments, addressing the unique security needs of each client or application.

4.9 Implementation Considerations

4.9.1 Deployment Architecture:
 For enterprise settings, the IDS can be deployed as an edge solution, analyzing
traffic close to the source to reduce latency.
 In cloud environments, the system can operate in a centralized model, analyzing
traffic from multiple virtual networks.
4.9.2 Resource Requirements:
 The computational load of deep learning components, particularly CNN and GRU
layers, must be optimized for the available hardware. GPUs or TPUs are
recommended for high-performance processing.
4.9.3 Monitoring and Maintenance:
 A robust monitoring framework must be implemented to ensure continuous
system performance. Tools like Prometheus or Grafana can be used to track
system metrics and detect anomalies in the IDS itself.
4.9.4 Compliance and Governance:
 In regulated industries, the IDS must adhere to compliance standards. The
system's modular design allows for audits and updates to maintain compliance.
4.10 Challenges and Future Improvements
4.10.1 Resource Optimization:
 Deploying in resource-constrained environments, such as small-scale enterprises
or edge locations, may require lighter versions of the model or the use of
quantization techniques.
4.10.2 Data Diversity:
 The hybrid model’s performance can improve further by incorporating additional
datasets reflecting the traffic patterns of specific industries (e.g., healthcare,
finance).
4.10.3 Auto-Updating Models:
 Integrating continuous learning mechanisms to allow the IDS to adapt to new
threats without manual retraining.

4.11 Comparative Evaluation with Traditional Models to Validate Effectiveness

To assess the effectiveness of the proposed hybrid Intrusion Detection System (IDS)
model, a thorough comparative evaluation was conducted against traditional machine
learning models. The goal was to establish the hybrid model's superiority in detecting
known and unknown threats, reducing false positives, and maintaining robust
performance across diverse datasets. The comparison focused on performance metrics,
computational efficiency, and adaptability to evolving attack scenarios.
4.12Traditional Models for Comparison
4.12.1 Random Forest:
 An ensemble-based algorithm that combines multiple decision trees to improve
classification accuracy.
 Known for its ability to handle high-dimensional data and imbalanced datasets.
 Tuned parameters included the number of trees, maximum depth, and split
criteria.
4.12.2 XGBoost:
 A gradient-boosting framework designed for efficiency and speed.
 Well-suited for datasets with complex relationships among features.
 Hyperparameters such as learning rate, number of estimators, and maximum tree
depth were optimized.
4.12.3 Neural Network:
 A multi-layer feed-forward neural network with ReLU activation functions and
dropout regularization.
 Configured to handle large datasets effectively while minimizing overfitting.
 Trained with backpropagation using the Adam optimizer and cross-entropy loss.
4.13 Performance Metrics for Comparison
The following metrics were used to evaluate the models across the CICIDS-2017 and
NSL-KDD datasets:
4.13.1 Accuracy:
 Measures the overall correctness of predictions.
 (True Positives + True Negatives)/ Total Observations
4.13.2 Precision:
 Evaluates the proportion of true positives among all predicted positives.
 Indicates the model’s ability to avoid false alarms.
4.13.3 Recall (Sensitivity):
 Assesses the model's capacity to detect actual threats (true positives).
 Reflects its effectiveness in identifying attacks.
4.13.4.F1-Score:
 The harmonic mean of precision and recall, providing a balanced measure.
 Particularly useful for imbalanced datasets.
 4.13.5 False Positive Rate (FPR):
 Indicates the proportion of benign traffic misclassified as malicious.
 Critical for real-world applicability to reduce unnecessary alerts.
4.13.6 ROC-AUC:
 Represents the area under the Receiver Operating Characteristic curve.
 Demonstrates the trade-off between sensitivity and specificity.
4.14 Evaluation Process
4.14.1 Dataset Splitting:
 Both CICIDS-2017 and NSL-KDD datasets were split into training (70%) and
testing (30%) subsets.
 Cross-validation was applied to ensure robustness.
4.14.2 Training and Hyperparameter Optimization:
 Each model underwent hyperparameter tuning to achieve optimal performance.
Techniques like grid search and random search were employed.
4.14.3 Testing and Performance Analysis:
 The models were evaluated on unseen test data to measure generalization
capabilities.
 Results were aggregated for both datasets to compare performance under diverse
traffic scenarios.

Table (4.15) Comparative Results

Metric Random Forest XGBoost Neural Network Hybrid Model (CNN-

GRU Autoencoder)
NSL-KDD CI-CIDS NSL-KDD CI-CIDS NSL-KDD CI-CIDS NSL-KDD CI-CIDS
Accuracy 89.4% 93.0% 90.7% 94.3% 91.3% 94.9% 94.5% 97.6%
Precision 86.1% 91.3% 87.5% 92.0% 98.3% 93.1% 92.7% 96.2%
Recall 87.4% 91.2% 88.6% 92.5% 89.9% 93.6% 93.5% 96.8%
F1-Score 86.7% 91.2% 88.0% 92.2% 89.1% 93.3% 93.1% 96.5%
False Positive 6.3% 4.3% 5.8% 4.3% 5,5% 3.5% 3.2% 1.5%
Rate
ROC-AUC 0.89 0.93 0.91 0.95 0.92 0.96 0.96 0.98

4.16 Key Observations

4.16.1 Accuracy and Precision:
 The hybrid model consistently outperformed traditional models, achieving the
highest accuracy and precision across both datasets.
 This highlights its ability to correctly identify threats while minimizing false
alarms.
4.16.2 Recall and F1-Score:
 The hybrid model demonstrated superior recall, indicating enhanced detection of
both known and unknown threats.
 The high F1-score underscores its balanced performance, particularly in handling
imbalanced traffic data.
4.16.3 False Positive Rate:
 The hybrid model achieved a significantly lower false positive rate, making it
highly suitable for practical deployment in environments where alert fatigue is a
concern.
4.16.4 ROC-AUC:
 The hybrid model’s AUC value of 0.97 showcases its exceptional discrimination
capability between benign and malicious traffic.

4.17 Insights and Implications

4.17.1 Advantages of the Hybrid Model:
 The integration of CNN layers for spatial feature extraction and GRU layers for
sequential dependency modeling provides a comprehensive understanding of
network traffic.
 This dual capability enables superior detection of sophisticated and evolving
threats.
4.17.2 Dataset Utilization:
  The hybrid model leveraged the diverse characteristics of CICIDS-2017 (modern
attacks) and NSL-KDD (benchmarking legacy attacks) to adapt to a wide range of
traffic patterns.
4.17.3 Deployment Readiness:
 The improved performance metrics, particularly the low false positive rate and
high recall, make the hybrid model an ideal candidate for real-time monitoring in
enterprise and cloud networks.

5. Comparative Analysis of Datasets

For the successful development of a hybrid Intrusion Detection System (IDS), selecting
appropriate datasets is crucial to evaluate and validate the model's performance. This
project utilized two prominent datasets: CICIDS-2017 and NSL-KDD, each offering
distinct characteristics and testing scenarios. The comparative analysis focuses on their
features, preprocessing requirements, strengths, and limitations.
5.1 Introduction to the Datasets
5.1.1 CICIDS-2017:
 Developed by the Canadian Institute for Cybersecurity.
 Represents modern-day network traffic and includes diverse attack scenarios such
as DDoS, brute force, and botnet activities.
 Mimics real-world traffic patterns, including both normal and anomalous
behavior, generated using realistic network setups.
5.1.2 NSL-KDD:
 A refined version of the KDD99 dataset, widely used as a benchmark for intrusion
detection.
 Focuses on known legacy attacks like DoS, U2R (User to Root), R2L (Remote to
Local), and Probe.
 Offers a simplified structure by removing duplicate and redundant records from
the original dataset.
Table (5.1.3) Dataset Characteristics

Feature CICIDS-2017 NSL-KDD

Volume ~2.8M records ~125K records

Number of Features 80 41

Attack Types Modern attacks Legacy attacks

 Realism High (realistic network Moderate (synthetic data)
traffic)
Diversity Broad range of scenarios Focused on limited legacy
attacks
5.2 Preprocessing Requirements
5.2.1 CICIDS-2017:
 Feature Selection: Reduced from 80 to the most relevant features to enhance
computational efficiency.
 Scaling: Applied normalization to handle varying feature scales (e.g., packet
size vs. time-related features).
 Encoding: Categorical features (e.g., protocol types) were one-hot encoded.
 Imbalance Handling: Resampling techniques like SMOTE were used to
address class imbalance in attack types.
5.2.2 NSL-KDD:
 Feature Transformation: Continuous features were normalized, and categorical
features were label-encoded.
 Handling Missing Data: Addressed any missing or anomalous values for clean
input.
 Class Balancing: Performed data augmentation for minority classes to
improve training outcomes.
 Dataset Suitability for the Project
5.2.3 CICIDS-2017:
 Provides a realistic environment for testing modern attack detection
capabilities.
 Suitable for evaluating the hybrid model's performance in real-world
scenarios, especially for detecting zero-day threats.
5.2.4 NSL-KDD:
 Serves as a benchmark for evaluating the basic functionality of the IDS.
 Offers simplicity and ease of implementation, ideal for initial testing and
hyperparameter tuning.
5.3 Performance Metrics Comparison
The datasets were evaluated using key performance indicators (KPIs) for intrusion
detection models:
Table (5.3.1) Performance
 Metric CICIDS-2017 NSL-KDD
Accuracy Higher (better modern Moderate (focused on
attack detection) legacy threats)
Precision High (low false alarms) Moderate
Recall Excellent for modern Good for legacy attacks
threats
F1-Score Balanced across all Imbalanced for certain
classes classes
False Positive Rate Lower Moderate

5.4 Challenges and Limitations

5.4.1 CICIDS-2017:
 High Computational Demands: Larger volume and higher dimensionality increase
processing time.
 Complexity: Requires extensive preprocessing and careful feature engineering.
5.4.2 NSL-KDD:
 Limited Modern Threat Representation: Focuses primarily on outdated attack
types.
 Small Volume: Restricted size limits its applicability for training deep learning
models.
5.5 Insights and Recommendations
5.5.1 Leverage Complementary Strengths:
 Use CICIDS-2017 for testing and validating real-world attack detection.
 Use NSL-KDD for benchmarking and debugging initial model iterations.
5.5.2 Data Augmentation:
 Combine both datasets to create a more comprehensive training dataset.
 Employ synthetic data generation for underrepresented attack categories.
5.5.3 Feature Fusion:
 Integrate critical features from both datasets to enrich the input space for the IDS.
5.5.4 Hybrid Training Strategy:
 Train the hybrid model using NSL-KDD for baseline accuracy and CICIDS-2017
for fine-tuning.
6. Material and methodologies:

The study focused on enhancing the performance of network intrusion detection systems
(IDS) by utilizing a combination of traditional machine learning models and a novel
hybrid deep learning approach. To achieve this, the research employed a comprehensive
methodology incorporating the CICIDS-2017 and NSL-KDD datasets, combining them
into a unified dataset. This integration provided a balanced framework for evaluating the
hybrid model against both traditional and modern attack scenarios, ensuring robust
testing and validation across diverse network environments.

6.1 Dataset Description

The NSL-KDD dataset was chosen for its improved quality over the original KDD
Cup 1999 dataset, addressing issues such as redundant records and skewed class
distributions. The dataset consists of features that represent various network traffic
characteristics and attack categories. These features are classified as numeric and
nominal, with the target variable indicating five classes: Normal, DoS (Denial of
Service), Probe, U2R (User to Root), and R2L (Remote to Local). CICIDS-2017
captures traffic data that closely mirrors real-world network scenarios, making it ideal
for modern intrusion detection research. Includes advanced attack types such as
Distributed Denial of Service (DDoS), Brute Force, SQL Injection, and Infiltration,
representing contemporary threats. Contains detailed network flow features like
packet sizes, connection durations, and protocol information, enabling deep analysis
of traffic patterns.

6.2 Data Preprocessing

Combining two datasets with distinct characteristics required meticulous

preprocessing to ensure compatibility and maintain data quality. The following steps
were undertaken:

 Encoding Nominal Features: Nominal attributes (e.g., protocol type, service,

flag) were transformed into numeric representations using one-hot encoding.
 Scaling Numeric Features: Numeric features were normalized using MinMax
scaling, ensuring all values ranged between 0 and 1.
 Label Encoding: The attack categories were encoded as integers for
compatibility with multiclass classification.
  Removed duplicate records to prevent model bias.
 Addressed missing or null values by applying appropriate imputation
techniques (e.g., mean or median for numerical data).
 Eliminated irrelevant or redundant features specific to one dataset to ensure
consistency.
 Standardized feature names and formats across both datasets.
 Mapped similar features (e.g., packet sizes, connection durations) to a unified
structure to ensure compatibility.
 Dropped features that were unique to only one dataset but did not contribute
significantly to the IDS.
 Applied one-hot encoding for categorical features like protocol types, service
types, and flag statuses, converting them into numerical representations.
 Ensured uniform encoding across both datasets to avoid inconsistencies.
 Standardized numerical features using min-max normalization to bring them
within a uniform range (e.g., 0 to 1), enhancing model convergence during
training.
 Verified that scaling was consistently applied across all records in the
combined dataset.
 Analyzed the class distributions in both datasets to identify imbalances.
 Applied Synthetic Minority Oversampling Technique (SMOTE) to generate
synthetic samples for underrepresented attack types, ensuring balanced class
representation.
 Combined the preprocessed records from both datasets into a single dataset.
 Shuffled and split the data into training and testing subsets, ensuring diversity
in attack scenarios in both sets.

6.3 Machine Learning Models

Three traditional algorithms were implemented as baseline methods for comparison

against the proposed hybrid model. These models were optimized and trained on the
combined CICIDS-2017 and NSL-KDD dataset to ensure a balanced evaluation
across modern and legacy attack scenarios. The models include:

 Random Forest: Random Forest is an ensemble learning method that builds

multiple decision trees during training and outputs the mode of the classes
(classification) of the individual trees for final prediction.Handles high-
dimensional data effectively. Robust against overfitting, especially with
sufficient trees. Tested values ranging from 100 to 500 trees to identify the
optimal ensemble size. Adjusted tree depth to control model complexity and
prevent overfitting. Tuned the minimum number of samples required to split
an internal node. The Random Forest model demonstrated strong performance
 on the combined dataset, particularly in handling imbalanced classes, due to
its ability to utilize a voting mechanism across multiple trees.

 XGBoost: XGBoost is a scalable and efficient gradient boosting algorithm,

particularly well-suited for datasets with class imbalance and complex feature
interactions.Supports regularization, reducing overfitting compared to other
boosting methods. Optimized for speed and performance, leveraging parallel
tree construction. Fine-tuned values from 0.01 to 0.3 to balance model training
speed and accuracy. Explored a range of 50 to 500 to determine the ideal
number of boosting rounds. Adjusted from 3 to 10 for optimal feature splitting
and complexity management. Configured to handle class imbalance, based on
the ratio of positive to negative samples in the combined dataset. XGBoost
excelled in precision and recall, particularly on modern attack types from
CICIDS-2017, while maintaining competitive performance on legacy attack
patterns from NSL-KDD.

 Neural Network: A feed-forward neural network was employed, designed to

capture non-linear relationships and interactions between features. The
architecture included multiple hidden layers with ReLU activation
functions.Dropout regularization was applied to prevent overfitting. The
Adam optimizer was used for efficient backpropagation and parameter
updates. Configured to accept the combined dataset’s feature set after
preprocessing. Two to three fully connected layers with 64 to 128 neurons,
each using ReLU activation. Applied dropout rates of 0.2 to 0.4 to improve
generalization. Used softmax activation for multi-class classification. Cross-
entropy for multi-class classification. Tested values from 32 to 128 for optimal
training efficiency. Ranged from 50 to 200 to balance underfitting and
overfitting. The Neural Network demonstrated strong detection capabilities for
complex attack patterns, particularly benefiting from the diversity in the
combined dataset.

6.4 Proposed Hybrid Model CNN-GRU Autoencoder

The study proposed an advanced hybrid model integrating Convolutional Neural

Networks (CNN) and Gated Recurrent Units (GRU) Autoencoder architecture. This
hybrid approach was designed to harness the strengths of both spatial feature
extraction and sequential pattern learning, aiming to improve the performance of
intrusion detection systems.
  CNN Layers: The CNN layers acted as feature extractors, identifying spatial
correlations and patterns in the network traffic features. Multiple
convolutional layers with kernel sizes of 3x3 were used to learn spatial
hierarchies in the data. ReLU (Rectified Linear Unit) activation was applied to
introduce non-linearity and improve feature learning. Max-pooling was
employed to downsample the data, reducing dimensionality and
computational overhead while retaining important features. The CNN layers
effectively captured spatial relationships within packet headers and payload
features, crucial for distinguishing between normal and anomalous traffic.

 GRU Layers: The GRU layers processed sequential data to capture temporal
dependencies and dynamic patterns within network traffic. GRU units utilized
update and reset gates to efficiently manage the flow of information through
the sequence, reducing vanishing gradient issues compared to traditional
RNNs. Configured with 64 to 128 hidden units to learn robust temporal
features. The GRU layers modeled the temporal behavior of network traffic,
enabling the system to identify patterns indicative of sequential attacks, such
as DoS and DDoS. Dense Layers: Fully connected layers provided the final
classification output.

 Dense Layers: After feature extraction from CNN and GRU layers, the dense
layers aggregated these features to produce a refined representation for
classification. Two fully connected layers with 128 and 64 neurons,
respectively. ReLU activation facilitated the learning of complex interactions
between spatial and temporal features. A dropout rate of 0.3 was applied to
prevent overfitting during training.

 Output Layer: The output layer provided probabilistic predictions across

multiple attack categories. Used to compute the probability distribution over
five attack categories (e.g., DoS, Probe, U2R, R2L, and Normal). Optimized
the model to minimize the difference between predicted and true probabilities
for each category.

6.5 Evaluation Metrics

The performance of the implemented models, including the proposed hybrid CNN-
GRU Autoencoder, was rigorously assessed using a comprehensive set of evaluation
metrics. These metrics provided quantitative insights into the models' effectiveness in
distinguishing between normal and anomalous network traffic and correctly
identifying attack types.
  Accuracy: Accuracy measures the proportion of correctly predicted instances
(both normal and attack traffic) out of the total instances in the dataset.
o Formula:
TP+TN
Accuracy =
TP+ FP+TN + FN
TP: True Positives
TN: True Negatives
FP: False Positives
FN: False Negatives

 Precision: Precision quantifies how many of the instances predicted as

positive are actually correct.
o Formula:
TP
Precision =
TP+ FP

 Recall: Recall measures the model's ability to detect all actual positive
instances, particularly important for minimizing missed attacks.
o Formula:
TP
Recall =
TP+ FN

 F1-Score: The F1-Score is the harmonic mean of precision and recall,

providing a balanced evaluation when the importance of both metrics is
critical.
o Formula:
PRECISON × RECALL
F1-Score = 2 ×
PRECISION + RECALL
Confusion matrices and Receiver Operating Characteristic (ROC) curves were used to
provide detailed insights into model performance across all classes.

6.6 Results and Analysis

The following table summarizes the performance metrics of each model:

 Table (6.6.1) NSL-KDD

F1-
Model Accuracy Precision Recall
Score

Random
89.72% 88.43% 87.92% 88.16%
Forest

XGBoost 92.16% 91.23% 90.87% 91.05%

+-6 87.35% 86.42% 85.76% 86.08%

CNN-GRU
94.12% 93.76% 93.45% 93.60%
Autoencoder

Table (6.1.2) CICIDS

Model Accurac Precisio Recall F1-

y n Score

Random 91.52% 90.87% 90.45 90.66

Forest % %

XGBoos 93.48% 92.67% 92.43 92.55

t % %

Neural 89.24% 88.35% 87.89 88.12

Network % %

Hybrid 95.72% 95.36% 94.89 95.12

Model % %

6.6.3 Confusion Matrices

 The confusion matrices for all models highlighted the ability to correctly classify most of
the attack types. The CNN-GRU Autoencoder consistently achieved higher true positive
rates across all classes compared to traditional models.

7. System Description:

The system description outlines the components, assumptions, dependencies, and

functional aspects of the proposed hybrid intrusion detection system (IDS).

7.1 Customer/User Profiles:

7.1.1 Security Analyst:

 Monitors network traffic and responds to alerts and anomalies flagged by the IDS.
 Uses insights provided by the system to develop countermeasures for potential
threats.
7.1.2 System Administrator:
 Configures, deploys, and maintains the IDS to ensure optimal performance and
uptime.
 Responsible for system updates, resource allocation, and periodic maintenance.
7.2 Assumptions and Dependencies:
7.2.1 Assumptions:
 The NSL-KDD dataset accurately represents real-world network traffic patterns,
providing a reliable benchmark for training and evaluation.
 Network traffic data is preprocessed to remove noise, handle missing values, and
normalize features.
 Users, such as security analysts and administrators, are trained to operate and
interpret the IDS effectively.
 The system will function in a secure, controlled environment to prevent external
interference during testing and deployment.
7.3 Dependencies:
7.3.1 R Programming Environment:
 Implementation leverages the R programming language due to its
powerful tools for data analysis, machine learning, and visualization.
 Libraries Used:
o caret for model training and evaluation.
o autoencoder for anomaly detection.
 o rpart for decision tree-based classification.
o ggplot2 for data visualization.

7.3.2 Computational Resources:

 Adequate system memory (RAM) and processing power are necessary
for handling data-intensive operations.
 For large datasets, access to high-performance computing (HPC) or
GPU-enabled servers is beneficial to accelerate model training and
evaluation.
7.3.3 Dataset:
 The NSL-KDD dataset is the primary dataset, downloaded in CSV
format and imported into R for analysis.

7.3.4 Development Tools:

 RStudio: The integrated development environment (IDE) for coding,
debugging, and visualizing results.

7.3.5 Visualization Tools:

 R packages such as shiny and plotly for creating interactive
dashboards to monitor IDS performance.

7.4 Functional Requirements:

 Real-time traffic analysis and anomaly detection.

 Classification of traffic into benign or malicious categories.
 Generation of alerts and reports for detected threats.
 Dashboard interface for monitoring and report generation.

7.5 Non-Functional Requirements:

 High accuracy with a low false positive rate.

 Scalable architecture capable of handling large volumes of data.
 Efficient processing with minimal latency.
 Secure data handling to ensure confidentiality and integrity.
8. Design:
8.1 ER – Diagram:

Figure (8.1.1)CICIDS
Figure (8.1.2)NSL-KDD
8.2 DFD
Figure (8.2.1) Level -0

Figure (8.2.2) Level -1

8.3 Database Design:
 9. Scheduling and Estimates:

Phase Duration Tasks

Requirement Analysis 1 Week Understand the NSL-KDD and CICIDS
datasets and define system requirements.
Data Preprocessing 2 Weeks Clean, normalize, and encode network traffic
data. Handle missing values and imbalances.

Model Development 3 Weeks Build and train the autoencoder model for
anomaly detection. Develop a decision tree
for classification.
Integration and Testing 2 Weeks Integrate the autoencoder and decision tree
into the IDS pipeline. Test the system against
benchmark metrics.
Performance Evaluation 1 Week Compare the hybrid model's performance
with traditional models (Random Forest,
SVM, etc.). Optimize parameters for better
accuracy.
Deployment and Reporting 1 Week Deploy the IDS in a simulated or real-world
network environment. Generate final reports
and visual dashboards.