Intrusion Detection System Using Machine Learning Algorithms — Complete Documentation & Simulation

Overview & System Context

Intrusion Detection System Using Machine Learning Algorithms: Comprehensive Web-Sourced Technical Investigation Exhaustive Research Dossier · Pure Web-Scraped Content: ~9662 Words · Real Software Screenshots: 3 1. Executive Overview & Technical Background Problem Statement: The task is to build a network intrusion detector, a predictive model capable of distinguishing between bad connections, called intrusions or attacks , and good normal connections.

Intrusion Detection System is a software application that detects network intrusion using various machine learning algorithms. IDS monitors a network or system for malicious activity and protects a computer network from unauthorized access by users, including perhaps insiders. The intrusion detector learning task is to build a predictive model (i.e., a classifier) capable of distinguishing between 'bad connections' (intrusion/attacks) and 'good (normal) connections'. Attacks fall into four main categories: You can download the dataset used in this project from Kaggle (the name of the dataset is Intrusion Detection System Using Machine Learning ).

Various Algorithms Applied: Gaussian Naive Bayes, Decision Tree, Random Forest, Support Vector Machine, Logistic Regression. Figure 1: How AI and ML Enhance Intrusion Detection in Kubernetes - Open Source ... Approach Used: I have applied various classification algorithms that are mentioned above on the KDD dataset and compare there results to build a predictive model.

Appending columns to the dataset and adding a new column name 'target' to the dataset. Reading the dataset('kddcup.data_10_percent_corrected") and adding Attack Type feature in the training dataset where attack type feature has 5 distinct values i.e. dos, normal, probe, r2l, u2r. Shape of X_train: (330994, 41), X_test: (163027, 41 Shape of y_train: (330994, 1), y_test: (163027, 1) Figure 2: Intrusion Detection Rulesets | Using Rulesets in Suricata IPS – PKYGD Shape of X_train after scaling: (330994, 30) Shape of X_test after scaling: (163027, 30) This repository contains the code for the project "IDS-ML: Intrusion Detection System Development Using Machine Learning".

The code and proposed Intrusion Detection System (IDSs) are general models that can be used in any IDS and anomaly detection applications. In this project, three papers have been published: This repository proposed three intrusion detection systems by implementing many machine learning algorithms, including tree-based algorithms ( decision tree, random forest, XGBoost, LightGBM, CatBoost etc. ), unsupervised learning algorithms ( k-means ), ensemble learning algorithms ( stacking, proposed LCCDE ), and hyperparameter optimization techniques ( Bayesian optimization )**.

Another intrusion detection system development code using convolutional neural networks (CNNs) and transfer learning techniques can be found in: Intrusion-Detection-System-Using-CNN-and-Transfer-Learning Figure 3: Intelligent Intrusion Detection System Against Various Attacks Based on ... A comprehensive hyperparameter optimization tutorial code can be found in: Hyperparameter-Optimization-of-Machine-Learning-Algorithms The use of autonomous vehicles (AVs) is a promising technology in Intelligent Transportation Systems (ITSs) to improve safety and driving efficiency. Vehicle-to-everything (V2X) technology enables communication among vehicles and other infrastructures. However, AVs and Internet of Vehicles (IoV) are vulnerable to different types of cyber-attacks such as denial of service, spoofing, and sniffing attacks. An intelligent IDS is proposed in this paper for network attack detection that can be applied to not only Controller Area Network (CAN) bus of AVs but also on general IoVs. The proposed IDS utilizes tree-based ML algorithms including decision tree (DT), random forest (RF), extra trees (ET), and Extreme Gradient Boosting (XGBoost).

The results from the implementation of the proposed intrusion detection system on standard data sets indicate that the system has the ability to identify various cyber-attacks in the AV networks. Furthermore, the proposed ensemble learning and feature selection approaches enable the proposed system to achieve high detection rate and low computational cost simultaneously. Paper 2: MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehicles Modern vehicles, including connected vehicles and autonomous vehicles, nowadays involve many electronic control units connected through intra-vehicle networks to implement various functionalities and perform actions. Modern vehicles are also connected to external networks through vehicle-to-everything technologies, enabling their communications with other vehicles, infrastructures, and smart devices. However, the improving functionality and connectivity of modern vehicles also increase their vulnerabilities to cyber-attacks targeting both intra-vehicle and external networks due to the large attack surfaces. To secure vehicular networks, many researchers have focused on developing intrusion detection systems (IDSs) that capitalize on machine learning methods to detect malicious cyber-attacks. In this paper, the vulnerabilities of intra-vehicle and external networks are discussed, and a multi-tiered hybrid IDS that incorporates a signature-based IDS and an anomaly-based IDS is proposed to detect both known and unknown attacks on vehicular networks. Experimental results illustrate that the proposed system can accurately detect various types of known attacks on the CAN-intrusion-dataset representing the intra-vehicle network data and the CICIDS2017 dataset illustrating the external vehicular network data. The proposed MTH-IDS framework consists of two traditional ML stages (data pre-processing and feature engineering) and four tiers of learning models: Paper 3: LCCDE: A Decision-Based Ensemble Framework for Intrusion Detection in The Internet of Vehicles Modern vehicles, including autonomous vehicles and connected vehicles, have adopted an increasing variety of functionalities through connections and communications with other vehicles, smart devices, and infrastructures. However, the growing connectivity of the Internet of Vehicles (IoV) also increases the vulnerabilities to network attacks. To protect IoV systems against cyber threats, Intrusion Detection Systems (IDSs) that can identify malicious cyber-attacks have been developed using Machine Learning (ML) approaches. To accurately detect various types of attacks in IoV networks, we propose a novel ensemble IDS framework named Leader Class and Confidence Decision Ensemble (LCCDE). It is constructed by determining the best-performing ML model among three advanced ML algorithms (XGBoost, LightGBM, and CatBoost) for every class or type of attack. The class leader models with their prediction confidence values are then utilized to make accurate decisions regarding the detection of various types of cyber-attacks. Experiments on two public IoV security datasets (Car-Hacking and CICIDS2017 datasets) demonstrate the effectiveness of the proposed LCCDE for intrusion detection on both intra-vehicle and external networks.

CICIDS2017 dataset, a popular network traffic dataset for intrusion detection problems CAN-intrusion dataset, a benchmark network security dataset for intra-vehicle intrusion detection If you are interested in hyperparameter tuning of machine learning algorithms, please see the code in the following link: https://github.com/LiYangHart/Hyperparameter-Optimization-of-Machine-Learning-Algorithms 2. System Architecture & Core Working Principles Please feel free to contact us for any questions or cooperation opportunities. We will be happy to help.

If you find this repository useful in your research, please cite one of the following two articles as: L. Yang, A. Moubayed, I. Hamieh and A. Shami, "Tree-Based Intelligent Intrusion Detection System in Internet of Vehicles," 2019 IEEE Global Communications Conference (GLOBECOM), 2019, pp.

1-6, 10.1109/GLOBECOM38437.2019.9013892. L. Yang, A. Moubayed, and A. Shami, “MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehicles,” L. Yang, A. Shami, G. Stevens, and S. DeRusett, “LCCDE: A Decision-Based Ensemble Framework for Intrusion Detection in 10.1109/GLOBECOM48099.2022.10001280.

Code for IDS-ML: intrusion detection system development using machine learning algorithms (Decision tree, random forest, extra trees, XGBoost, stacking, k-means, Bayesian optimization..) Pega.com is not optimized for Internet Explorer. For the optimal experience, please use: In a world accelerating at the speed of AI, understanding how these incredible capabilities get smarter is critical to driving business outcomes. Artificial intelligence (AI) is the theory of computer systems that can perform complex tasks. Machine learning (ML) is the process by which computers, including AI, learn new things; people use ML to feed data to the AI and train it over time.

For a computer to think on its own and perform complex tasks to become AI – it must digest and internalize more and more information and logic over time. That process is machine learning. As the computer takes on more work, ML enables it to become ever more “intelligent.” This is how AI understands and reacts to nuances and distinct scenarios. Computer logic is only as sound as the data that feeds it. If an AI learns biased inputs, it will output biased decisions. This can even challenge organizations with the best of intentions. It’s not always apparent when a strategy contains an implicit bias, but it’s essential for AI practitioners to identify it and adjust accordingly .

Workflow automation, real-time decisioning and low-code accessibility make it an industry standout. Numerous industries already use AI for more efficiency, accuracy, and better customer experiences. It’s also powerful for enterprise business process management , where it can analyze and consolidate workflows, automate back-end tasks, and flag redundant processes. The result? Streamlining at every step.

Artificial intelligence is the theory and development of computer systems trained to “think” and perform tasks that would normally require human intelligence. Machine learning is a subfield of AI; it’s the process that inputs data to train an AI system. ML gives the computer the ability to learn and “think” without explicit programming.

Machine learning informs AI to make individualized offers and meet specific customer needs. As machine learning feeds more nuanced data to the AI, companies can identify new opportunities. With more data from machine learning, the AI provides the next best action at every point in a process. Find out how enterprise generative AI transforms business operations, boosts productivity, and drives innovation.

Discover how enterprise AI transforms businesses by streamlining operations, enhancing insights, and driving innovation. Discover how agentic AI can autonomously plan, learn, and adapt—driving agile workflows across diverse applications. Discover how AI workflow automation is revolutionizing business operations by boosting organizational efficiency. 3. Software Implementation, Frameworks & Protocols Discover how AI automation reduces errors, cuts manual tasks, and frees up your team to focus on strategic initiatives.

Discover why businesses are increasingly relying on AI orchestration for better decision-making. Reimagine your processes and turn any workflow into a build-ready application with confidence. Visualize customer journeys and engagement strategies across all touchpoints and activate them. The increase in network attacks has necessitated the development of robust and efficient intrusion detection systems (IDS) capable of identifying malicious activities in real-time. In the last five years, deep learning algorithms have emerged as powerful tools in this domain, offering enhanced detection capabilities compared to traditional methods. This review paper studies recent advancements in the application of deep learning techniques, including Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Deep Belief Networks (DBN), Deep Neural Networks (DNN), Long Short-Term Memory (LSTM), autoencoders (AE), Multi-Layer Perceptrons (MLP), Self-Normalizing Networks (SNN) and hybrid models, within network intrusion detection systems. we delve into the unique architectures, training models, and classification methodologies tailored for network traffic analysis and anomaly detection. Furthermore, we analyze the strengths and limitations of each deep learning approach in terms of detection accuracy, computational efficiency, scalability, and adaptability to evolving threats. Additionally, this paper highlights prominent datasets and benchmarking frameworks commonly utilized for evaluating the performance of deep learning-based IDS. This review will provide researchers and industry practitioners with valuable insights into the state-of-the-art deep learning algorithms for enhancing the security framework of network environments through intrusion detection.

Application systems and computer networks are essential for effective business processes in today’s quickly evolving technology world. They enable the seamless exchange of resources, including data, processing power, storage, and information frontpor (2019) . As the need for automated systems that can quickly and efficiently accomplish organizational goals has grown, so has the use of application systems in networked environments Abraham and Bindu (2021) . At the same time, privacy and security issues in application systems and networks have become more well-known, highlighting the necessity of strengthening security controls against the constantly changing threats in today’s cyberspace Song (2022) .

Within the larger framework of modern civilization, the Internet is a vital resource that promotes international communication and information sharing. The Internet is essential for exchanging important information among authorities or wirelessly transmitting basic photographs on social media Momand et al. (2023) . Even with significant progress, persistent problems, including vulnerability to cyberattacks and difficulties enforcing international protection rules, exist, necessitating proactive measures on the part of businesses to defend their interests from possible security breaches and incursions.

The need to shield networks from an array of threats, including denial-of-service (DoS) assaults, unauthorized information disclosure, and data manipulation or destruction, is more critical than ever in an era where computer systems are becoming increasingly interconnected. In the realm of modern network security, ensuring the availability, confidentiality, and integrity of vital information systems has become paramount Vinayakumar et al. (2019) . When paired with firewalls, intrusion detection systems (IDS) are among the essential security elements that can effectively handle a wide range of security risks Lansky et al. (2021) . IDS techniques fall into two categories: Network Intrusion Detection Systems based on signatures and Anomaly Detection Systems based on anomaly detection. NIDS relies on signatures to identify intrusions and does so by comparing patterns across all the data they retrieve. On the other hand, anomaly-based alerts for intrusions detect any appreciable departures from the typical traffic pattern in the user behavior being monitored. Consequently, anomaly detection NIDS performs better when dealing with new attack patterns, while signature-based NIDS has a higher detection rate for recognized attack types. However, it frequently results in false alarms because of variations in intruder behavior Momand et al. (2023) .

Finding abnormal patterns in the system audit record might aid in identifying security breaches. In the past ten years, researchers have put forth a variety of Machine Learning and Deep Learning techniques to enhance Intrusion Detection Systems’ capacity to identify hostile efforts Liu and Lang (2019) . One kind of artificial intelligence technique called machine learning has the potential to extract pertinent facts from massive databases automatically. When sufficient training data is provided, machine learning-based intrusion detection systems can achieve higher detection levels.

Additionally, machine learning-based IDSs are easy to create and implement since they do not heavily rely on domain expertise Liu and Lang (2019) . One branch of machine learning that can produce very good outcomes is called deep learning. Jakhar and Kaur define Deep Learning as a subset of Machine Learning that is applied in computational problem-solving by using models and algorithms that mimic biological neural networks of the brain. Deep Learning, just like the brain, works by interpreting information, classifying it, and assigning it to various classes Jakhar and Kaur (2020) .

Functionally, Deep Learning incorporates a set of Machine Learning algorithms that attempt learning using artificial neural network concepts in multiple levels that correspond to different levels of abstraction Deng and Yu (2014) . Feature representations could be learned using deep learning techniques to transform unprocessed input into final outputs. One characteristic of deep learning that sets it apart is its deep structure, which consists of many hidden layers. However, traditional machine learning models have one or no hidden layer, including the support vector machine (SVM) and k-nearest neighbor (KNN) Liu and Lang (2019) . These conventional machine learning models are frequently called ”shallow models.” Deep learning approaches perform better when working with vast amounts of data than standard machine learning techniques Lansky et al. (2021) . Nevertheless, there are many barriers that IDS systems must overcome to promptly detect malicious intrusions due to the exponential increase in network traffic and associated security risks. Deep Learning Techniques for Intrusion Detection Systems (IDS) are currently being researched, and there is still more space to advance this technology for effective intrusion detection in IDS.

This systematic study aims to examine the current algorithms utilized to create intrusion detection systems and explain how they have been applied to guarantee everyone’s safety in cyberspace. This report will consider the current research from 2019 to 2023 to achieve this goal. The following research questions will be addressed in this work: What are the attacks covered by various datasets used in developing Deep Learning Network Intrusion detection Systems? This paper is organized in the following order. Section two surveys the related literature works regarding the deep learning algorithms used in Intrusion Detection Systems. Section three reports the results of performance for the various algorithms.

Section four gives a conclusion to this paper as well as giving directions for future works. This section will analyze the various deep learning algorithms based on recent research within the last five years. But first, we define Deep Learning as a computing concept in Machine Learning and Artificial Intelligence. Deep learning models that have been applied in Intrusion Detection Systems include Deep Belief Networks (DBNs), Convolutional Neural networks (CNNs), Recurrent Neural Networks (RNNs), Long-Short-Term-Memory (LSTM), Multilayer Perceptrons (MLPs), autoencoders, Self Organizing Maps (SOMs) Biswal (2023) .

Khan et al. in Khan et al. (2019) proposes an Improved CNN Model for network intrusion detection that combines the CNN algorithm and softmax algorithm. This model executes three processes in its intrusion detection method. First, data pre-processing is carried out through the conversion of symbolic data to numeric data then normalization is of the data undertaken. Secondly, a CNN model with three hidden layers is used for data training and feature extraction. Finally, the softmax classifier is utilized when obtaining the classification results. The proposed NIDS is tested using the KDD99 data set.

The results of the experiment exhibit an improved performance in terms of the accuracy in intrusion detection compared to Support Vector Machines and Deep Belief Networks. This accuracy is improved by increasing the number of epochs in the proposed model. The proposed model should be tested for its performance against newer state-of-the-art network intrusion algorithms and its applicability in IoT and Cloud computing where there are large-scale data and sophisticated attacks.

A novel NIDS is proposed by Liu et al. in Liu et al. (2019) that improves intrusion detection by employing CNN combined with Fast Fourier transformation for representation learning. The proposed method works by converting network traffic to an image after the preprocessing stage by modeling a visual data transformation representation algorithm. A CNN classifier i.e. the ResNet50 framework by MathWorks, is then applied for image processing. Image processing for anomaly detection is the core method that Liu et al. propose to improve intrusion detection using CNN. The algorithm is tested using the KDDCup’99 dataset. The proposed algorithm shows overall better performance in both binary and multi-class classification. It also shows improved accuracy when compared with other algorithms. However, this algorithm does not detect U2R and R2L anomalies since they are very few in the used data set. Furthermore, it is not tested against state-of-the-art deep learning algorithms to compare its performance.

Hu et al., built and implemented a Wi-Fi sensing system for detecting intrusion employing Channel State Information (CSI) at the physical layer of a Wi-Fi network as a detection signal in Hu et al. (2021) . To boost the sensitivity of a passive intrusion detection system, they used a path decomposition method and Deep learning convolutional neural network. CNN was used to enable the computer to learn and identify intrusion without extracting numerical characteristics. They used Channel State Information (CSI) dataset. In four different cases or scenarios that they had in their experiment, the average detection accuracy was 98.69% with a single person and 98.91% with multiple participants. They concluded that in comparison to prior methods, IDSDL can detect human movements on non-line of sight (NLOS) pathways more sensitively and increase system reliability.

Arun et al. Kumar Silivery et al. (2023) introduced an innovative intrusion detection algorithm designed to bolster network security in the wake of escalating thefts and system breaches. Leveraging Convolutional Neural Network (CNN), the authors present a high-level Intrusion Detection System (IDS) that meticulously examines network abnormalities, representing a substantial advancement in information security technology. The study emphasizes the pressing need for resilient systems capable of identifying novel attacks amidst the rapid evolution of networks and technology. By amalgamating machine learning and deep learning algorithms, the proposed IDS focuses on discerning elusive assault bundles. Significantly, the advanced system demonstrated a noteworthy enhancement in precision, fortifying its ability to accurately detect intricate attack patterns. This study underscores the pivotal role of Intrusion Detection Systems in fortifying information security and addressing the surge in cyber threats within the dynamic technological landscape (citation28). The specific network intrusion detection techniques and datasets employed in this study is unsw-nb15 Arun et al. Kumar Silivery et al. (2023) study the escalating occurrences of thefts and system breaches that underscore the pressing need for advanced security measures.

They introduce an innovative approach to address this challenge through the deployment of a high-level Intrusion Detection System (IDS) incorporating Convolutional Neural Network (CNN) technology. By leveraging machine learning and deep learning algorithms, the IDS aims to detect network abnormalities effectively. The primary objective of the study is to identify elusive attack bundles, enhancing precision in intrusion detection. The proposed system exhibits remarkable precision improvements, indicating its efficacy in recognizing intricate attack patterns. The research emphasizes the significance of information security technology, particularly IDS, in the evolving landscape of network threats. Through the application of high-performance intrusion detection systems, the study demonstrates a proactive approach to identifying and mitigating potential security breaches, thereby contributing to the overall enhancement of network security.

Alissa et al. in A. Alissa et al. (2022) address the pressing challenge of cybersecurity in the context of the Internet of Things (IoT), where a multitude of small smart devices transmit vast amounts of data over the Internet. Recognizing the inherent security flaws in these IoT gadgets, exacerbated by the lack of hardware security support, the study focuses on the development of an innovative solution for lightweight intrusion detection in resource-constrained IoT environments. The proposed approach, named Planet Optimization with Deep Convolutional Neural Network for Lightweight Intrusion Detection (PODCNN-LWID), leverages a Deep Convolutional Neural Network (DCNN) for intrusion identification and integrates Planet Optimization (PO) as a hyperparameter tuning process. The primary goal of PODCNN-LWID is to identify and categorize intrusions efficiently. The model undergoes two major processes: intrusion identification using DCNN and hyperparameter tuning through PO. Experimental validation on a benchmark dataset demonstrates the effectiveness of the PODCNN-LWID model, showcasing enhancements over alternative approaches in terms of intrusion detection performance. The study contributes valuable insights into mitigating cybersecurity challenges in resource-constrained IoT environments.

Man and Sun in Man and Sun (2021) proposed a CNN model for network intrusion detection strategy based on residual learning and focal loss. It aimed to enhance network security by introducing a novel approach based on residual learning. The techniques employed involved the implementation of residual connections within the neural network architecture using UNSW-NB15 dataset. Its results revealed that residual learning models are easier to train, with smaller loss values on training data and higher accuracy rates on testing data.

A fuzzy aggregation method for intrusion detection is proposed by Yang et al Yang et al. (2019) . This system is applicable in the Internet of Things (IoT) and cloud environments as well as software-defined networks. This research tries to address the problem of large data sets in the mentioned environments as well as capturing emerging attacks that are not captured in the training data set by combining the Modified Density Peak Clustering Algorithm (MDPCA)and the Deep Belief Network (DBN) model. MDPCA is used in the extraction of similar features from intricate and extensive network data then clustering it into various training subsets based on the similarity of the network traffic. This reduces the complexity of the training subsets. DBN then automatically applies the feature extraction and classification module to the training subsets where extraction of sophisticated abstract features is carried out while avoiding manual procedures and heuristic rules. The experimental tests of this proposed IDS are carried out using the NSL-KDD and UNSW-NB15 data sets. The results show overall best performance when compared to other state-of-the-art models in terms of accuracy and detection rate on the UNSW-NB15 except for CASCADE-ANN which ranks higher than MDPCA-DBN on the False Positive Rate parameter. Further tests show overall best performance when compared to other state-of-the-art models in terms of accuracy, detection rate, and false positive rate on the NSL-KDD data set. MDPCA-DBN also applies at least two test data sets which produce almost similar results in terms of efficiency thus proving its efficacy in solving the stated multi-class classification problems. The areas of application are clearly stated as well ranging from Software-defined networks, to IoT and Cloud Environments.

Thamilarasu and Chawla in Thamilarasu and Chawla (2019) identified a security vulnerability in IoT due to their large attack surface. They propose a secure, portable, and ready-to-deploy IDS suitable for an IoT environment. The heterogeneous nature of IoT networks makes them susceptible to cyber-attacks Al-Hadhrami and Hussain (2021) . The proposed algorithm forms a DNN using a DBN. The ability of DBN to be trained using unsupervised learning allows the DNN to be trained using unsupervised learning which is faster compared to supervised learning. Ideally, network virtualization and a DNN binary classifier are employed in detecting abnormal traffic in an IoT network. Upon testing the proposed model, there is overall improved accuracy over Inverse Weight Clustering in the simulation environment and the testbed. The wormhole attack however performs poorly in the test bed environment as compared to the simulation environment. This model is one of the few that have been tested in a testbed with real IoT network traffic with fairly good performance. However, the study does not state the dataset used in the simulation environment.

4. Practical Deployment, Analysis & Case Studies Sandeep Gurung et al. in Gurung et al. (2019) propose a deep learning intrusion detection system that uses a sparse auto-encoder and logistic regression in solving the network intrusion problem. A sparse auto-encoder that has a sparsity constraint is used to train the model. Logistic regression is then applied in the classification phase that results to a binary classification of either a normal user or intruder. The model is trained and tested using the NSL-KDD data set. This model results to a higher accuracy rate when compared to Signature-Based Intrusion Detection approaches with reduced chances of False Positives and False Negatives. This approach is best applied in real-time network monitoring servers although there is no evidence of real-world application of this model.

Telikani and Gandomi in Telikani and Gandomi (2021) propose a Cost-sensitive stacked auto-encoders model for intrusion detection in the Internet of Things networks with the aim of solving the class imbalance problem in IDSs. This model works by modifying the cost function of a Stacked Auto-Encoder to be more sensitive toward misclassification of minority class by reducing the effect of unbalanced data on the performance of intrusion detection systems. Batch Normalization method is used to speed up the learning phase while Softmax classifier, a multi-class version of Logistic Regression, is used in the classification of anomalies. Extraction of probabilities for each class is achieved using cross-entropy loss. This study uses the KDD cup 99 and NSL-KDD Datasets to train at test the proposed model. Experimental results show that it performs better than Stacked Auto-Encoder (SAE) and Non-symmetric Deep Auto-Encoder (NDAE) in all metrics when tested using both datasets.

An IoT intrusion detection system using asymmetric parallel auto-encoder (APAE) is proposed by Basati and Faghih in Basati and Faghih (2023) for detecting real-time cyber-attacks in IoT networks. This study uses the UNSW-NB15, CICIDS2017, and KDDCup99 datasets for training, testing and validating the model. An APAE is first trained on a dataset in the proposed model in order to estimate the identity function for the training data. In order to estimate the identity function for the training data, an APAE is first trained on a dataset in the suggested model. After that, the finished model is achieved by joining the first three components of the learned APAE—the transfer layer, encoder, and latent features—with a fully linked classifier layer at the end. Subsequently, the final model is retrained using the training data in order to determine the classifier weights and adjust the APAE encoder weights for precise classification. The proposed APAE model Slightly better in classification compared to other proposed auto encoder models. It also performs the best classification in the minority classes. This study achieves a lightweight NIDS suitable for IoT networks and devices which are limited in terms of processing power, memory, and energy efficiency.

A study by Kamalov et al. in Kamalov et al. (2021) , the authors present an Autoencoder-based Intrusion Detection System (IDS) designed for detecting Distributed Denial of Service (DDoS) attacks. The algorithm identifies intrusions by flagging anomalous traffic flows with higher reconstruction loss. The study utilizes the CSE-CIC-IDS2018 dataset for evaluation. The proposed strategy demonstrates superior performance compared to benchmark unsupervised systems in the detection of DDoS attacks.

A Scalable Hybrid Intrusion Detection System Alertnet (SHIA) is proposed by R. Vinayakumar et al. in Vinayakumar et al. (2019) . This approach Employs distributed deep learning with DNNs for analyzing and handling large sets of data in real time with the aim of solving the problem of detection and classification of unforeseen and unpredictable cyber-attacks. This intrusion detection system is modeled using a scalable computing framework based on Apache Spark Cluster Computing platform. A System Call trace is used for text representation and classification of process behaviours. The DNN is modelled using Feed forward Neural (FFN)Network and Multilayer Perceptron (MLP). This paper trains and tests the model with several datasets including KDD Cup 99, NSL-KDD, UNSW-NB15, WSN-DS, CICIDS 2017, Kyoto. The test results show overall superior performance than classical machine learning classifier models for NIDs and HIDs such RF, SVM, KNN, LR, AB. The experiments using a variety of datasets shows the versatility of the model and its ability to work in an environment with large scale data. The authors offer the proposed model for use Efficient real-time network monitoring for the network traffic and host-level operation such as Malicious host detection.

A hybrid classifier model for intrusion detection known as Spider Monkey Optimization and Deep Neural Network (SMO-DNN) is proposed by Khare et al. in Khare et al. (2020) . In this study the NSL-KDD and KDD Cup 99 Datasets have been used. At the data preprocessing stage, data cleansing is done using the Min-Max normalization technique then 1-N encoding is used to achieve consistent and unified data. The Spider Monkey Optimization is utilized for dimensionality reduction after which the reduced dataset is split into the Training and testing datasets then forwarded to the Deep Neural Network for classification which uses the the softmax classifier at the output layer. The resulting classification is either normal traffic or intrusion traffic.

SMO-DNN is tested in comparison with Principal Component Analysis (PCA) PCA-DNN and DNN and it exhibits better performance than the two models. PCA is tested as a reduction algorithm with DNN as the classifier. This model is a binary classifier that is applicable in large network. However, the study should have tested the model against other state-of-the-art network intrusion detection algorithms to further prove its efficacy.

A Transfer Learning for Network Intrusion Detection System (TL-NID) is proposed by Masum and Shahriar in Masum and Shahriar (2020) . This model combines a Deep Neural Network for Classification while transfer learning is used for feature extraction. This algorithm executes two procedures in finding a solution to the malicious network intrusion problem. Two step method is modelled using the pre-trained VGG-16 for feature extraction. Extracted features are then fed into a DNN for classification. The sigmoid layer of the DNN carries out binary classification. This model is trained and tested using the NSL-KDD dataset. The model is tested against traditional machine learning models i.e. Support Vector Machines (SVM), Logistic Regression (LR), Decision Tree (DT), and Random Forest (RF) and its performance is analyzed. The proposed model exhibits the best Performance in terms of Accuracy, Recall and F1-Score using both KDDTest+ and KDDTest-21 datasets.

Decision trees (DT) perform better than the proposed model on Precision and False Alarm Rate using the KDDTest+ dataset. SVM performs better than the proposed model on Precision and False alarm rate using the KDDTest-21 dataset. Khan et al., in Khan et al. (2021) suggested a DNN-based intrusion detection system for MQTT-enabled IoT smart systems. The suggested model’s performance was evaluated using a MQTT-IoT-IDS2020 dataset and another Message Queuing Telemetry Transport (MQTT) dataset. The proposed DL-based IDS’s performance with a default learning rate and the ADAM optimizer was compared to the performance of conventional ML-based IDSs such as KNN, NB, DT, and RF. The suggested model was further evaluated for binary-class and multi-class attack classification with different activation functions at the output layers. According to the results, the DL-based model for Bi-flow and Uni-flow featured data reached 99% and 98% accuracy for binary and multi-class attack classification, respectively.

Gulia et al. in Gulia et al. (2023) have made noteworthy strides in advancing network security, specifically within the domain of cloud computing, by introducing an Intrusion Detection System (IDS) that leverages the Group-Artificial Bee Colony Algorithm (G-ABC) algorithm alongside a Deep Neural Network (DNN). In their study, this innovative approach effectively addressed security concerns inherent in cloud computing environments, resulting in a commendable 96% detection rate for improved intrusion detection capabilities. Beyond the proposed IDS, the authors conducted an exhaustive literature review, highlighting various studies on both host-based (HIDS) and network-based (NIDS) IDSs, showcasing diverse methodologies, including hybrid models, optimization techniques, and robust IDS designs. These collective efforts contribute to overcoming challenges such as system complexity and limitations in alert systems. The proposed two-phase framework, combining G-ABC for optimal feature selection and DNN for classification, not only aims to elevate IDS performance but also demonstrates promising results, solidifying its potential impact on the future of network security in cloud environments.

Syariful et al. Ikhwan et al. (2022) delve into the critical domain of cybersecurity within the context of the Internet of Things (IoT). As the proliferation of IoT devices continues to enhance human lives, it also amplifies the susceptibility of networks to potential attacks. Recognizing the inherent vulnerabilities in IoT networks, the study employs artificial intelligence as a proactive measure to safeguard these networks by detecting and preventing attacks effectively. The research specifically focuses on network detection utilizing the Deep Neural Network (DNN) algorithm, a sophisticated artificial intelligence technique. The experimentation involved testing on the UNSW Bot-IoT dataset, with training data comprising 75% of the overall dataset. Remarkably, the DNN algorithm demonstrated exceptional performance, achieving an average detection accuracy of 99.999%. Despite the presence of a small validation loss indicative of overfitting, the study attests to the robust capability of the algorithm in effectively identifying and mitigating attacks in IoT networks.

Ashwaq et al., Almutairi and Abdulghani Alshargabi (2022) address the escalating vulnerabilities and attacks prevalent in the Internet of Things (IoT) environment. As the IoT landscape continues to expand, facilitating the connection of over 20 billion items by 2024, the imperative of securing this interconnected ecosystem becomes increasingly evident. To address this, the researchers propose a novel approach employing Recurrent Neural Network (RNN) deep learning algorithms for intrusion detection within the IoT environment. The study utilizes the NSL-KDD dataset for both training and testing purposes. The proposed RNN model demonstrates notable success, achieving an accuracy of 87% in detecting intrusions. The research underscores the significance of leveraging machine learning and deep learning techniques for enhancing security within the evolving IoT paradigm. Furthermore, the authors express their commitment to future work, intending to explore optimization algorithms to further augment the detection accuracy of their proposed model.

A novel network-based detection system for IoT attacks is modelled by Aldhaheri et al. in Aldhaheri et al. (2020) . The BoT-IoT dataset is used with the proposed algorithm combining Self-Normalizing Neural Networks (SNNs) with Dendritic Cell Algorithm (DCA). The aim is to classify IoT intrusion and minimizing false alarm generation. The proposed algorithm automates and smoothens feature extraction and categorization stage which in turn improves the classification performance. Features are either classified as safe or dangerous signals using the Self-Normalizing Neural Network (SNN). The proposed DeepDCA model exhibits better performance as a classifier compared to other models such as KNN, NB, MLP and SVM. This model also showed a high detection rate for IoT attacks with a high accuracy rate and a low False-Positive Rate. Self-normalization Neural Network is a fairly new technique and further studies could be carried out to explore its potential in solving intrusion detection problems and other deep learning problem domains.

Louati and Ktata propose a deep learning based multi agent system for intrusion detection (DL-MFID) in Louati and Ktata (2020) . Data pre-processing is carried out using pre-processor agent to achieve numericalization and normalized data. Feature reduction is then performed using an auto-encoder as a reducer agent to select relevant features. Finally multi-class classification is done using into 5 classes using two classifier agents i.e. MLP and KNN. This model is trained and tested using the KDD cup 99 dataset. Experimental results show that this model exhibits the highest accuracy rate of 99.95% compared to other proposed models which used the same dataset. This model Has however not been applied in real network traffic to test its efficiency. The approach of using multiagent classifiers is a new approach that requires further research to study how it can improve network intrusion detection systems.

Abdulrahman et al. in Mohammed et al. (2020) addressed the critical issue of securing network systems against Distributed Denial of Service (DDoS) attacks by developing two effective Intrusion Detection System (IDS) models based on multilayer perceptron neural networks (MLP). In the first model, MLP1, a one-hidden-layer MLP with 38 input nodes, 11 hidden nodes, and 5 output nodes achieved a remarkable detection accuracy of 95.6%. Building upon this, the second model, MLP2, enhanced with two hidden layers, surpassed MLP1, achieving an accuracy 2.2% higher. The study demonstrates the efficacy of MLP-based IDS in accurately classifying different request types and highlights the importance of robust models in countering DDoS attacks. The training and evaluation were conducted using the NSL-KDD dataset, showcasing the practical application of MLP-based IDS in real-world scenarios A multilayer perceptron for network intrusion detection is proposed by Rosay et al. in Rosay et al. (2022) aimed at preventing cyber-attacks in Vehicular IoT networks. This model work by training a multilayer perceptron model that quantifies the benefits of using deep learning strategies in network intrusion detection systems. The CIC-IDS2017 is used to define the IDS solution while the CSE-CIC-IDS2018 dataset is used to validate and test the proposed model. At the pre-processing stage the training Dataset is cleaned up to remove imbalances before the dataset is split to three categories namely; Training set – 50%, validation set - 25%, test set - 25%. Feature selection is then carried out to remove irrelevant features before the data normalization is done using the Z-normalization technique. At the experimental level, the model is built and trained using python and the tensor flow framework as the deep learning model. Experimental results show that the proposed model performs better than traditional ML models in terms of accuracy with significantly low false positive rates. This model was further deployed in a practical system on chip vehicle allowing test to be carried out in an embedded system. Several shortcomings were identified where some attacks are underrepresented and some features are not well calculated in both datasets used by the model. Testing this model in a real-world scenario means it can be easily re-engineered for practical implementation in the automotive industry.

Susilo and Sari in Susilo and Sari (2020) evaluate the effectiveness of Random forests, CNN and Multi-layer Perceptrons in IoT Network Intrusion Detection Systems. Their study uses the BoT-IoT dataset to for evaluation of intrusion detection. Owing to the heterogeneous nature of IoT networks and traffic, multi-class classification of intrusions was preferred. Experimental results show that Random Forests and CNNs perform better than MLP in terms of accuracy and Area Under Curve parameters in a multi-class classification scenario. The study proposes a combination or hybridization of algorithms as future studies in implementing NIDs for IoT environment. It is worth noting that the dataset used is specific to the research problem of intrusion detection in IoT application. However, the study ought to have considered other datasets in testing the efficacy of the designed models as well as testing them against other state-of-the-art algorithms for Network Intrusion Detection.

Dutta et al. in Dutta et al. (2020) propose a model for efficient network anomaly detection or cyber-attack detection system that employs a stacked ensemble classifier approach. This approach is meant for implementation in an IoT and cloud computing environment. This study used heterogenous datasets from IoT environments such as IoT-23, LITNET-2020 and NetML-2020. Dutta et al. combines DNN with LSTM in a stack generalization approach for initial classification, then uses logistic regression as a meta classifier. The method begins with data preprocessing followed by feature engineering that uses a Deep Sparse Auto-Encoder. Classification as the third step employs DNN and LSTM models followed by logistic regression which helps in eliminating biases from training sets. The end result is a binary classification results of the data as either normal or anomalous. The experimental results shows that the proposed stacked ensemble algorithm performs better than state-of-the-art classifiers and meta-classifiers. The use of three recent datasets that fit the problem domain is an indicator that the model would perform well if implemented in a real-world scenario.

In their work titled ”Intrusion Detection System using MLP and Chaotic Neural Networks”, Shettar et al. Shettar et al. (2021) propose a hybrid intrusion detection model that uses multi-layer perceptron and chaotic neural network to achieve its objective. They addressed the challenges in developing a robust Network Intrusion Detection System (NIDS) capable of identifying unexpected and impulsive cyber-attacks. While existing techniques focus on enhancing accuracy, the authors introduced a hybrid model integrating Multilayer Perception (MLP) and chaotic neural networks to simultaneously improve accuracy, precision, and notably, the false alarm rate—a critical aspect often overlooked in NIDS development. The methodology involved employing MLP for attack detection and chaotic neural networks to specifically target and reduce false alarm rates. The researchers conducted experiments using the KDD Cup’99 benchmark dataset, a widely used standard for intrusion detection evaluations. The results demonstrated that the hybrid approach outperformed the standalone MLP model by effectively mitigating false alarm rates while maintaining comparable accuracy, precision, and recall performance. This research contributes to the field of intrusion detection by offering a holistic solution that not only enhances the system’s ability to detect cyber-attacks accurately but also addresses the vital issue of minimizing false alarms. The findings suggest that incorporating chaotic neural networks in conjunction with MLP can lead to more reliable and efficient NIDS, crucial for maintaining the integrity and security of computer networks in the face of evolving cyber threats.

Alkahtani and Aldhyani in Alkahtani and Aldhyani (2021) developed a comprehensive framework system for detecting intrusions based on the IoT environment. The proposed technique was developed using an IoT ID20dataset attack; a dataset from the IoT infrastructure. To identify the incursion in this framework, three advanced deep learning algorithms were used: a convolution neural network (CNN), a long short-term memory (LSTM), and a hybrid convolution neural network with the long short-term memory (CNN-LSTM) model. The dimensionality of the network dataset was reduced, and the particle swarm optimization method (PSO) was utilized to identify key characteristics from the network dataset to improve the suggested system. Deep learning methods were used to process the collected features. The results revealed that the proposed systems obtained the following levels of accuracy: CNN (96.60%), LSTM (99.82%), and CNN-LSTM (98.80%).

Ashiku and Dagli in Ashiku and Dagli (2021) researched the use of deep learning architectures in the development of an adaptive and resilient network intrusion detection system (IDS) to identify and categorize network threats. The emphasis was on CNN with regularized multi-layer perceptron to enable customizable IDS with learning power to identify known and unknown network threats. They used UNSW-NB15 dataset. When compared to the results of similar deep learning-based network IDSs, the suggested deep-learning classification architecture combined with the semi-dynamic hyperparameter tuning approach revealed considerable improvements to multiclass models. According to the models, their proposed approach achieved an overall accuracy of 95.4% and 95.6% for pre partitioned and user defined multiclass categorization, respectively.

Muhuri et al in Muhuri et al. (2020) propose a network intrusion detection system based on RNN model. They combine Long Short-Term Memory and RNN for classification while Genetic Algorithm is utilized for optimal feature selection. This model is aimed at ensuring high detection rates compared to existing models using the NSL-KDD dataset. Optimal feature selection using GA is applied to the LSTM-RNN model to remove bias in the classification process. This helps in minimizing misclassification rate, training time as well as maximizing the accuracy of the model. In the proposed model the original features in the dataset are 122 and thus GA optimizes these features to a subset of 99 features. The proposed model LSTM-RNN with GA produced the best performance compared to RF and SVM in both binary and multiclass classification tests. The LSTM-RNN with GA therefore offers significant improvement in NIDs accuracy and detection rates compared to traditional machine learning approaches. This model is applicable in large dataset environments such a large computer networks with high traffic.

5. Performance Evaluation & Extended Technical Observations In their quest to fortify the security of the Internet of Things (IoT), Safi et al. Ullah et al. (2022) address the escalating concern of cyberattacks targeting IoT networks. Recognizing the susceptibility of numerous IoT nodes that process sensitive data, the researchers propose a novel Intrusion Detection System (IDS) leveraging a deep-convolutional-neural-network (DCNN). The DCNN architecture encompasses two convolutional layers and three fully connected dense layers, aiming to enhance overall performance while minimizing computational demands. The study conducts rigorous experiments on the IoTID20 dataset, employing diverse optimization techniques, including Adam, AdaMax, and Nadam. Performance evaluation metrics such as accuracy, precision, recall, and F1-score are systematically analyzed. Notably, the proposed DCNN-based IDS exhibits superior accuracy and robustness when compared to existing deep learning (DL) and traditional machine learning (ML) techniques. This research contributes to advancing intrusion detection methodologies tailored for IoT environments, providing an effective shield against evolving cyber threats.

S Amutha et al. Amutha et al. (2022) proposed an innovative approach, the NID-Recurrent Neural Network (RNN), to enhance the efficiency of Secure Network Intrusion Detection Systems (NIDS). Recognizing the importance of preventing malicious activities and cyber-attacks, the study delves into the realm of deep learning, specifically focusing on the integration of NID with Long Short-Term Memory (LSTM). Unlike traditional machine learning methods that rely on manually crafted features, deep learning approaches, including deep neural network (DNN), Convolutional Neural Network (CNN), and RNN, are explored for their potential to elevate the performance of intrusion detection. The researchers conduct a comprehensive analysis, comparing the accuracy and precision of each model, considering binary and multi-class classifications on the NSL-KDD dataset. Notably, the results reveal an 8% improvement in accuracy for the proposed NID-RNN approach, outperforming other deep learning algorithms. The RNN model achieves an impressive 99.4% accuracy in classifying attack types, showcasing its efficacy in bolstering network security.

Sunil et al. In Gautam et al. (2022) this research on intrusion detection systems (IDS), the authors introduced a novel approach by developing a composite model that combines Bidirectional Recurrent Neural Network (RNN) using Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The study utilized the CICIDS2017 dataset for simulations and evaluation. Notably, the hybrid RNN IDS achieved an outstanding 99.13% classification accuracy in predicting network attacks, showcasing the effectiveness of the proposed model. This performance surpassed the accuracy and False Positive rate of the Naïve Bayes classifier, highlighting the superiority of the deep learning-based hybrid approach. The model’s efficiency was further emphasized by its ability to achieve robust results using only 58% of the dataset attributes, demonstrating resource efficiency and practical applicability.

Sydney Mambwe Kasongo in Kasongo (2023) , the escalating volume of data transmission in communication infrastructures prompted the implementation of an Intrusion Detection System (IDS) framework utilizing advanced Machine Learning (ML) techniques. The framework employed various Recurrent Neural Networks (RNNs), such as Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and Simple RNN, to enhance the security of network systems. To address the challenge of low test accuracy in detecting new attacks with increasing feature dimensions, an XGBoost-based feature selection algorithm was integrated into the framework. The evaluation was performed on benchmark datasets, including NSL-KDD and UNSW-NB15, revealing significant achievements. For binary classification tasks on NSL-KDD, the XGBoost-LSTM model demonstrated superior performance with a test accuracy of 88.13%, a validation accuracy of 99.49%, and a training time of 225.46 seconds.

On UNSW-NB15, the XGBoost-Simple-RNN emerged as the most efficient model with a test accuracy of 87.07%. For multiclass classification, the XGBoost-LSTM achieved a test accuracy of 86.93% on NSL-KDD, while the XGBoost-GRU obtained 78.40% on UNSW-NB15. These outcomes underscore the effectiveness of the proposed IDS framework in comparison to existing methods, offering optimal intrusion detection performance in the face of evolving cyber threats.

Table 1 summarizes the methods by each of the intrusion detection mechanisms discussed in section 2 of literature. The thorough examination of intrusion detection systems (IDS) in the attached article highlights an important trend: the sparse use of IDS in the context of the Internet of Things (IoT). There is a clear gap in the application of intrusion detection systems (IDS) that are specifically designed for Internet of Things (IoT) contexts, despite the fact that cyber threats are constantly changing and networked environments are becoming more sophisticated. Convolutional neural networks (CNNs), deep belief networks (DBNs), autoencoders (AE), deep neural networks (DNNs), recurrent neural networks (RNNs), and self-normalizing neural networks (SNNs) are among the many deep learning models that are widely discussed and used in a variety of contexts, including cloud computing and targeted attack scenarios like distributed denial of service (DDoS). However, there seems to be less use of these models in the Internet of Things (IoT) space.

In order to improve the efficiency and accuracy of intrusion detection, researchers have shown ingenuity in fusing these deep learning models with cutting-edge algorithms such as Dendritic Cell Algorithm (DCA), Spider Monkey Optimization (SMO), and Group-Artificial Bee Colony (G-ABC). Nonetheless, the results highlight how important it is to pay more attention to and investigate IDS designed for IoT applications. The underrepresentation of IDS in IoT contexts raises concerns about a potential gap in addressing the particular issues presented by IoT networks, despite the abundance of research concentrated on other domains. This observation points to a need for additional study and development to strengthen IoT environments against new cyberthreats and to advance intrusion detection technologies in general inside the quickly growing network of linked devices.

In this systematic study, we were able to analyze deep learning algorithms that have been used to develop IDS for the last 5 years i.e 2019-2023. The research aimed to examine the current algorithms utilized to create intrusion detection systems and explain how they have been applied to guarantee everyone’s safety in cyberspace. Some of the key things that were considered include the datasets used to come up with the algorithms, what Deep learning algorithms have been used in IDS, the attacks covered in the datasets, and the problems that the models developed intended to solve.

In future work, it would be of great help to ensure that more studies have been done to come up with better IDS models that can be utilized in the rapidly growing IoT field to prevent advanced cyber threats. Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

This study discusses two widely-recognized deep learning approaches for network intrusion detection: a Deep Neural Network (DNN) and a Recurrent Neural Network (RNN). Both models are trained and evaluated on three widely used benchmark datasets: KDDCup99, NSL-KDD (each with five classes), and UNSW-NB15 (ten classes). Multiple optimizers, including Adam, SGD, Adamax, AdamW, and Adadelta, are then explored, with Adam consistently providing the best performance.

CrossEntropyLoss is found to be the most effective loss function for these multi-class classification tasks. Designed to automatically learn and extract relevant features from raw data, the models reduce reliance on manual feature engineering. Performance is assessed using accuracy, precision, recall, F1-score, and false positive rate. Experimental results show that both models achieve over 99% accuracy on KDDCup99, with improved detection rates and false positive rates below 1% for KDDCup99 and NSL-KDD. On the more complex UNSW-NB15 dataset, false positive rates also remain under 8%, demonstrating the models’ robustness and generalizability across diverse intrusion scenarios.

Generating detectors from anomaly samples via negative selection for network intrusion detection A multi-level dynamic sample augmentation and heterogeneous feature fusion framework for intelligent intrusion detection The growing reliance on the internet has increased exposure to cyber threats, making intrusion detection a critical security requirement. An “Intrusion Detection System (IDS)” monitors network traffic and identifies suspicious or abnormal activity.

IDSs can be passive, issuing alerts, or active, attempting real-time mitigation, and may suffer from false positives. Traditional IDS tools, such as Snort and Zeek, which rely on signatures and rules, struggle to handle today’s complex and evolving attacks, especially zero-day exploits. To address these limitations, modern IDS frameworks increasingly adopt deep learning, which enables systems to automatically learn patterns of normal and malicious behaviors. This shift provides more adaptive, accurate, and scalable detection capabilities for real-world cybersecurity environments.

Misuse-based detection: This method, which is also known as misuse detection, is predicated on a previously established database of acknowledged attack signatures or patterns. Each signature is a unique fingerprint of a specific type of malicious behavior that has been previously identified and stored in the system. When new network traffic is analyzed, it is compared against this database to detect any matches. If a match is found, the system flags the event as a known attack and may take appropriate action, such as blacklisting the source or destination node 1 . Signature-based IDS offers high accuracy and low false positives when dealing with known threats, making it computationally efficient and reliable in environments with predictable attack patterns. However, their primary limitation lies in their inability to detect novel or zero-day attacks, since no predefined signature exists for such exploits. This leaves them vulnerable to sophisticated and evolving threats that target previously unknown system weaknesses.

Anomaly dependent detection: The IDS, which works on anomalies, works on a fundamentally different principle. Rather than relying on a database of known threats, it establishes a baseline of what constitutes “normal” behavior within the network–typically derived through statistical analysis, machine learning, or historical observation 2 . Any deviation from this learned normal behavior is flagged as potentially malicious 3 . Anomaly-based IDS is particularly effective at detecting zero-day or previously unknown attacks, since they do not depend on predefined threat signatures. By modeling normal behavior, they can identify subtle deviations that may signal intrusions, malware, or insider threats. However, these systems often suffer from higher false positive rates, especially during the learning phase or in dynamic environments where “normal” behavior shifts frequently. They also demand greater computational resources and careful tuning to reduce misclassifications.

Hybrid-based detection: Hybrid IDS combines the accuracy of signature-based detection with the flexibility of anomaly-based methods. Typically, traffic is first checked against known signatures; if no match is found, anomaly analysis is applied. This approach reduces false alarms while still detecting zero-day attacks, offering broader coverage than either method alone. With advances in deep learning and ensemble techniques, hybrid IDSs have gained importance for their ability to fuse features effectively and withstand diverse attack scenarios. Their balanced design makes them well-suited for deployment in complex, real-time network environments.

IDS can be categorized based on the underlying development strategy, i.e., how the system is built to process and detect intrusions. Broadly, these systems can be classified into two major types: 1) rule-based systems and 2) learning-based systems. This classification is centered around whether the system depends on predefined rules or evolves through data-driven learning methods.

Rule-based systems: Rule-based IDSs are among the most traditional and widely deployed intrusion detection approaches. Examples include well-established open-source platforms such as Snort and Zeek (formerly Bro) . These systems operate based on a predefined set of human-written rules or signatures that represent known patterns of malicious activity. These rules are typically constructed using domain knowledge, previous incident logs, or known attack signatures stored in structured rule files or configuration scripts 4 . Rule-based IDS detects threats by matching traffic against predefined attack signatures, issuing alerts or responses when a match occurs. They are effective for known attacks but struggle with zero-day exploits and evolving threats. Constant rule updates also demand heavy manual effort, making them less practical in today’s fast-changing, high-traffic networks.

Machine learning/deep learning-based systems: Learning-based IDS uses machine learning and deep learning to detect intrusions by modeling traffic patterns and system behavior, enabling recognition of both known and evolving threats. Classical ML methods (e.g., Random Forests, SVM (Support Vector Machine), KNN (K-Nearest Neighbors)) achieve strong accuracy on datasets, like NSL-KDD 5 and UNSW-NB15 6 , while deep learning models, such as RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), Convolutional Neural Network (CNN), and DNN (Deep Neural Network), further enhance detection by capturing temporal or hierarchical features. These systems are adaptive but face challenges, including reliance on large labeled datasets, high training costs, and potential overfitting. To balance precision and adaptability, modern IDS often adopts hybrid designs that integrate rule-based and learning-based approaches for more resilient protection.

Artificial Intelligence (AI) enables machines to replicate human-like reasoning, decision-making, and problem-solving, with Artificial Neural Networks (ANNs) serving as the primary inspiration. Within AI, “Machine Learning (ML)” allows systems to learn patterns from data instead of relying on fixed rules. Algorithms, such as Support Vector Machines (SVM), Decision Trees, and “K-Nearest Neighbors (KNN)” have been widely applied across domains, offering adaptability and generalization to unseen data. Deep Learning (DL), a specialized branch of ML, uses multi-layered neural networks to automatically extract hierarchical features from complex data such as images, text, and network traffic. Advances in architectures like CNNs, RNNs, LSTMs, and Transformers, coupled with big data and GPU acceleration, have made DL particularly powerful for high-dimensional problems. In IDS, AI-driven methods represent a shift from static, rule-based detection to adaptive learning.

Models trained on benchmark datasets (e.g., KDDCup99 7 , NSL-KDD, UNSW-NB15) can identify anomalies, novel threats, and multi-stage attacks, making AI, ML, and DL critical foundations for building scalable and intelligent IDS. Harvested Online Documentation & Reference Sources • https://www.geeksforgeeks.org/machine-learning/intrusion-detection-system-using-machine-learning-algorithms/ • https://github.com/Western-OC2-Lab/Intrusion-Detection-System-Using-Machine-Learning • https://www.bing.com/aclick?ld=e8pppwqW5fXaEEeVt65gGAEjVUCUwKaIhYKpgmvcTKFvL7IewU2fCPfAxDSQjpV79zpUQ hc1hycxSFvqA0x-nrZNguEfde2bT5jqSV91P1TvWsZASaf6AGxflGdsWYZRpCy7cELJ9VKdxmmWGl2WzF05R9Mrs8EYF3yN 32o3-9_h6W4xXBM8neaeXaAEYp8lKv5lHP7g&u=aHR0cHMlM2ElMmYlMmZhZC5kb3VibGVjbGljay5uZXQlMmZzZWFyY2h hZHMlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q0MzcwMDA3MDM1NDkxMTI1MSUyNmRzX3Nfa3dnaWQlM2Q1ODcwMD AwNzc1MDE5NjQzNSUyNmRzX2FfY2lkJTNkNDAwODE3MjUwJTI2ZHNfYV9jYWlkJTNkMTY2NTIzNTUzMTMlMjZkc19hX2 FnaWQlM2QxMzUyNzMzMjk5MzYlMjZkc19hX2xpZCUzZGt3ZC0yOTgzNjI0NTQwNjYlMjYlMjZkc19lX2FkaWQlM2Q4MTg0N TAyMTczMzA4NyUyNmRzX2VfdGFyZ2V0X2lkJTNka3dkLTgxODQ1MTEyNjY4NDQzJTNhbG9jLTEyOSUyNiUyNmRzX2Vfb mV0d29yayUzZG8lMjZkc191cmxfdiUzZDIlMjZkc19kZXN0X3VybCUzZGh0dHBzJTNhJTJmJTJmd3d3LnBlZ2EuY29tJTJmY WktbWFjaGluZS1sZWFybmluZyUzZnV0bV9zb3VyY2UlM2RiaW5nJTI2dXRtX21lZGl1bSUzZGNwYyUyNnV0bV9jYW1wYWln biUzZEJfTkxfTm9uQnJhbmRfQUlfQ0VfQnJvYWRfKENQTi0xMTEwNTIpX0VOJTI2dXRtX3Rlcm0lM2RhZHZhbmNlZCUyNTI wbWFjaGluZSUyNTIwbGVhcm5pbmclMjZnbG9jJTNkMTUxNTcwJTI2dXRtX2NvbnRlbnQlM2RwY3JpZCU3YyU3Y3BrdyU3Y 2t3ZC04MTg0NTExMjY2ODQ0MyUzYWxvYy0xMjklN2NwbXQlN2NwJTdjcGR2JTdjYyU3YyUyNmdjbGlkJTNkMDQ1NDZkZ mNhY2QzMWQ5ZTg3NzAwNzI4NGU0YTYzNDclMjZnY2xzcmMlM2QzcC5kcyUyNiUyNm1zY2xraWQlM2QwNDU0NmRmY2 FjZDMxZDllODc3MDA3Mjg0ZTRhNjM0NyUyNnV0bV9zb3VyY2UlM2RiaW5nJTI2dXRtX21lZGl1bSUzZGNwYyUyNnV0bV9jY W1wYWlnbiUzZEJfTkxfTm9uQnJhbmRfQUlfQ0VfQnJvYWRfKENQTi0xMTEwNTIpX0VOJTI2dXRtX3Rlcm0lM2RhZHZhbmNl ZCUyNTIwbWFjaGluZSUyNTIwbGVhcm5pbmclMjZ1dG1fY29udGVudCUzZE1hY2hpbmUlMjUyMExlYXJuaW5n&rlid=04546d fcacd31d9e877007284e4a6347 • https://arxiv.org/html/2402.17020v1 • https://www.nature.com/articles/s41598-026-38317-w