Honeypot Log Analysis Using Machine Learning Python — Complete Documentation & Simulation

Overview & System Context

Honeypot Log Analysis Using Machine Learning Python: Comprehensive Web-Sourced Technical Investigation Exhaustive Research Dossier · Pure Web-Scraped Content: ~9523 Words · Real Software Screenshots: 3 1. Executive Overview & Technical Background This project is a sophisticated cybersecurity honeypot that uses a two-part AI system to detect intruders. It first generates a realistic but synthetic "normal" environment and then uses an unsupervised anomaly detection model to spot any activity that deviates from that norm.

The project demonstrates a realistic, iterative machine learning workflow, including advanced feature engineering, model tuning, and debugging. A live simulation of the honeypot detecting a malicious log entry injected into a stream of normal, AI-generated traffic. The project runs in three stages. Run these commands in order from the root directory.

Figure 1: Designing and Evaluating a Flexible and Scalable HTTP Honeypot Platform ... Stage 2: Train the anomaly detection model. (Note: The first time you run this, it will download the pre-trained Sentence Transformer model, which is a few hundred MB). This creates the necessary model files in the models/ directory, which are ignored by Git.

Watch the terminal to see the stream of normal logs followed by the detection of an injected malicious log. Internet → Honeypot Listener (decoy ports) → Logger (local file-based) → Dashboard (not included in this codebase) Figure 2: Figure 1 from Malware Detection Using Honeypot and Machine Learning ... This repository is intended for research and lab use only. Running a honeypot can expose your system to adversarial input and risk. Deploy onto isolated infrastructure, do not use on production networks without proper controls.

Generative AI-driven Honeypot for cybersecurity. Simulates realistic server logs with AI and detects intrusions using unsupervised anomaly detection (Isolation Forest + NLP embeddings). Showcases synthetic data generation, feature engineering, and end-to-end ML workflow. A modular Python-based threat intelligence toolkit for analysing HTTP honeypot logs. Built for real-world deployment as part of a deception technology engagement at a critical industrial infrastructure client.

Each script is standalone and targets a specific aspect of attacker behaviour: event classification, credential harvesting, geolocation, endpoint targeting, and temporal analysis. Figure 3: Python Streamlit web app dashboard for Machine Learning Model project ... Extracts from POST body parameters ( password= , pass= , username= , user= , etc.). Filters out path traversal and exploit garbage automatically.

All scripts share a consistent regex-based detection engine. A log line is classified as an attack if it matches any of the following patterns: ANSI escape sequences are stripped from all log lines before processing to handle coloured terminal output formats. Dhruv Chopra Associate – Deception Technology, C3iHub @ IIT Kanpur github.com/thisisdhruvchopra Python toolkit for HTTP honeypot log analysis - classifies attack events by severity, geolocates threat actors, extracts credential attempts, and generates weekly threat intelligence reports for critical infrastructure deployments.

Automatically discover and classify different types of attacker behavior (e.g., port scanning, login brute-forcing, web scanning) by applying unsupervised clustering to honeypot logs, providing actionable intelligence for security operations teams. - Threat Intelligence : Automatically categorize attacker behaviors without prior knowledge of attack signatures - Pattern Recognition : Identify emerging attack patterns and group similar campaigns together - Strategic Defense : Make data-driven decisions about security controls based on observed attacker clusters - Alert Reduction : Focus on behavioral patterns rather than individual alerts to reduce noise - Proactive Defense : Understand attacker methodologies before they impact production systems - Resource Allocation : Prioritize security investments based on most common attack patterns - Threat Hunting : Identify novel attack patterns that bypass signature-based detection - Log Entries : Realistic honeypot connection attempts with source IPs, ports, protocols, and messages - Behavioral Patterns : Distinct patterns per attack type (many ports vs single port, different error messages) - Scale : Multiple source IPs with varying numbers of connection attempts per attacker Generate synthetic honeypot logs with realistic attack patterns for behavioral analysis.

ssh_auth_failures=('message', lambda x: x.str.contains('Authentication failed').sum()), - Cluster Separation : Clear distinction between behavioral patterns in cluster centroids - Elbow Method Validation : Clear optimal k selection matching expected number of behaviors - Feature Interpretability : Logical cluster centroids reflecting attack behavior patterns - Real Honeypot Integration : Connect with actual honeypot systems (Cowrie, DionaeaArtifact) - Anomaly Detection : Identify completely novel attack patterns outside known clusters - Multi-source Integration : Combine honeypot data with network traffic and endpoint logs This project demonstrates practical application of unsupervised learning for cybersecurity intelligence, providing both automated threat categorization and strategic insights for security operations teams.

In cybersecurity, a honeypot is a decoy system that’s designed to attract and then detect potential attackers attempting to compromise the system. Just like a pot of honey sitting out in the open would attract flies. Think of these honeypots as security cameras for your system. Just as a security camera helps us understand who's trying to break into a building and how they're doing it, these honeypots will help you understand who's trying to attack your system and what techniques they're using.

By the end of this tutorial, you'll be able to write a demo honeypot in Python and understand how honeypots work. Before we start designing our own honeypot, let’s quickly understand their different types: Production Honeypots: These types of honeypots are placed in an actual production environment and are used to detect actual security attacks. They are typically simple in design, easy to maintain and deploy, and offer limited interaction to reduce risk.

Research Honeypots: These are more complex systems set up by security researchers to study attack patterns, perform empirical analysis on these patterns, collect malware samples, and understand new attack techniques that aren’t discovered previously. They often emulate entire operating systems or networks rather than behaving like an application in the production environment.

2. System Architecture & Core Working Principles For this tutorial, we will be building a medium-interaction honeypot that logs connection attempts and basic attacker behavior. Let’s begin by setting up your development environment in Python. Run the following commands: We will be sticking to the built in libraries so won’t be needing to install any external dependencies. We will be storing our logs in the honeypot_logs directory.

This class has a lot of important information in it, so let’s go over each function one by one. The __init__ function records the ip and port numbers on which we’ll host the honeypot, as well as the path / filename of the log file. We will also be maintaining a record of the total number of active connections we have to the honeypot. The log_activity function is going to receive the information about the IP, the data, and the port to which the IP attempted a connection. Then we’ll append this information to our JSON-formatted log file.

The handle_connection function is going to mimic these services that will be running on the different ports we have. We will have the honeypot running on ports 21, 22, 80 and 443. These services are for FTP, SSH, HTTP and the HTTPS protocol, respectively. So any attacker attempting to interact with the honeypot should expect these services on these ports. To mimic the behavior of these services, we’ll use the service banners that they use in reality. This function will first send the appropriate banner when the attacker connects, and then receive the data and log it. The honeypot will also send a fake response “ Command not recognized ” back to the attacker.

Now let’s implement the network listeners that will be handling the incoming connections. For this, we’ll be using simple socket programming. If you aren’t aware of how socket programming works, check out this article that explains some concepts related to it. The start_listener function will start the server and listen on the provided port. The bind_ip for us is going to be 0.0.0.0 which indicates that the server will be listening on all network interfaces.

Now, we will handle each new connection in a separate thread, since there could be instances where multiple attackers attempt to interact with the honeypot or an attacking script or tool is scanning the honeypot. If you aren’t aware of how threading works, you can check out this article that explains threading and concurrency in Python. This function instantiates the Honeypot class and starts the listeners for each of our defined ports (21,22,80,443) as a separate thread. Now, we’ll keep our main thread that is running our actual program alive by putting it in an infinite loop. Put this all together in a script and run it.

Now let’s try to simulate some attack scenarios and target our honeypot so that we can collect some data in our JSON log file. Realistic attack patterns: The simulator will simulate common attack patterns like port scanning, brute force attempts, and service-specific exploits. Variable intensity: The simulator will adjust the intensity of the simulation to test how your honeypot handles different loads.

Several attack types: It will demonstrate different types of attacks that real attackers might attempt, helping you understand how your honeypot responds to each. Concurrent connections: The simulator will use threading to test how your honeypot handles multiple simultaneous connections.

We have a lot going on in this simulation script, so let’s break it down one by one. I’ve also added comments for every function and operation to make this a bit more readable in the code. We first have our utility class called the HoneypotSimulator . In this class, we have the __init__ function that sets up the basic configuration for our simulator. It takes two parameters: a target IP address (defaulting to localhost ) and an intensity level (defaulting to "medium").

We also define three important components: the target ports to probe (common services like FTP, SSH, HTTP), attack patterns specific to each service (like login attempts and commands), and intensity settings that control how aggressive our simulation will be through thread counts and timing delays. The simulate_connection function handles individual connection attempts to a specific port. It creates a socket connection, tries to get any service banners (like SSH version information), and then sends appropriate attack commands based on the service type. We have added error handling for common network issues and also added realistic delays between commands to mimic human interaction.

Our simulate_port_scan function acts like a reconnaissance tool, that will systematically chec each port in our target list. It's similar to how tools like nmap work – going through ports one by one to see what services are available. For each port, it calls the simulate_connection function and adds small random delays to make the scan pattern look more natural. The simulate_brute_force function maintains lists of common usernames and passwords, attempting different combinations against services like FTP and SSH. For each attempt, it creates a new connection, sends the login credentials in the correct format for that service, and then closes the connection. This helps us to test how well the honeypot detects and logs credential stuffing attacks.

The run_continuous_simulation function runs for a specified duration, randomly choosing between different attack types like port scanning, brute force, or specific service attacks. It uses Python's ThreadPoolExecutor to run multiple attacks simultaneously based on the specified intensity level. Finally, we have the main function that provides the command-line interface for the simulator. It uses argparse to handle command-line arguments, letting users specify the target IP, intensity level, and duration of the simulation. It creates an instance of the HoneypotSimulator class and manages the overall execution, including proper handling of user interruptions and errors.

After putting the simulator code in a separate script, run it with the following command: Since we are running the honeypot as well as the simulator on the same machine locally, the target will be localhost . But it can be something else in a real scenario or if you are running the honeypot in a VM or a different machine – so make sure you confirm the IP before running the simulator.

Let’s quickly write a helper function that will allow us to analyze all the data collected by the Honeypot. Since we’ve stored this in a JSON log file, we can conveniently parse it using the built-in JSON package. You can place this in a separate script file and call the function on the JSON logs. This function will provide us comprehensive insights from the JSON file based on the data collected.

Our analysis begins by grouping the data into several categories like IP-based statistics, port targeting patterns, hourly attack distributions, and payload characteristics. For every IP, we are tracking total attempts, first and last seen times, targeted ports and unique payloads. This will help us build unique profiles for attackers. We also examine port-based attack patterns here that monitor for most frequently targeted ports, and by how many unique attackers. We also perform an attack sophistication analysis that helps us identify targeted attackers, considering factors like ports targeted and unique payloads used. This analysis is used for separating simple scanning activities and sophisticated attacks.

Temporal analysis helps us to identify patterns in hourly attack attempts revealing patterns in attack timing and potential automated targeting campaigns. Finally, we publish commonly seen payloads to identify commonly seen attack strings or commands. While deploying this honeypot, make sure you consider the following security measures: Run your honeypot in an isolated environment. Typically inside a VM, or on your local machine that is behind a NAT and a firewall.

Run the honeypot with minimal system privileges (typically not as root) to reduce risk if compromised. Be cautious with collected data if you plan to ever deploy it as a production-grade or research honeypot as it may contain malware or sensitive information. Implement robust monitoring mechanisms to detect attempts to break out of the honeypot environment.

With this we have built our honeypot, written a simulator to simulate attacks for our honeypot and analyzed the data from our honeypot logs to make a few simple inferences. It is an excellent way to understand both offensive as well as defensive security concepts. You can consider building upon this to create more complex detection systems and think of adding features like: Integration with threat intelligence systems that will perform better inference analysis of these collected honeypot logs Gather even comprehensive logs beyond the IP, port and network data through advanced logging mechanisms Remember that even though honeypots are powerful security tools, they should be a part of a comprehensive defensive security strategy, not the only line of defense.

I hope you learnt about how honeypots work, what is their purpose as well as a bit of Python programming as well! Learn to code for free. freeCodeCamp's open source curriculum has helped more than 40,000 people get jobs as developers. Get started 3. Software Implementation, Frameworks & Protocols The scarcity of real-world attack data significantly hinders progress in cybersecurity research and education. Although honeypots like Cowrie effectively collect live threat intelligence, they generate overwhelming volumes of unstructured and heterogeneous logs, rendering manual analysis impractical. As a first step in our project on secure and efficient AI automation, this study explores the use of AI agents for automated log analysis. We present a lightweight and automated approach to process Cowrie honeypot logs. Our approach leverages AI agents to intelligently parse, summarize, and extract insights from raw data, while also considering the security implications of deploying such an autonomous system. Preliminary results demonstrate the pipeline’s effectiveness in reducing manual effort and identifying attack patterns, paving the way for more advanced autonomous cybersecurity analysis in future work.

The ever-evolving landscape of cyber threats necessitates continuous study and analysis. However, for independent researchers, students, and small organizations, accessing real-world attack datasets is a significant hurdle, as this data is often proprietary. Honeypots provide an effective method for collecting live data by acting as decoy systems designed to be probed and attacked. While successful in data collection, a honeypot can generate hundreds of thousands of log entries daily, creating a secondary problem of data overload. Manually analyzing this data to identify patterns and attacker TTPs (Tactics, Techniques, and Procedures) is a time-consuming and inefficient process.

This study addresses the need for an accessible and automated method to analyze honeypot data. The primary contribution of this work is the design and implementation of a complete approach which uses AI agents. This system automates the entire workflow, from reading raw Cowrie logs to generating high-level summaries and visualizations. The goal is to provide a clear and extensible framework that can empower researchers to quickly gain insights from their honeypot data without requiring a heavy and expensive SIEM (Security Information and Event Management) infrastructure. AI agents are used as a solution.

In the next section, the fundamental concepts of honeypots and AI agents will be given. Then the related works will be given in Section 3. In Section 4, details of the system proposal are given. The implementation of the system is given in Section 5. Results are given and analyzed in Section 6. The conclusion and future works will be given in the last section. A honeypot is a security mechanism intended to lure and trap cyber attackers by mimicking a legitimate target system.

Honeypots are artificial decoy systems that offer significant advantages in cybersecurity. Key advantages include providing real-time threat data, capturing only malicious traffic with zero false positives, revealing attackers’ tactics, techniques, and procedures (TTP), and being used to test IDS/IPS systems. However, if configured incorrectly, they can pose a threat to the actual system and also pose legal risks, such as the attacker claiming to be a victim themselves. Importantly, honeypots are never a sufficient solution on their own and must be used in integration with other security systems.

Cowrie is a popular medium-interaction SSH and Telnet honeypot designed to log brute-force attacks and, more importantly, the shell interactions an attacker performs after gaining access. It outputs these interactions into structured JSON files, which serve as the raw data source for our project. An AI agent is an autonomous system designed to perform tasks by generating and executing its own action plans, leveraging a set of tools to achieve its objectives [ 4 ] . It leverages advanced natural language processing capabilities to comprehend complex goals, break them down into logical, sequential steps, and determine the necessary external tools and when to call them [ 5 ] . While conventional AI systems typically operate within predefined parameters and require explicit instructions for each task, AI agents demonstrate greater autonomy in goal-directed behavior [ 6 ] .

Recent progress in large language models (LLMs), with their enhanced reasoning capabilities, has significantly advanced the development of AI agents. Today’s agents use these powerful LLMs as their central brain, enhancing them with additional components for memory, planning, using tools, and interacting with their environment [ 4 ] . This difference enables us to accomplish more complex tasks, such as implementing highly adaptive and intelligent honeypot systems specifically designed for strategic cyber deception and advanced threat intelligence gathering.

A rule-based system is a simple yet effective form of an expert system that uses a collection of predefined rules to make decisions. In our context, the ”expert” is a security analyst, and the rules are derived from their knowledge of typical attacker behavior. For example, a rule could be: ”IF a session contains the command wget or curl , THEN classify the intent as Malware Deployment .” Our system employs this approach to create a fast and transparent analysis engine.

Song and firends introduce the Kyoto 2006+ dataset, built from real honeypot traffic over three years, to address the limitations of outdated datasets like KDD’99 in evaluating modern network intrusion detection systems [ 1 ] . Another work [ 2 ] presents the HoneyAnalyzer, a tool that analyzes honeyd log data using an RDBMS and visual interface, allowing precise attack signature extraction through algorithms like ”Longest Common Substring”. Another study [ 3 ] evaluates honeypots as reliable tools for collecting real attack data, highlighting their advantage over IDSs in avoiding false positives and enabling deeper analysis through automated malware submission. This study focuses on using AI agents efficiently and securely to implement honeypot data analysis.

The model in this study is based on the Cowrie SSH Honeypot. Main functions of Cowrie honeypot are shown in Fig. 1. Cowrie Honeypot is formed of the data collection, emulation and analysis functions. Data Collection: All session logs (commands typed, IP, timestamp) are collected. SSH/Telnet/HTTP are emulated. Extraction of attack patterns is done in the analysis phase. Alert system gives the appropriate warnings.

In the context of our project, agentic architecture is deployed with two primary objectives: autonomous system hardening and automated attack pattern (TTP) extraction. The agent utilizes advanced NLP capabilities to not only comprehend the attacker’s intent in real-time but to engage them in a dynamic, multi-step dialogue. The agent autonomously and strategically deploys decoy services and credentials to probe the attacker’s tools and techniques. Crucially, every command executed, API call made, and lateral movement attempted by the adversary is perceived by the agent as a state within its environment. The agent’s planning module then reasons about these states, determining which subsequent deceptive action will best expose the next step in the attack chain. Its integrated memory module continuously logs all interaction sequences in a structured format, creating a rich, contextual dataset of the attack. Furthermore, the agent autonomously executes its core function: pattern extraction. With its tool-calling capabilities, it invokes external analytics modules to process the captured data, clustering similar behaviors, and determining the underlying tactics. This transforms raw log data to a structured and labeled dataset, achieving our ultimate goal. This proactive, agent-driven approach allows us to move beyond mere intrusion detection to active threat modeling and system fortification based on the learned attack patterns.

Our honeypot architecture based on Cowrie is shown in Fig 2. Steps of the model is as follows: Ingestion: The script begins by discovering and reading all cowrie*.json log files from a specified logs directory. It handles potential file errors and consolidates all events into a single data structure. Processing: Using the pandas library, the raw list of events is converted into a structured DataFrame. The data is then filtered to isolate command-input events and grouped by the unique session ID provided by Cowrie. This step transforms thousands of individual log lines into coherent attacker sessions, each containing the source IP and a chronological list of executed commands.

Analysis: This is the core of the system. A function named analyze_session_with_rules iterates through the commands of each session. It uses a predefined dictionary of keywords (e.g., wget , ls , rm ) to score each session across several categories (Reconnaissance, Malware Deployment, etc.). Based on these scores, it applies a set of heuristics to classify the session’s primary intent and the attacker’s estimated skill level. It also extracts suspicious artifacts like URLs for the final report.

Reporting & Visualization: The results of the analysis are first compiled into a final DataFrame. This DataFrame is then used to generate the final outputs: a .csv file for data portability, a .html file for easy viewing, and two .png image files containing summary visualizations (a bar chart for intents and a pie chart for skill levels). The proposed system was implemented entirely in Python 3, relying on a few key open-source libraries. Pandas is used for all data manipulation tasks, including reading, filtering, grouping, and structuring the data. Matplotlib & seaborn is used for generating the final data visualizations.

The prototype system is performed on a standard machine (4 CPU cores, 8 GB RAM) and successfully processed over 300,000 log events in a few minutes. This project is published on its project GitHub page ( https://github.com/EfeEmirYuce/Cowrie-Honeypot-Log-Analysis-Engine ).

The system was evaluated using a dataset of 313,412 log events collected from 10 log files. The pipeline successfully identified and grouped 26,368 unique attacker sessions that contained command inputs. The rule-based engine was able to assign a classification to the vast majority of these sessions, demonstrating the system’s ability to handle a significant volume of data and produce a high-level summary.

The application of the analysis process to the dataset yielded insightful results on the nature of attacks targeting the honeypot. The primary finding, as visualized in the generated bar chart, is that the overwhelming majority of attacker sessions were classified with the intent of ”Shallow Probe” representing bots or attackers who perform minimal reconnaissance (1-2 commands) before disconnecting. The second most common intent was ”Malware Deployment”. This indicates that a large amount of attacks are automated and aim to quickly compromise a system to deploy a payload, which is often a cryptocurrency miner or a DDoS bot (see Figure 3). This is further supported by the skill level analysis, which shows that almost all activity is classified as ”Low (Script Kiddie)” or ”Medium (Automated Script)”. This suggests that the honeypot is primarily attracting non-targeted, opportunistic attacks rather than sophisticated, manual operations by skilled adversaries.

The system also generates reports that provide a detailed view of each session, including the source IP and any suspicious URLs extracted, allowing for a deeper dive into specific attack instances. The analysis of our dataset confirmed that the threat landscape for exposed SSH servers is dominated by automated, low-skill attempts to deploy malware.

This study successfully demonstrated the design and implementation of an automated process for analyzing Cowrie honeypot data. The system effectively transforms high-volume, low-level log data into structured reports and easy-to-understand visualizations. The key contribution is a lightweight, transparent, and extensible process that can be used to derive meaningful threat intelligence. This can be easily setup at honeypot deployments without the need for complex infrastructure.

For a future work, several enhancements are planned. The rule-based engine could be made more sophisticated by incorporating more complex TTP signatures. Most significantly, the system can be integrated with a Large Language Model (e.g., Google Gemini) via a framework like LangChain. The deployed agents could be equipped with tools to perform real-time threat intelligence enrichment by querying APIs like AbuseIPDB for IP reputation or VirusTotal for URL analysis, providing a much deeper level of contextual understanding for each attack.

In this tutorial, we’ll build a simplified, AI-flavored SIEM log analysis system using Python. Our focus will be on log analysis and anomaly detection. We’ll walk through ingesting logs, detecting anomalies with a lightweight machine learning model, and even touch on how the system could respond automatically.

This hands-on proof-of-concept will illustrate how AI can enhance security monitoring in a practical, accessible way. Security Information and Event Management (SIEM) systems are the central nervous system of modern security operations. A SIEM aggregates and correlates security logs and events from across an IT environment to provide real-time insights into potential incidents. This helps organizations detect threats faster and respond sooner.

These systems pull together huge volumes of log data — from firewall alerts to application logs — and analyze them for signs of trouble. Anomaly detection in this context is crucial, and unusual patterns in logs can reveal incidents that might slip past static rules. For example, a sudden spike in network requests might indicate a DDoS attack, while multiple failed login attempts could point to unauthorized access attempts.

AI takes SIEM capabilities a step further. By leveraging advanced AI models (like large language models), an AI-powered SIEM can intelligently parse and interpret logs, learn what “normal” behavior looks like, and flag the “weird” stuff that warrants attention. In essence, AI can act as a smart co-pilot for analysts, spotting subtle anomalies and even summarizing findings in plain language. Recent advancements in large language models allow SIEMs to reason over countless data points much like a human analyst would — but with far greater speed and scale. The result is a powerful digital security assistant that helps cut through the noise and focus on real threats.

Python 3.x installed on your system. The code examples should work in any recent Python version. Basic familiarity with Python programming (looping, functions, using libraries) and an understanding of logs (for example, what a log entry looks like) will be helpful. Python libraries: We’ll use a few common libraries that are lightweight and don’t require special hardware: scikit-learn for the anomaly detection model (specifically, we’ll use the IsolationForest algorithm).

A set of log data to analyze. You can use any log file (system logs, application logs, and so on) in plain text or CSV format. For demonstration, we’ll simulate a small log dataset so you can follow along even without a ready-made log file. Let’s set up a simple project structure. Create a new directory for this SIEM anomaly detection project and navigate into it. Inside, you can have a Python script (for example, siem_anomaly_demo.py ) or a Jupyter Notebook to run the code step by step.

Make sure your working directory contains or can access your log data. If you’re using a log file, it might be a good idea to place a copy in this project folder. For our proof-of-concept, since we will generate synthetic log data, we won’t need an external file — but in a real scenario you would. Initialize the environment – If you prefer, create a virtual environment for this project (optional but good practice): Prepare a data source – Identify the log source you want to analyze. This could be a path to a log file or database. Ensure you know the format of the logs (for example, are they comma-separated, JSON lines, or plain text?). For illustration, we will fabricate some log entries.

Set up your script or notebook – Open your Python file or notebook. We’ll start by importing the necessary libraries and setting up any configurations (like random seeds for reproducibility). 4. Practical Deployment, Analysis & Case Studies By the end of this setup, you should have a Python environment ready to run our SIEM log analysis code, and either a real log dataset or the intention to simulate data along with me.

In a full SIEM system, log analysis involves collecting logs from various sources and parsing them into a uniform format for further processing. Logs often contain fields like timestamp, severity level, source, event message, user ID, IP address, and so on. The first task is to ingest and preprocess these logs. If your logs are in a text file, you can read them in Python. For example, if each log entry is a line in the file, you could do: If the logs are structured (say, CSV format with columns), Pandas can greatly simplify reading: This will give you a DataFrame df with your log entries organized in columns. But many logs are semi-structured (for example, components separated by spaces or special characters). In such cases, you might need to split each line by a delimiter or use regex to extract fields. For instance, imagine a log line: This has a timestamp, a log level, a message, and a user. We can parse such lines with Python’s string methods: Now we have structured the logs into a table. In a real scenario, you would continue parsing all relevant fields from your logs (for example, IP addresses, error codes, and so on) depending on what you want to analyze.

With the logs in a structured format, the next step is to derive features for anomaly detection. Raw log messages (strings) by themselves are hard for an algorithm to learn from directly. We often extract numeric features or categories that can be quantified. Some examples of features could be: Event counts: number of events per minute/hour, number of login failures for each user, and so on.

Duration or size: if logs include durations or data sizes (for example, file transfer size, query execution time), those numeric values can be directly used. Categorical encoding: log levels (INFO, ERROR, DEBUG) could be mapped to numbers, or specific event types could be one-hot encoded.

For this proof-of-concept, let’s focus on a simple numeric feature: the count of login attempts per minute for a given user. We’ll simulate this as our feature data. In a real system, you would compute this by grouping the parsed log entries by time window and user. The goal is to get an array of numbers where each number represents "how many login attempts occurred in a given minute." Most of the time this number will be low (normal behavior), but if a particular minute saw an unusually high number of attempts, that’s an anomaly (possibly a brute-force attack).

To simulate, we’ll generate a list of 50 values representing normal behavior, and then append a few values that are abnormally high: Most values are in the single digits, but at the end we have three minutes with 30, 40, and 50 attempts – clear outliers. This is our prepared data for anomaly detection. In a real log analysis, this kind of data might come from counting events in your logs over time or extracting some metric from the log content.

To detect anomalies in our log-derived data, we’ll use a machine learning approach. Specifically, we’ll use an Isolation Forest – a popular algorithm for unsupervised anomaly detection. The Isolation Forest works by randomly partitioning the data and isolating points. Anomalies are those points that get isolated (separated from others) quickly, that is, in fewer random splits. This makes it great for identifying outliers in a dataset without needing any labels (we don’t have to know in advance which log entries are “bad”).

We reshaped login_attempts to a 2D array X with one feature column because scikit-learn requires a 2D array for training ( fit ). We set contamination=0.05 to give the model a hint that roughly 5% of the data might be anomalies. In our synthetic data we added 3 anomalies out of 53 points, which is ~5.7%, so 5% is a reasonable guess. (If you don’t specify contamination, the algorithm will choose a default based on assumption or use a default 0.1 in some versions.) At this point, the Isolation Forest model has been trained on our data. Internally, it has built an ensemble of random trees that partition the data. Points that are hard to isolate (that is, in the dense cluster of normal points) end up deep in these trees, while points that are easy to isolate (the outliers) end up with shorter paths.

Now comes the exciting part: using our trained model to detect anomalies in the log data. We’ll have the model predict labels for each data point and then filter out the ones flagged as outliers. In our case, we expect the anomalies to be the large numbers we inserted (30, 40, 50). The output might look like: Even without knowing anything about “login attempts” specifically, the Isolation Forest recognized those values as out-of-line with the rest of the data.

This is the power of anomaly detection in a security context: we don’t always know what a new attack will look like, but if it causes something to drift far from normal patterns (like a user suddenly making 10 times more login attempts than usual), the anomaly detector shines a spotlight on it. In a real analysis, it’s often useful to visualize the data and the anomalies. For instance, we could plot the login_attempts values over time (minute by minute) and highlight the anomalies in a different color.

In this simple case, a line chart would show a mostly flat line around 3-8 logins/min with three huge spikes at the end. Those spikes are our anomalies. You could achieve this with Matplotlib if you’re running this in a notebook: For text-based output as we have here, the printed results already confirm that the high values were caught. In more complex cases, anomaly detection models also provide an anomaly score for each point (for example, how far it is from the normal range). Scikit-learn’s IsolationForest, for example, has a decision_function method that yields a score (where lower scores mean more abnormal).

For simplicity, we won’t delve into the scores here, but it’s good to know you can retrieve them to rank anomalies by severity. With the anomaly detection working, what can we do when we find an anomaly? That leads us to thinking about automated responses. Detecting an anomaly is only half the battle — the next step is responding to it. In enterprise SIEM systems, automated response (often associated with SOAR – Security Orchestration, Automation, and Response) can dramatically reduce reaction time to incidents.

What could an AI-powered SIEM do when it flags something unusual? Here are some possibilities: Alerting: The simplest action is to send an alert to security personnel. This could be an email, a Slack message, or creating a ticket in an incident management system. The alert would contain details of the anomaly (for example, “User alice had 50 failed login attempts in 1 minute, which is abnormal”). GenAI can help here by generating a clear natural-language summary of the incident for the analyst.

Automated mitigation: More advanced systems might take direct action. For instance, if an IP address is showing malicious behavior in logs, the system could automatically block that IP on the firewall. In our login spike example, the system might temporarily lock the user account or prompt for additional authentication, under the assumption that it might be a bot attack. AI-based SIEMs today can indeed trigger predefined response actions or even orchestrate complex workflows when certain threats are detected (refer to AI SIEM: How SIEM with AI/ML is Revolutionizing the SOC | Exabeam for more information).

Investigation support: Generative AI could also be used to automatically gather context. For example, upon detecting the anomaly, the system could pull related logs (surrounding events, other actions by the same user or from the same IP) and provide an aggregated report. This saves the analyst from manually querying multiple data sources. It’s important to implement automated responses carefully — you don’t want the system to overreact to false positives. A common strategy is a tiered response: low-confidence anomalies might just log a warning or send a low-priority alert, whereas high-confidence anomalies (or combinations of anomalies) trigger active defense measures.

In practice, a AI-powered SIEM would integrate with your infrastructure (via APIs, scripts, and so on) to execute these actions. For our Python PoC, you could simulate an automated response by, say, printing a message or calling a dummy function when an anomaly is detected. For example: While our demonstration is simple, it’s easy to imagine scaling this up. The SIEM could, for instance, feed anomalies into a larger generative model that assesses the situation and decides on the best course of action (like a chatbot Ops assistant that knows your runbooks). The possibilities for automation are expanding as AI becomes more sophisticated.

In this tutorial, we built a basic AI-powered SIEM component that ingests log data, analyzes it for anomalies using a machine learning model, and identifies unusual events that could represent security threats. We started by parsing and preparing log data, then used an Isolation Forest model to detect outliers in a stream of login attempt counts. The model successfully flagged out-of-norm behavior without any prior knowledge of what an “attack” looks like – it purely relied on deviations from learned normal patterns.

We also discussed how such a system could respond to detected anomalies, from alerting humans to automatically taking action. Modern SIEM systems augmented with AI/ML are moving in this direction: not only do they detect issues, but they also help triage and respond to them. Generative AI further enhances this by learning from analysts and providing intelligent summaries and decisions, effectively becoming a tireless assistant in the Security Operations Center.

You can try this approach on real log data. For example, take a system log file and extract a feature like “number of error logs per hour” or “bytes transferred per session” and run anomaly detection on that. Experiment with other algorithms like One-Class SVM or Local Outlier Factor for anomaly detection to see how they compare. 5. Performance Evaluation & Extended Technical Observations Incorporate a simple language model to parse log lines or to explain anomalies. For instance, an LLM could read an anomalous log entry and suggest what might be wrong (“This error usually means the database is unreachable”).

Extend the features: in a real SIEM, you’d use many signals at once (failed login counts, unusual IP geolocation, rare process names in logs, and so on). More features and data can improve the context for detection. Learn to code for free. freeCodeCamp's open source curriculum has helped more than 40,000 people get jobs as developers. Get started An advanced Python-based AI honeypot system designed for comprehensive cyber threat detection and analysis. The system leverages machine learning, website monitoring, and real-time vulnerability scanning to provide robust security insights through a sleek, terminal-styled dashboard.

An advanced Python-based AI honeypot system designed for comprehensive cyber threat detection and analysis. The system leverages machine learning, website monitoring, and real-time vulnerability scanning to provide robust security insights through a sleek, terminal-styled dashboard. Cybersecurity threats are becoming increasingly sophisticated, and cloud infrastructures like AWS are popular targets. This project focuses on building a machine learning model to detect and predict attacks based on real-world honeypot data collected from AWS servers.

Using a carefully designed pipeline — including data preprocessing, feature engineering, model training, hyperparameter optimization, and evaluation — we aim to accurately differentiate between normal and malicious activities. The project demonstrates how traditional machine learning techniques can still achieve high reliability (91%+ accuracy) in a highly sensitive domain like cybersecurity.

Machine learning model for real-time attack detection based on AWS honeypot data, achieving over 91% accuracy using Random Forest and feature engineering techniques The advent of Large Language Models (LLMs) promised to resolve the long-standing paradox in honeypot design: achieving high-fidelity deception with low operational risk. However, despite a flurry of research since late 2022, progress has been incremental, and the field lacks a cohesive understanding of the emerging architectural patterns, core challenges, and evaluation paradigms. To fill this gap, this Systematization of Knowledge (SoK) paper provides the first comprehensive overview of this new domain. We survey and systematize three critical, intersecting research areas: first, we provide a taxonomy of honeypot detection vectors, structuring the core problems that LLM-based realism must solve; second, we synthesize the emerging literature on LLM-honeypots, identifying a canonical architecture and key evaluation trends; and third, we chart the evolutionary path of honeypot log analysis, from simple data reduction to automated intelligence generation. We synthesize these findings into a forward-looking research roadmap, arguing that the true potential of this technology lies in creating autonomous, self-improving deception systems to counter the emerging threat of intelligent, automated attackers.

Cybersecurity honeypots—decoy systems designed to lure and analyze attackers—have long grappled with a fundamental design paradox. The spectrum of honeypots ranges from “low-interaction” systems, which are safe and easy to deploy but simplistic and easily detected, to “high-interaction” systems, which offer high-fidelity engagement at the cost of significant risk and management overhead. For decades, the central goal of honeypot research has been to resolve this tension: to achieve the high fidelity of a real system with the low risk and effort of a simulated one.

The recent advent of Large Language Models (LLMs) appeared to offer a silver bullet. With their uncanny ability to generate convincing text, code, and shell interactions, LLMs promised a paradigm shift for honeypots. The vision was clear: low-interaction decoys could finally be imbued with dynamic, adaptive, and believable personalities, luring attackers into revealing their tools and techniques. 1 1 1 While our focus is on enterprise network server emulation, researchers have explored other creative applications, including honeypots for IoT devices [ 1 ] , systems to misdirect web scrapers [ 2 ] , and chatbots designed to waste the resources of email and phone scammers [ 3 , 4 ] .

This vision spurred a flurry of research over the last two years seeking to integrate LLMs into existing honeypot frameworks. As this work was often performed concurrently, many of these formative papers do not cite or build directly upon each other, creating a fragmented landscape. A comprehensive understanding of how this new field is developing is needed to identify its fundamental limitations and most promising directions. To address this, we provide the first systematic review of LLM-powered honeypots, uniquely situating it between surveys of two critical, flanking topics: honeypot fingerprinting, to understand the core problems of realism, and honeypot data analysis, to understand the ultimate goal of intelligence generation. The goal of this Systematization of Knowledge (SoK) paper is to provide the necessary foundational understanding of this burgeoning research area and to accelerate the development of effective LLM-powered defenses.

This paper is organized as follows. Section II provides background on cybersecurity and honeypots for readers seeking more context. Section III contains our three core systematizations. Section IV presents our synthesized takeaways and V is a forward-looking research roadmap. To situate our work, we conducted a review of recent surveys and Systematization of Knowledge (SoK) papers. Our search on Google Scholar for the terms ("survey" OR "systemization of knowledge" OR "SoK") AND "honeypot" AND ("Large Language Model" OR "LLM") from 2024 to the present yielded no prior SoKs on this specific topic.

The closest works are broader surveys on the impact of LLMs on cybersecurity [ 5 , 6 , 7 ] . While these papers acknowledge honeypots as a potential application, they are discussed only briefly within a much larger context. Our work is therefore complementary, providing a deep and focused systematization solely on this intersection. The most relevant prior survey on modern deception technology is by Javadpour et al. [ 8 ] . This comprehensive work offers a detailed review of cyber deception literature from 2008-2023, establishing a clear baseline for the pre-LLM era.

Our work distinguishes itself from these prior surveys not only through its focused scope but by providing several novel systematizations that structure this emerging field. While Javadpour et al. provide a definitive look at the classic era of deception technology, our work provides the first systematic treatment of the LLM-driven era by contributing the following: We provide a foundational framework for how honeypots are detected, which systematizes the core problems that LLM-based realism must solve, a perspective not offered in prior work.

We synthesize the literature on LLM-powered honeypots into an emerging canonical model, identifying the common architectural patterns (e.g., filter/router logic, state management) that have rapidly evolved. We chart the progression of honeypot log analysis through four distinct phases of maturity, from simple data reduction to the modern goal of operationalizing intelligence with agents.

Finally, we synthesize these findings into a forward-looking research roadmap, identifying high-impact directions such as the need for dynamic evaluation ecosystems and autonomous, self-improving feedback loops. By providing these frameworks, our SoK offers a structured, multi-faceted understanding of the challenges, architectures, and future trajectory of LLM-powered honeypots, a contribution distinct from both the broad cybersecurity surveys and the pre-LLM deception literature.

Our systematization reveals several key findings regarding the current state of the art. Historic honeypot fingerprinting principles often still apply as even with LLM simulation, fundamental limitations will give these systems away to a reasonably intelligent adversary. Architecturally, we identify an emerging canonical model for LLM-honeypots that incorporates components for filtering, state management, and generation, but we note a significant gap in open-source tooling. We find the field has developed a valuable trichotomy of evaluation criteria and methodologies: deceptiveness (human evaluation), accuracy (statistical), overall (real-world deployments). Yet there persists an evaluation paradox: while multiple methodologies exist, their utility is hampered by a real-world ”data desert” of low-sophistication traffic, making it difficult to measure progress against meaningful threats. This leads to our finding that the appropriate target for these systems is not skilled humans, whom they are unlikely to deceive, but the emerging threat of sophisticated automated agents. Finally, we find that while automated threat intelligence generation is now possible, it remains in a nascent, proof-of-concept stage, primarily blocked by the lack of large-scale, labeled training data.

Based on these findings, we propose a forward-looking research roadmap centered on the development of autonomous and self-improving deception systems. Architecturally, we highlight the need for open-source, modular frameworks and research into efficient, privacy-preserving models using techniques like LoRA. We advocate for the creation of a dynamic adversarial research ecosystem to solve the ”data desert” problem and enable meaningful evaluation against the true target: sophisticated, automated attackers. Finally, we outline a path toward true autonomy through the implementation of feedback loops, enabling systems to automatically generate intelligence for security operations centers and reconfigure themselves to adapt to the evolving threat landscape.

The goal of security—viewed broadly, be it for a bike, a home, or a network of computing assets—is to control access, ensure availability, and prevent harm. The practice of security is to create asymmetry, essentially making a breach prohibitively difficult. Implicit in this definition is the realization that no system is 100% secure; for example, in a building, adding doors, locks, thicker walls, and alarm systems cannot guarantee the security of its contents, but may make a successful attack prohibitively difficult or costly.

Cybersecurity is commonly defined as ensuring the confidentiality, integrity, and availability of data and networked computing assets. The practice of cybersecurity begins with strategically designing a network with a good security posture (e.g., protecting important assets via layers of defense with increasing restrictions, employing the principle of least privilege). This is complemented by a ”defense-in-depth” strategy, which uses many heterogeneous tools for prevention (e.g., firewalls, anti-malware), detection (e.g., signature-based or anomaly-based detectors), and management (e.g., vulnerability scanning) [ 9 ] . The overall effect is that well-defended networks enjoy protection against most previously seen attack patterns. This provides asymmetry, as attackers who use known methods have a decreased probability of success and are forced to develop or acquire new attack patterns, which is presumably more costly.

Note that in network audit logs, it is often very difficult and resource-intensive to distinguish malicious from benign activity, except when known attack patterns are flagged by automated detectors. This is partly because widespread alert and logging capabilities produce an enormous volume and variety of data that obscures the activity of a sophisticated attacker. Individual log entries often lack sufficient information, so building a picture of network activity from heterogeneous logs requires a combination of domain expertise and network-specific knowledge—often a slow, labor-intensive process [ 9 , 10 , 11 , 12 , 13 , 14 , 15 ] .

Automation has aided both attackers and defenders. The ease of access to attacking capabilities and the use of automated attacks have increased the scale of threats [ 16 , 17 ] , which in turn decreases the cost for the attacker. On the other hand, automated prevention techniques remain effective against these known, high-volume attacks. This landscape presents two clear questions for defenders: “How can we continually learn new attack patterns?” and “How can we deter attackers from targeting our network?” Frameworks for understanding cyber attacks provide a common language to facilitate attack identification, detection, and threat intelligence sharing. The term TTP (Tactics, Techniques, and Procedures) is common in attack analysis. In this context, tactics are high-level adversarial objectives (e.g., Reconnaissance). Techniques are the methods used to achieve a tactic (e.g., Port Scanning). Finally, procedures are the specific commands or code used to implement a technique (e.g., a specific nmap command).

The Lockheed Martin Cyber Kill Chain [ 18 ] was influential in framing attacks as logical sequences of tactics. More commonly used today is the MITRE ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge) framework [ 19 ] . Released in 2015, MITRE ATT&CK is a globally accessible knowledge base detailing adversary behavior organized into 14 tactics, each with numerous techniques and documented procedures. Unlike the Kill Chain’s linear model, MITRE ATT&CK provides a broad matrix of TTPs based on real-world observations. This organized language allows for the labeling and mapping of attack patterns, and the infrastructure to document and share them assists with attribution and cyber defense.

Honeypots are decoy computing systems designed to enhance security by gathering threat intelligence and occupying attacker resources. Consequently, honeypots have been a defensive tactic in cybersecurity since at least the 1980s [ 20 , 21 ] . By convincing unknowing attackers to interact with them, honeypot systems allow for the direct observation of attackers’ actions, which in turn informs defensive mechanisms. This obviates the tedious process of finding malicious activity in a sea of benign system logs. Furthermore, honeypots are generally isolated environments designed so they can be attacked with no risk to production network assets.

The existence of honeypots also serves to slow cyber attackers. Novel attack patterns are presumably more expensive to develop than reusing existing tools. The potential presence of honeypots incentivizes an attacker to ensure they are not interacting with a decoy before using a valuable or secret technique. Notably, scripts to identify common honeypots exist [ 22 ] , so even automated attacks may avoid uncamouflaged decoys. It is believed that attackers will not only seek to detect honeypots but will also attempt cheaper, known attacks before risking a more expensive, novel one [ 23 ] . All of this slows the attacker, benefiting the defender.

Honeypots are designed to (1) occupy attacker resources and (2) accelerate threat intelligence (e.g., learn TTPs or reduce the time to detect a breach). If an attacker is interacting with a honeypot, they are, for that time, occupied and not attacking a real host. This defensive tactic can be the sole goal of a honeypot. For example, CloudFlare uses generative AI to create decoy web networks to occupy malicious web-scraping bots [ 2 ] . Similarly, researchers have used AI to create fake Active Directory users [ 24 ] or to automatically engage with email and phone scammers to waste their time [ 3 , 4 ] .

In addition to occupying attackers, honeypots are also used to lower the time-to-detection for a cyber breach. Often called “canaries,” these honeypots are designed to entice attacker interaction to discover an active breach faster, thereby preventing further damage [ 16 , 25 ] . Honeypots are also used to gain threat intelligence by directly observing attacker TTPs. Examples include honeypots that capture malware [ 26 ] , distributed networks that gain a panoramic view of attack trends [ 27 ] , and recent designs intended to identify and counter LLM-powered offensive agents [ 28 , 29 ] .

With these goals in mind, honeypots mimic servers with varying degrees of realism and risk. They are conceptualized on a spectrum from low-interaction to high-interaction. Low-interaction honeypots are almost fully simulated with limited capabilities. Because they have little real connectivity, they pose minimal risk to the defender’s network, but they lack the realism to entice sophisticated attacker behavior. Medium-interaction honeypots have some real functionality but are restricted to limit risk. High-interaction honeypots are generally real systems, which require more resources to deploy and entail greater risk. Many templates for these honeypot types are open-sourced [ 30 ] . In general, lower interaction means faster deployment and less risk, but at the cost of being less convincing. Experts have stated that low-interaction honeypots will likely never convince a skilled human attacker [ 31 ] , and our survey (Section III-A ) shows there are many ways to detect a simulated system. The vision driving the integration of LLMs into honeypot systems is to achieve the fidelity of a high-interaction honeypot with the speed, ease, and low risk of a lower-interaction system.

Overall, we posit two observations. Firstly, for a honeypot to succeed, it must be sufficiently convincing to elicit attacks in the wild. Secondly, if one seeks to gain threat intelligence, the data produced must be efficiently converted into actionable information, meaning the cost to the defender must be less than the value of the intelligence gained. Large Language Models (LLMs) are generative machine learning models that produce text and possess an extremely broad knowledge base. They use the transformer architecture [ 32 ] , which utilizes attention mechanisms to effectively capture long-range dependencies between tokens. Tokens can be understood as generalizations of words or sub-words.

LLMs typically undergo at least two primary training phases. The initial phase, pre-training, involves training the model on next-token prediction using vast amounts of text data. The second phase, fine-tuning, adapts the model using curated input-output text pairs, where inputs might be questions or instructions, and outputs are the desired responses. This two-stage process enables LLMs to perform a wide array of tasks, such as text generation, summarization, and code writing.

Prominent examples include OpenAI’s GPT series [ 33 ] , Meta’s Llama family [ 34 ] , and Google’s Gemini models [ 35 ] . Furthermore, specialized LLMs are being developed for security applications, such as the offensive security-focused WhiteRabbitNeo [ 36 ] and research into cybersecurity-tuned LLMs [ 37 ] . The rapid expansion of LLM research is documented in various surveys [ 38 , 39 , 40 ] . LLMs are often augmented with tools, such as a web search capability or a Python interpreter, that allow for greater functionality. Of particular use are retrieval-augmented generation (RAG) systems, in which an LLM queries a vector store database to gain extended context.

Harvested Online Documentation & Reference Sources • https://github.com/haripatel07/ai-honeypot • https://github.com/thisisdhruvchopra/log-analyser • https://anil-sn.github.io/ml-learning/projects/016_Network_Honeypot_Log_Analysis/project.html • https://www.freecodecamp.org/news/build-a-honeypot-with-python/ • https://arxiv.org/html/2509.05306v1 • https://www.freecodecamp.org/news/how-to-create-a-python-siem-system-using-ai-and-llms/ • https://github.com/Deepakkumar200201/AI-Powered-Honeypot-System • https://github.com/aleenagibi/HoneyTrap • https://arxiv.org/html/2510.25939v1