VelLMes: AI-Based Deception Framework

• VelLMes is an AI-based deception framework that simulates realistic, high-interaction honeypots (SSH, MySQL, POP3, HTTP) without executing real commands.
• It uses LLM-powered protocol-specific personality prompts, a Python orchestration layer, and YAML configurations to ensure plausible simulation and state management.
• Evaluated through controlled human studies and Internet deployment, VelLMes demonstrates robust output generation and effective deception against attackers.

Searching arXiv for VelLMes and related LLM-based deception/honeypot work to ground the article. arxiv_search.query({"search_query":"all:VelLMes OR ti:VelLMes OR abs:\"AI-based deception\" OR abs:honeypot LLM deception framework", "start": 0, "max_results": 10, "sort_by": "submittedDate", "sort_order": "descending"}) VelLMes is an AI-based deception framework that uses LLMs to simulate multiple networked services as high-interaction honeypots, including an SSH-accessible Linux shell, a MySQL command-line client, a POP3 mail server, and an HTTP server such as a printer’s web interface. It is designed around interactivity and realism for human attackers while ensuring that no real commands are executed on the host. The framework was introduced as “VelLMes: A high-interaction AI-based deception framework” (Sladić et al., 8 Oct 2025).

1. Definition, scope, and objectives

VelLMes (“Vel-L-M-es”, from the Slavic deity Veles + LLMs) is presented as a high-interaction AI-based deception framework rather than as a single honeypot. Its stated goals are to provide high-interaction, realistic honeypots based on LLMs with no real commands executed on the host, support multiple protocols and services rather than just SSH shells, enable easy deployment of many instances as a flexible deception framework, and rigorously evaluate both generative correctness and deception effectiveness (Sladić et al., 8 Oct 2025).

The framework is positioned against conventional honeypot categories. Traditional low-interaction systems emulate banners and a few commands or protocol fields; medium-interaction systems simulate more logic but remain limited and are often backed by scripts or state machines; high-interaction systems usually expose real operating-system or application stacks in sandboxes or virtual machines. VelLMes is described as reaching high interaction without running a real OS or DB: attackers can issue arbitrary shell commands, SQL queries, POP3 commands, and HTTP requests, while the LLM fabricates outputs and pseudo-state such as files, tables, emails, and web pages (Sladić et al., 8 Oct 2025).

A central feature of VelLMes is that it generalizes earlier LLM-based honeypots beyond SSH. The framework supports SSH, MySQL, POP3, and HTTP under one design, uses a systematic unit-testing framework for LLM behavior, includes a human-attacker deception study with 89 participants, and reports a deployment of 10 Internet-facing SSH honeypots for real-world attack collection (Sladić et al., 8 Oct 2025).

VelLMes is also published as free software at https://github.com/stratosphereips/VelLMes-AI-Deception-Framework, which makes the framework notable not only as an experimental system but also as a reusable platform for applied deception research (Sladić et al., 8 Oct 2025).

2. System architecture and service model

VelLMes is essentially a Python-based wrapper around LLMs plus carefully crafted prompts, exposing several network personalities as honeypots. Its main components are an LLM backend, protocol-specific personality prompts, a Python orchestration layer, YAML-based configuration, and session history management (Sladić et al., 8 Oct 2025).

The primary LLM backends reported are GPT-3.5, GPT-3.5-16k (fine-tuned), and GPT-4 from OpenAI, accessed through the OpenAI API using a .env file for API keys. For each service, VelLMes defines a detailed personality prompt, protocol-specific instructions, examples, guardrails, and a final instruction that determines how the interaction begins, such as a shell prompt, a MySQL banner, or a POP3 greeting. The orchestration layer, implemented in tool.py, reads a YAML configuration file, establishes the relevant protocol interface, manages per-session context and logging, and handles protocol-specific exit behavior (Sladić et al., 8 Oct 2025).

The YAML configuration includes fields for personality, type, reset_prompt, prompt, final_instr, model, temperature, max_tokens, output, and log. The prompt is the main personality, final_instr specifies how the conversation starts, and reset_prompt is used when a session restarts so that the LLM can continue from prior interaction history (Sladić et al., 8 Oct 2025).

A notable architectural property is cross-session statefulness. Each interaction is appended to an output history file, and upon reconnection VelLMes reloads that file and instructs the LLM to continue the conversation while preserving files, directories, users, or analogous service state. This yields persistence from the attacker’s perspective, although the paper notes a limitation: when history exceeds the model’s context length, older context is discarded and the system starts fresh (Sladić et al., 8 Oct 2025).

The supported services can be summarized as follows.

Service	Simulated role	Notable behavior
SSH Linux shell	shelLM honeypot	Arbitrary shell commands with generated file-system state
MySQL	Command-line client	MySQL-style banners, prompts, and SQL error handling
POP3	Mail service via terminal	Greeting, mailbox operations, and deletion continuity
HTTP	Internal web server	Multi-line request handling and generated HTML/CSS pages

This service model matters because it shifts high-interaction deception away from real service stacks and toward prompt-conditioned protocol simulation. A plausible implication is that VelLMes reduces breakout risk relative to fully exposed operating systems while retaining a broader behavioral surface than banner emulation or finite-state scripting.

3. Prompt engineering, protocol semantics, and interaction lifecycle

VelLMes relies on explicit prompt design principles for each protocol. The prompts specify the role of the model, enforce protocol adherence, prohibit disclosure that the service is LLM-based, and encourage realistic content such as plausible directory trees, database tables, emails, and web pages. The prompts reportedly employ chain-of-thought and step-by-step reasoning patterns, concrete input/output examples, and repetition of critical constraints such as always ending a MySQL response with mysql> (Sladić et al., 8 Oct 2025).

For MySQL, the model is instructed to behave as a MySQL database command-line client, accept only valid MySQL commands and command-line client commands, respond to invalid inputs as MySQL would, never reveal that it is not a real database client, and always end output with the mysql> string. Additional instructions make it resemble an IT company database with realistic, non-generic table names and contents (Sladić et al., 8 Oct 2025).

For POP3, the prompt directs the model to behave like a POP3 service accessed via terminal, begin with an initial message mentioning the escape character, ask for username, stop generating output, and wait for user input. It emphasizes detailed header fields, provides examples of message retrieval, and requires each message to end with a > on a new line. The service supports USER, PASS, STAT, LIST, RETR, DELE, and QUIT, with deletion semantics preserved within the context window (Sladić et al., 8 Oct 2025).

For HTTP, the model is instructed not to generate textual response after every input line, because a request will often consist of multiple lines. It should wait until input is complete or invalid, impersonate an internal printer web interface, generate styled HTML/CSS content, and honor the Connection: close header by outputting Connection closed by foreign host. before the Python wrapper closes the session (Sladić et al., 8 Oct 2025).

For SSH, VelLMes uses shelLM, which in this work is backed by a fine-tuned GPT-3.5-16k model. It emulates a Linux bash shell, supports plausible behavior for commands such as ls, cd, cat, ps, top, crontab, and ifconfig, creates enticing files, and responds to invalid commands with errors rather than natural-language explanation. The initial prompt is constructed to yield a shell prompt such as user@host:~$ (Sladić et al., 8 Oct 2025).

At runtime, VelLMes is started with:

python tool.py -e .env -c configFile.yml

The interaction flow is defined as a sequence: a connection arrives; the framework loads the personality prompt, history, and either reset_prompt or final_instr; the LLM generates the initial banner or prompt; each subsequent attacker input and model reply is appended to history; and explicit exit or close behavior is enforced by the Python wrapper at the network level (Sladić et al., 8 Oct 2025).

4. Evaluation methodology and generative performance

VelLMes introduces unit tests for LLMs as a reusable methodology for protocol simulation. Because LLM outputs are non-deterministic, the tests do not rely on exact string equality. Instead they check for the presence of expected substrings, output-length constraints, and consistency across commands, such as creating a file or table and then verifying that it exists (Sladić et al., 8 Oct 2025).

The service-specific test suites consist of 12 tests for shelLM and 10 tests each for MySQL, POP3, and HTTP. Evaluation is conducted in two modes. In Whole mode, all tests run sequentially in a single conversation so that history is preserved. In Split mode, each test runs in a fresh session with no prior context. The reported passing rate is defined as the fraction of passed tests over total tests for a given model-service pair (Sladić et al., 8 Oct 2025).

The results show that protocol simulation quality depends on both the service and the model.

For shelLM, GPT-3.5 achieved 7/12 tests in both Whole and Split mode, GPT-4 achieved 12/12 in Whole and 10/12 in Split, and the fine-tuned GPT-3.5-16k achieved 11/12 in Whole and 12/12 in Split. The paper attributes many GPT-3.5 failures to prompt-injection vulnerabilities and subtle inconsistencies, while fine-tuning plus careful prompt design is described as very effective in the stateless Split condition (Sladić et al., 8 Oct 2025).

For MySQL, both GPT-3.5 and GPT-4 achieved 10/10 in Whole mode. In Split mode, GPT-3.5 achieved 7/10 and GPT-4 achieved 8/10. The paper characterizes MySQL as generally well simulated, especially when the model has accumulated conversational history (Sladić et al., 8 Oct 2025).

For POP3, GPT-3.5 achieved 8/10 in both Whole and Split mode, while GPT-4 achieved 5/10 in both modes. The paper notes that GPT-3.5 outperforms GPT-4 on POP3 in this setup, possibly because of differences in strict protocol adherence and message formatting (Sladić et al., 8 Oct 2025).

For HTTP, GPT-3.5 achieved 7/10 in Whole mode and 5/10 in Split mode, while GPT-4 achieved 9/10 in Whole mode and 4/10 in Split mode. The paper suggests that HTTP is more fragile in Split mode, likely because the test design depends on multi-step interactions (Sladić et al., 8 Oct 2025).

Taken together, these results support the claim that LLMs can simulate multiple protocols with relatively high correctness when prompts are carefully engineered, context is preserved, or fine-tuning is applied. Passing rates of 100% are explicitly reported for some model-service-mode combinations, including GPT-4 on shelLM in Whole mode, fine-tuned GPT-3.5 on shelLM in Split mode, and both GPT-3.5 and GPT-4 on MySQL in Whole mode (Sladić et al., 8 Oct 2025).

5. Human-attacker evaluation and Internet deployment

VelLMes includes a controlled deception study with 89 human participants. The experiment compared a real Ubuntu Linux shell with a shelLM honeypot based on fine-tuned GPT-3.5-16k. Participants were randomly assigned to one of the two systems with 50/50 probability, told that they might be interacting with a real system or a honeypot, and asked to “Exfiltrate a secret crypto wallet key without being detected” through a web page containing an embedded terminal (Sladić et al., 8 Oct 2025).

After the interaction, participants reported whether they believed the system was real Ubuntu or a honeypot, their confidence on a 1–5 scale, and the reasons for that judgment. Of the 46 participants assigned to real Ubuntu, 30 thought it was a honeypot and 16 thought it was a real system. Of the 43 participants assigned to shelLM, 30 thought it was a honeypot and 13 thought it was a real system. The deception success for the LLM honeypot on the real-versus-honeypot question is therefore

$P_{\text{deceived}} = \frac{13}{43} \approx 0.3023$

which the paper summarizes as about 30% of attackers believing they were interacting with a real system when assigned the LLM-based honeypot (Sladić et al., 8 Oct 2025).

The interpretation of this result is complicated by suspicion bias. A striking finding is that 65.22% of those who interacted with the real Ubuntu shell still classified it as a honeypot. The paper attributes this to the experimental context, especially the fact that participants were informed they might be facing a honeypot and were drawn from a security class. Roughly 30% of answers for both systems were rated as “3” on the 1–5 confidence scale, indicating substantial uncertainty (Sladić et al., 8 Oct 2025). This suggests that the direct deception rate should be read together with the unusually high background suspicion level.

The framework was also evaluated through real-world deployment. The authors deployed 10 shelLM SSH honeypots on DigitalOcean servers running Ubuntu 24.10 x64 with 1 CPU, 512 MB RAM, and 10 GB disk, all in Frankfurt, over 5 days from February 11 to 15, 2025. Administrative SSH was moved to port 900, while port 22/TCP exposed shelLM via Docker. Authentication accepted only admin:admin, and non-interactive single-command SSH invocations were rejected (Sladić et al., 8 Oct 2025).

The deployment yielded 2,825 total sessions, of which 151 were fully interactive and usable for analysis. Across those 151 sessions, 276 commands were executed. Of the 276 commands, 98.91% produced correct outputs and 1.09% produced incorrect or broken outputs. No hallucinations were associated with direct command responses; three hallucinations were observed only at the end of sessions, where the model appended unnecessary content such as extra explanatory text (Sladić et al., 8 Oct 2025).

The attack traffic was grouped into four categories. Type I consisted of 21 Mirai-like automated attacks with 147 commands and a 98.64% correct-output rate. Type II consisted of 46 automated Oracle-targeting attacks with 46 commands and a 97.88% correct-output rate. Type III consisted of 83 automated file-transfer attacks with 83 file transfers and a 100% correct-output rate for those commands. Type IV consisted of one manual human attack with seven commands including ifconfig, ls, crontab, top, history, and ps, of which six were correct, yielding 85.71% (Sladić et al., 8 Oct 2025).

These deployment results indicate that shelLM can handle unstructured and unexpected Internet attacks with a high rate of plausible response generation. A plausible implication is that LLM-backed shell deception is sufficiently stable for live collection of attacker behavior, at least under the kinds of automated probing and short interactive sessions reported in the paper.

6. Security properties, limitations, and relation to prior work

VelLMes is designed so that shell commands, SQL operations, and analogous inputs are never actually executed. The host runs the orchestration code and LLM client rather than the emulated services themselves. This is identified in the paper as a positive security property because it reduces the direct risk of attacker escape through OS-level exploitation of the simulated service layer (Sladić et al., 8 Oct 2025).

At the same time, the framework inherits LLM-specific attack surfaces. Prompt injection and model manipulation are explicitly discussed: attackers may attempt to coerce the model into revealing its prompts or admitting that it is an LLM. The unit tests indicate that basic GPT-3.5 is less robust to such attacks, whereas fine-tuned GPT-3.5 and GPT-4 perform better but not perfectly. The paper also notes risks of prompt leakage, model-behavior inference, and resource abuse if attackers recognize the system as a “free LLM” (Sladić et al., 8 Oct 2025).

Another limitation is dependence on the context window. Session continuity relies on conversation history, so when history becomes too long the system loses earlier pseudo-state such as files, database modifications, or deleted emails. The paper further notes that even before the context limit is reached, LLMs may become “lost in the middle,” forgetting older portions of the prompt or prior interaction (Sladić et al., 8 Oct 2025).

Realism is also incomplete. The paper reports that some commands behave inconsistently, with examples from the human study including partial ps output and inability to set an alias. Advanced shell behavior, multi-user concurrency, and coordinated cross-service state are not fully simulated. Skilled attackers may still detect the deception through unusual probes, /proc-level inspection, timing irregularities, or edge-case command sequences (Sladić et al., 8 Oct 2025).

Within the literature, VelLMes is distinguished from traditional honeypots such as Honeyd, DejaVU, and Cowrie, which are generally based on state machines, scripts, or real operating systems. It is also distinguished from earlier LLM-based deception systems such as shelLM, HoneyLLM, and Galah, which the paper describes as mostly focusing on one service, commonly SSH or arbitrary HTTP requests, with limited or no systematic unit testing and smaller-scale human evaluation. VelLMes’ novelty is identified as multi-protocol support, unified architecture and tooling, a structured methodology of unit tests for LLMs, a human evaluation with 89 attackers, and a multi-instance Internet deployment with quantitative correctness analysis (Sladić et al., 8 Oct 2025).

The paper outlines future directions that include more capable LLMs, additional protocols and services, improved long-term state management beyond a single context window, stronger defenses against prompt injection, and better human-evaluation designs that reduce honeypot-specific suspicion bias. This suggests that VelLMes is best understood as a transition point in cyber-deception research: it demonstrates that high-interaction honeypots can be generated rather than executed, but also shows that realism, persistence, and adversarial robustness remain open technical problems (Sladić et al., 8 Oct 2025).