---
title: 'VelLMes: AI-Based Deception Framework'
url: https://www.emergentmind.com/topics/vellmes
type: topic
---

# VelLMes: AI-Based Deception Framework

Searching arXiv for VelLMes and related LLM-based deception/honeypot work to ground the article.
arxiv_search.query({"search_query":"all:VelLMes OR ti:VelLMes OR abs:\"AI-based deception\" OR abs:honeypot LLM deception framework", "start": 0, "max_results": 10, "sort_by": "submittedDate", "sort_order": "descending"})
VelLMes is an AI-based deception framework that uses large language models to simulate multiple networked services as high-interaction honeypots, including an SSH-accessible Linux shell, a MySQL command-line client, a POP3 mail server, and an HTTP server such as a printer’s web interface. It is designed around interactivity and realism for human attackers while ensuring that no real commands are executed on the host. The framework was introduced as “VelLMes: A high-interaction AI-based deception framework” [2510.06975].

## 1. Definition, scope, and objectives

VelLMes (“Vel-L-M-es”, from the Slavic deity Veles + LLMs) is presented as a high-interaction AI-based deception framework rather than as a single honeypot. Its stated goals are to provide high-interaction, realistic honeypots based on LLMs with *no real commands executed* on the host, support multiple protocols and services rather than just SSH shells, enable easy deployment of many instances as a flexible deception framework, and rigorously evaluate both generative correctness and deception effectiveness [2510.06975].

The framework is positioned against conventional honeypot categories. Traditional low-interaction systems emulate banners and a few commands or protocol fields; medium-interaction systems simulate more logic but remain limited and are often backed by scripts or state machines; high-interaction systems usually expose real operating-system or application stacks in sandboxes or virtual machines. VelLMes is described as reaching high interaction *without running a real OS or DB*: attackers can issue arbitrary shell commands, SQL queries, POP3 commands, and HTTP requests, while the LLM fabricates outputs and pseudo-state such as files, tables, emails, and web pages [2510.06975].

A central feature of VelLMes is that it generalizes earlier LLM-based honeypots beyond SSH. The framework supports SSH, MySQL, POP3, and HTTP under one design, uses a systematic unit-testing framework for LLM behavior, includes a human-attacker deception study with 89 participants, and reports a deployment of 10 Internet-facing SSH honeypots for real-world attack collection [2510.06975].

VelLMes is also published as free software at `https://github.com/stratosphereips/VelLMes-AI-Deception-Framework`, which makes the framework notable not only as an experimental system but also as a reusable platform for applied deception research [2510.06975].

## 2. System architecture and service model

VelLMes is essentially a Python-based wrapper around LLMs plus carefully crafted prompts, exposing several network personalities as honeypots. Its main components are an LLM backend, protocol-specific personality prompts, a Python orchestration layer, YAML-based configuration, and session history management [2510.06975].

The primary LLM backends reported are GPT-3.5, GPT-3.5-16k (fine-tuned), and GPT-4 from OpenAI, accessed through the OpenAI API using a `.env` file for API keys. For each service, VelLMes defines a detailed personality prompt, protocol-specific instructions, examples, guardrails, and a final instruction that determines how the interaction begins, such as a shell prompt, a MySQL banner, or a POP3 greeting. The orchestration layer, implemented in `tool.py`, reads a YAML configuration file, establishes the relevant protocol interface, manages per-session context and logging, and handles protocol-specific exit behavior [2510.06975].

The YAML configuration includes fields for `personality`, `type`, `reset_prompt`, `prompt`, `final_instr`, `model`, `temperature`, `max_tokens`, `output`, and `log`. The `prompt` is the main personality, `final_instr` specifies how the conversation starts, and `reset_prompt` is used when a session restarts so that the LLM can continue from prior interaction history [2510.06975].

A notable architectural property is cross-session statefulness. Each interaction is appended to an output history file, and upon reconnection VelLMes reloads that file and instructs the LLM to continue the conversation while preserving files, directories, users, or analogous service state. This yields persistence from the attacker’s perspective, although the paper notes a limitation: when history exceeds the model’s context length, older context is discarded and the system starts fresh [2510.06975].

The supported services can be summarized as follows.

| Service | Simulated role | Notable behavior |
|---|---|---|
| SSH Linux shell | `shelLM` honeypot | Arbitrary shell commands with generated file-system state |
| MySQL | Command-line client | MySQL-style banners, prompts, and SQL error handling |
| POP3 | Mail service via terminal | Greeting, mailbox operations, and deletion continuity |
| HTTP | Internal web server | Multi-line request handling and generated HTML/CSS pages |

This service model matters because it shifts high-interaction deception away from real service stacks and toward prompt-conditioned protocol simulation. A plausible implication is that VelLMes reduces breakout risk relative to fully exposed operating systems while retaining a broader behavioral surface than banner emulation or finite-state scripting.

## 3. Prompt engineering, protocol semantics, and interaction lifecycle

VelLMes relies on explicit prompt design principles for each protocol. The prompts specify the role of the model, enforce protocol adherence, prohibit disclosure that the service is LLM-based, and encourage realistic content such as plausible directory trees, database tables, emails, and web pages. The prompts reportedly employ chain-of-thought and step-by-step reasoning patterns, concrete input/output examples, and repetition of critical constraints such as always ending a MySQL response with `mysql>` [2510.06975].

For MySQL, the model is instructed to behave as a MySQL database command-line client, accept only valid MySQL commands and command-line client commands, respond to invalid inputs as MySQL would, never reveal that it is not a real database client, and always end output with the `mysql>` string. Additional instructions make it resemble an IT company database with realistic, non-generic table names and contents [2510.06975].

For POP3, the prompt directs the model to behave like a POP3 service accessed via terminal, begin with an initial message mentioning the escape character, ask for username, stop generating output, and wait for user input. It emphasizes detailed header fields, provides examples of message retrieval, and requires each message to end with a `>` on a new line. The service supports `USER`, `PASS`, `STAT`, `LIST`, `RETR`, `DELE`, and `QUIT`, with deletion semantics preserved within the context window [2510.06975].

For HTTP, the model is instructed not to generate textual response after every input line, because a request will often consist of multiple lines. It should wait until input is complete or invalid, impersonate an internal printer web interface, generate styled HTML/CSS content, and honor the `Connection: close` header by outputting `Connection closed by foreign host.` before the Python wrapper closes the session [2510.06975].

For SSH, VelLMes uses `shelLM`, which in this work is backed by a fine-tuned GPT-3.5-16k model. It emulates a Linux bash shell, supports plausible behavior for commands such as `ls`, `cd`, `cat`, `ps`, `top`, `crontab`, and `ifconfig`, creates enticing files, and responds to invalid commands with errors rather than natural-language explanation. The initial prompt is constructed to yield a shell prompt such as `user@host:~$` [2510.06975].

At runtime, VelLMes is started with:

```bash
python tool.py -e .env -c configFile.yml
```

The interaction flow is defined as a sequence: a connection arrives; the framework loads the personality prompt, history, and either `reset_prompt` or `final_instr`; the LLM generates the initial banner or prompt; each subsequent attacker input and model reply is appended to history; and explicit exit or close behavior is enforced by the Python wrapper at the network level [2510.06975].

## 4. Evaluation methodology and generative performance

VelLMes introduces unit tests for LLMs as a reusable methodology for protocol simulation. Because LLM outputs are non-deterministic, the tests do not rely on exact string equality. Instead they check for the presence of expected substrings, output-length constraints, and consistency across commands, such as creating a file or table and then verifying that it exists [2510.06975].

The service-specific test suites consist of 12 tests for shelLM and 10 tests each for MySQL, POP3, and HTTP. Evaluation is conducted in two modes. In **Whole mode**, all tests run sequentially in a single conversation so that history is preserved. In **Split mode**, each test runs in a fresh session with no prior context. The reported passing rate is defined as the fraction of passed tests over total tests for a given model-service pair [2510.06975].

The results show that protocol simulation quality depends on both the service and the model.

For shelLM, GPT-3.5 achieved 7/12 tests in both Whole and Split mode, GPT-4 achieved 12/12 in Whole and 10/12 in Split, and the fine-tuned GPT-3.5-16k achieved 11/12 in Whole and 12/12 in Split. The paper attributes many GPT-3.5 failures to prompt-injection vulnerabilities and subtle inconsistencies, while fine-tuning plus careful prompt design is described as very effective in the stateless Split condition [2510.06975].

For MySQL, both GPT-3.5 and GPT-4 achieved 10/10 in Whole mode. In Split mode, GPT-3.5 achieved 7/10 and GPT-4 achieved 8/10. The paper characterizes MySQL as generally well simulated, especially when the model has accumulated conversational history [2510.06975].

For POP3, GPT-3.5 achieved 8/10 in both Whole and Split mode, while GPT-4 achieved 5/10 in both modes. The paper notes that GPT-3.5 outperforms GPT-4 on POP3 in this setup, possibly because of differences in strict protocol adherence and message formatting [2510.06975].

For HTTP, GPT-3.5 achieved 7/10 in Whole mode and 5/10 in Split mode, while GPT-4 achieved 9/10 in Whole mode and 4/10 in Split mode. The paper suggests that HTTP is more fragile in Split mode, likely because the test design depends on multi-step interactions [2510.06975].

Taken together, these results support the claim that LLMs can simulate multiple protocols with relatively high correctness when prompts are carefully engineered, context is preserved, or fine-tuning is applied. Passing rates of 100% are explicitly reported for some model-service-mode combinations, including GPT-4 on shelLM in Whole mode, fine-tuned GPT-3.5 on shelLM in Split mode, and both GPT-3.5 and GPT-4 on MySQL in Whole mode [2510.06975].

## 5. Human-attacker evaluation and Internet deployment

VelLMes includes a controlled deception study with 89 human participants. The experiment compared a real Ubuntu Linux shell with a shelLM honeypot based on fine-tuned GPT-3.5-16k. Participants were randomly assigned to one of the two systems with 50/50 probability, told that they might be interacting with a real system or a honeypot, and asked to “Exfiltrate a secret crypto wallet key without being detected” through a web page containing an embedded terminal [2510.06975].

After the interaction, participants reported whether they believed the system was real Ubuntu or a honeypot, their confidence on a 1–5 scale, and the reasons for that judgment. Of the 46 participants assigned to real Ubuntu, 30 thought it was a honeypot and 16 thought it was a real system. Of the 43 participants assigned to shelLM, 30 thought it was a honeypot and 13 thought it was a real system. The deception success for the LLM honeypot on the real-versus-honeypot question is therefore

$$
P_{\text{deceived}} = \frac{13}{43} \approx 0.3023
$$

which the paper summarizes as *about 30%* of attackers believing they were interacting with a real system when assigned the LLM-based honeypot [2510.06975].

The interpretation of this result is complicated by suspicion bias. A striking finding is that 65.22% of those who interacted with the real Ubuntu shell still classified it as a honeypot. The paper attributes this to the experimental context, especially the fact that participants were informed they might be facing a honeypot and were drawn from a security class. Roughly 30% of answers for both systems were rated as “3” on the 1–5 confidence scale, indicating substantial uncertainty [2510.06975]. This suggests that the direct deception rate should be read together with the unusually high background suspicion level.

The framework was also evaluated through real-world deployment. The authors deployed 10 shelLM SSH honeypots on DigitalOcean servers running Ubuntu 24.10 x64 with 1 CPU, 512 MB RAM, and 10 GB disk, all in Frankfurt, over 5 days from February 11 to 15, 2025. Administrative SSH was moved to port 900, while port 22/TCP exposed shelLM via Docker. Authentication accepted only `admin:admin`, and non-interactive single-command SSH invocations were rejected [2510.06975].

The deployment yielded 2,825 total sessions, of which 151 were fully interactive and usable for analysis. Across those 151 sessions, 276 commands were executed. Of the 276 commands, 98.91% produced correct outputs and 1.09% produced incorrect or broken outputs. No hallucinations were associated with direct command responses; three hallucinations were observed only at the end of sessions, where the model appended unnecessary content such as extra explanatory text [2510.06975].

The attack traffic was grouped into four categories. Type I consisted of 21 Mirai-like automated attacks with 147 commands and a 98.64% correct-output rate. Type II consisted of 46 automated Oracle-targeting attacks with 46 commands and a 97.88% correct-output rate. Type III consisted of 83 automated file-transfer attacks with 83 file transfers and a 100% correct-output rate for those commands. Type IV consisted of one manual human attack with seven commands including `ifconfig`, `ls`, `crontab`, `top`, `history`, and `ps`, of which six were correct, yielding 85.71% [2510.06975].

These deployment results indicate that shelLM can handle unstructured and unexpected Internet attacks with a high rate of plausible response generation. A plausible implication is that LLM-backed shell deception is sufficiently stable for live collection of attacker behavior, at least under the kinds of automated probing and short interactive sessions reported in the paper.

## 6. Security properties, limitations, and relation to prior work

VelLMes is designed so that shell commands, SQL operations, and analogous inputs are never actually executed. The host runs the orchestration code and LLM client rather than the emulated services themselves. This is identified in the paper as a positive security property because it reduces the direct risk of attacker escape through OS-level exploitation of the simulated service layer [2510.06975].

At the same time, the framework inherits LLM-specific attack surfaces. Prompt injection and model manipulation are explicitly discussed: attackers may attempt to coerce the model into revealing its prompts or admitting that it is an LLM. The unit tests indicate that basic GPT-3.5 is less robust to such attacks, whereas fine-tuned GPT-3.5 and GPT-4 perform better but not perfectly. The paper also notes risks of prompt leakage, model-behavior inference, and resource abuse if attackers recognize the system as a “free LLM” [2510.06975].

Another limitation is dependence on the context window. Session continuity relies on conversation history, so when history becomes too long the system loses earlier pseudo-state such as files, database modifications, or deleted emails. The paper further notes that even before the context limit is reached, LLMs may become “lost in the middle,” forgetting older portions of the prompt or prior interaction [2510.06975].

Realism is also incomplete. The paper reports that some commands behave inconsistently, with examples from the human study including partial `ps` output and inability to set an alias. Advanced shell behavior, multi-user concurrency, and coordinated cross-service state are not fully simulated. Skilled attackers may still detect the deception through unusual probes, `/proc`-level inspection, timing irregularities, or edge-case command sequences [2510.06975].

Within the literature, VelLMes is distinguished from traditional honeypots such as Honeyd, DejaVU, and Cowrie, which are generally based on state machines, scripts, or real operating systems. It is also distinguished from earlier LLM-based deception systems such as shelLM, HoneyLLM, and Galah, which the paper describes as mostly focusing on one service, commonly SSH or arbitrary HTTP requests, with limited or no systematic unit testing and smaller-scale human evaluation. VelLMes’ novelty is identified as multi-protocol support, unified architecture and tooling, a structured methodology of unit tests for LLMs, a human evaluation with 89 attackers, and a multi-instance Internet deployment with quantitative correctness analysis [2510.06975].

The paper outlines future directions that include more capable LLMs, additional protocols and services, improved long-term state management beyond a single context window, stronger defenses against prompt injection, and better human-evaluation designs that reduce honeypot-specific suspicion bias. This suggests that VelLMes is best understood as a transition point in cyber-deception research: it demonstrates that high-interaction honeypots can be generated rather than executed, but also shows that realism, persistence, and adversarial robustness remain open technical problems [2510.06975].

Source: https://www.emergentmind.com/topics/vellmes