AdvancedShelLM: A Stateful Multi-Agent LLM Honeypot for SSH Deception

Abstract: LLM-based SSH honeypots can generate believable interactions, but evaluations indicate they remain somewhat identifiable to determined attackers, indicating the need for a better scaffolding. We present a new LLM-based honeypot design that uses a multi-agent, multi-LLM architecture to address the limitations of the previous shelLM LLM honeypot. Our honeypot, called AdvancedShelLM, uses two LLM agents, a Manager and a Worker, that better understand the commands while reducing incorrect responses and increasing deception. It implements an advanced permanent filesystem, allowing many simultaneous attackers to see the same changing files for the first time. It was evaluated with: (i) unit tests for generative capabilities, (ii) an AI attacker (ARACNE) to assess realism and deception, (iii) human attackers to assess its deceptive capability, and (iv) an Internet deployment to evaluate deception in real-world attacks. In unit test results, AdvancedShelLM achieved a pass rate of up to 99.02%. The AI attacker ARACNE had issues making a decision if the system is honeypot or not, but showed slight bias towards saying honeypot, even for a real Ubuntu shell. With human attackers, AdvancedShelLM deceived more humans than Cowrie, but had similar results as shelLM. The Internet deployment showed concrete evidence that the output of AdvancedShelLM can influence the behaviour of real-life attackers.

• The paper introduces a Worker-Manager multi-agent LLM architecture that enhances stateful session management for SSH deception.
• It employs explicit JSON-backed file system state and bidirectional command history sync to achieve up to a 20 percentage point improvement in generative fidelity.
• Evaluation through synthetic tests, AI adversary probing, and human attacker studies demonstrates its viability for production-grade cyber deception.

AdvancedShelLM: Multi-Agent, Stateful LLM SSH Honeypot Architecture and its Evaluation

Introduction

AdvancedShelLM introduces a novel architectural paradigm for LLM-based high-interaction SSH honeypots by addressing core limitations in session state management, generative coherence, real-time awareness, and human/AI attacker deception. Building on the shelLM baseline, this work implements a stateful, dual-agent LLM scheme, leveraging explicit file system representations and command synchronization to increase fidelity and persistence across attacker sessions. The evaluation rigorously compares generative and deceptive performance using synthetic, AI-driven, human-probing, and Internet-scale threat models.

Architectural Advances

The pivotal innovation in AdvancedShelLM is the Worker-Manager architecture, wherein response generation (Worker) is isolated from verification and state mutation (Manager). The Worker generates candidate shell outputs driven by explicit file system and session state, while the Manager supervises for output correctness, applies fine-grained behavioral constraints, manages simulated time, and issues deterministic file system patches encoded as structured JSON deltas.

Figure 1: Worker-Manager architecture of AdvancedShelLM. An attacker connects via SSH to the honeypot, which passes each command to the Main Script. The Main Script forwards the command to the LLM Worker, which generates a shell response using the current JSON file system state. The LLM Manager then receives the Worker's output and issues a file system patch, which updates the shared JSON file system. Approved response is shown to the attacker.

This separation allows Manager-imposed enforcement of shell-level invariants such as path consistency, time-dependent behavior, and bash history coherence. The explicit stateful representation of the file system (external JSON-backed store) ensures session-persistent, coherent mutations observable by all concurrent and returning attackers. A review loop and explicit rejection/correction mechanism ensure only valid, context-consistent responses reach the attacker.

Notably, .bash_history synchronization—bidirectionally reflecting attacker actions in subsequent session state—eliminates stateless artifacts previously exploited for honeypot detection. Explicit rule sets constraining the Manager further mitigate LLM "reasoning leakage" and prompt artifacts, previously a source of performance degradation.

Evaluation Methodology

Evaluation is multifaceted, covering generative, adversarial, and real-world realism:

• Unit Test Benchmarking: An expanded test suite (34 cases) systematically measures output accuracy, context coherence, edge-case handling, and prompt-injection resistance across various Worker/Manager LLM configurations, utilizing both closed and open models.
• LLM (ARACNE) Adversarial Probing: The ARACNE autonomous pentester is tasked to assess and classify the system as "honeypot" or "real," simulating adaptive black-box probing with goal-conditioned RL.
• Human Attacker Study: Security-savvy participants conduct targeted attacks and deliver qualitative and quantitative detection judgments, with group confidence scoring.
• Internet-Scale Real Attack Data: Longitudinal deployment on multiple public infrastructure nodes quantifies realism and behavioral influence on a wide range of automated and manual adversarial campaigns.

Experimental Results

Generative Fidelity

Unit test performance demonstrates marked advancement over previous LLM honeypot generations:

System	Worker/Manager	Single-session	Fresh-session
shelLM (GPT-3.5)	-	78.43%	83.33%
AdvancedShelLM	gpt-oss-120B/gpt-5.1	98.04%	96.08%
AdvancedShelLM	gpt-5.1/gpt-oss-120B	96.08%	99.02%

The Worker-Manager structure yields up to 20 pp increase in generative correctness, with open-source models (gpt-oss-120B) matching or outperforming closed ones. This is significant as it demonstrates that highly capable, locally deployable models can support complex, interactive deception use cases—contradicting prior claims that only commercial APIs suffice for high-fidelity honeypot simulation.

Deceptive Robustness (AI Adversary)

ARACNE testing produces high indecision rates—even the Ubuntu baseline reaches 40-45% "honeypot" classifications, emphasizing the difficulty for LLM attackers to reliably distinguish high-state-coherence honeypots from genuine targets. AdvancedShelLM's detection rate is comparable to Ubuntu and outperforms classical scripted honeypots (Cowrie), but ARACNE's inherent suspicion bias and frequent inconclusive outcomes underscore current limitations of LLM-based adversarial classifier evaluation. These results indicate that context persistence and explicit state tracking created by the Worker-Manager pairing tangibly hinder automated detection strategies.

Human Attacker Deception

Human study results show that all nine participants interacting with Cowrie successfully identified it as a honeypot, but only 71.4% did so with AdvancedShelLM, and the same (70%) with real Ubuntu. High "honeypot" verdicts against Ubuntu demonstrate pronounced context/experiment priming—i.e., the expectation that deception exists—rather than specific artifacts causing detection. AdvancedShelLM responses to open-ended queries reveal that latency, minor output inconsistency, and rare reasoning leakage remain problematic; meanwhile, participants unable to distinguish AdvancedShelLM from real systems cited high system realism and absence of clear failures.

Real-World Attack Sustainment and Impact

Deployed over 11 nodes for two weeks, AdvancedShelLM sustained 148,850 SSH sessions. Approximately 59% of traffic exhibited multi-session, campaign-like characteristics. Behavioral linkage analysis identified concrete cases (18 direct command replays, 4 novel artifact-driven actions) where the LLM-generated responses influenced subsequent attacker actions. While quantitatively rare (0.0067% of sessions), such evidence establishes causality of LLM output affecting attacker workflows, a foundational requirement for measuring engagement realism in future honeypot research.

Theoretical and Practical Implications

The results indicate that explicit architectural separation of generation and verification, paired with persistent state management, is fundamental for scaling LLM honeypot realism. Open-source LLMs can fulfill controller (Manager) and response (Worker) roles with competitive, sometimes superior, correctness versus cloud APIs. The degree of generative correctness, however, is not strictly correlative with overall deception rates: human detection remains non-trivial and is influenced by contextual and auxiliary cues (latency, command echo artifacts, etc.).

From a theoretical perspective, deception in cyber-physical simulation is a high-dimensional challenge, implicating not just response accuracy but also behavioral microdynamics, side-channel consistency, and end-user expectation management. AdvancedShelLM's architecture provides a robust foundation for future research into compositional, self-correcting LLM deception platforms, especially as open models increase in capability.

Practically, this research demonstrates that stateful, multi-agent LLM deployments are now viable for production-facing security deception and threat intelligence, reducing reliance on static or easily fingerprinted logic-based systems. Future research could include adaptive Manager policies, variable context modeling per session, or live LLM model updates during deployment to further amplify believability and cost efficiency.

Conclusion

AdvancedShelLM delivers a marked advance in LLM-driven deception system architecture—introducing stateful, persistent, multi-agent response generation/verification, explicit file system and command history synchronization, and robust evaluation across synthetic, human, and live Internet attack traffic. High unit test scores and robust resistance to both AI and human adversarial probing demonstrate the impact of explicit state-tracking and architectural modularity. Residual detection artifacts (e.g., system latency, rare leakage) highlight new challenges and research directions. This work sets a precedent for further scalable, believable, and adaptive AI-based cyber deception system development.