Steganography Without Modification: Hidden Communication via LLM Seeds

Abstract: We demonstrate that widely deployed LLM inference stacks harbor a steganographic channel that requires no modification to model weights, sampling code, or output distributions. The channel exploits a structural property of deterministic decoding: pseudo-random number generators (PRNGs) used in inverse-transform sampling produce a seed-dependent sequence of token-level probability intervals that can be reconstructed from the generated text alone. A sender encodes a secret message in the PRNG seed before generation; a receiver reconstructs the intervals and recovers the seed, and thus the hidden payload, by exhaustive search over the seed space. We formalize two operational modes. In the known-prompt setting, sender and receiver share the prompt, enabling exact interval reconstruction and perfect seed recovery via forced alignment. In the unknown-prompt setting, only the generated text is available; approximate interval reconstruction combined with a maximum-hit-count scoring strategy still permits reliable recovery from sufficiently long outputs. Extensive experiments across six model families and five heterogeneous text domains show that, in the known-prompt setting, full 32-bit seed recovery from the complete 2^32 candidate space achieves up to 100% accuracy, depending on model and text domain, within 300 tokens and under 35 seconds on a single GPU. In the unknown-prompt setting, recovery reaches near-perfect accuracy at 600-800 tokens in about 12 seconds. We further analyze the influence of prompting strategies, tokenization ambiguities, and sampling hyperparameters on channel reliability. Moreover, we discuss several applications of our results: First, it allows for the steganographic transmission of 32 bits, but also shows that ignorance of the prompt is not a valid security assumption.

• The paper introduces a method for covert communication by encoding 32-bit messages into PRNG seeds during standard LLM token sampling.
• It demonstrates near-perfect seed recovery in both known-prompt and unknown-prompt settings with recovery times under 35 seconds.
• The study highlights that inherent LLM randomness can be exploited for hidden channels, urging revised security protocols in model deployments.

Steganography without Modifying LLMs: Exploiting PRNG Seeds for Covert Communication

Introduction

The paper "Steganography Without Modification: Hidden Communication via LLM Seeds" (2606.09135) systematically analyzes and demonstrates a hitherto unaddressed steganographic channel present in standard LLM inference stacks. Unlike existing approaches that require modifications to model weights or the sampling process, this channel leverages the deterministic behavior of pseudo-random number generators (PRNGs) underlying standard token sampling schemes. The authors formalize two adversarial settings—known-prompt and unknown-prompt—and present exhaustive empirical evidence of practical, high-reliability covert communication enabled by manipulation and recovery of PRNG seeds.

Technical Approach

The channel fundamentally relies on the observation that the sequence of tokens generated by an LLM under deterministic, seeded sampling is uniquely determined by the PRNG seed, the decoding strategy (e.g., top-k), and the prompt. The sender encodes a payload in the 32-bit seed and generates text using standard decoding. The receiver, observing only the output text, reconstructs the PRNG's per-token sample intervals via the LLM's probability distributions and launches a brute-force search over the entire seed space. The statistical rarity of the correct sequence-seed alignment allows for robust seed recovery.

Figure 1: Seed search histogram—almost all of the $2^{32}$ candidate seeds match at chance, with the true seed emerging as a clear statistical outlier, enabling message recovery.

Determinants and Operational Modes

• Known-Prompt Setting: The receiver possesses the exact prompt and decoding configuration. This enables forced alignment, resolving tokenization ambiguities and achieving perfect or near-perfect seed recovery with relatively short outputs.
• Unknown-Prompt Setting: Only the generated output is available. The receiver reconstructs probability intervals from a proxy or empty prompt, using offset-tolerant matching to mitigate tokenization mismatches, which necessitates longer sequences for reliable recovery.

The authors also analyze sources of error: ambiguous tokenization, the presence of no-draw tokens under top-p sampling, and the roles of temperature and top-k filtering in discriminative power and search complexity.

Empirical Evaluation

The practical viability of this channel is supported by experiments spanning six model families and five text domains. The study uses commodity hardware and llama.cpp-compatible quantized models to reflect realistic deployment conditions.

Known-Prompt Recovery

The method achieves up to 100% recovery accuracy for 32-bit seeds with output lengths as short as 300 tokens and search times under 35 seconds. Recovery rate is influenced by the semantic entropy of the output domain—code generation tasks (dominated by deterministic next tokens) are more challenging due to high no-draw token rates, while news and narrative domains yield near-perfect success.

Figure 2: Cumulative recovery fraction per model and domain in the known-prompt setting, illustrating rapid convergence to nearly 100% as token length increases.

Prompt Reconstruction Strategies

Analysis reveals that, for unknown prompts, neutral or empty proxies maximize overlap between reconstructed and true sampling intervals; further semantic reconstruction adds little and can even degrade alignment due to distributional shifts induced by differing prompts.

Influence of Sampling Hyperparameters

Temperature is the dominant hyperparameter: increasing it narrows token selection intervals, enhancing the discriminative power of each position and thus seed recovery accuracy. Top-k has notably less influence.

Figure 3: Seed recovery success as a function of temperature and top-k, with temperature exerting primary impact on discriminability and recovery.

Sequence Length vs. Recovery Accuracy

Recovery in the unknown-prompt regime converges to near-perfect accuracy at 600–800 tokens, with mean search times of approximately 12 seconds for exhaustive enumeration of the 32-bit seed space. Domains with low per-token entropy or high tokenization ambiguity require longer outputs.

Figure 4: Seed recovery accuracy as a function of sequence length in the unknown-prompt setting; most model-task pairs attain full recovery well before 1,000 tokens.

Security Implications

A critical theoretical contribution is the demonstration that security in generative model-based stegosystems should not rely on keeping prompts secret. The results establish that the deterministic mapping from seed to output, plus the ability to reconstruct or approximate probability intervals (even without prompt sharing), suffices for recovery. Therefore, adversarial threat models must assume both model and prompt availability. This finding calls into question the adequacy of some earlier NLP steganography threat assumptions, and extends arguments from the cryptographic steganography literature.

The channel itself is equivalent, under standard formalisms, to an Algorithm Substitution Attack: randomness consumed during unmodified LLM decoding intrinsically encodes up to $\log_2$(seed space) bits, irrespective of detection at the LLM or user level. This channel is perfectly secure, in the information-theoretic sense, as it preserves the output distribution of the model.

Broader Implications and Future Directions

Practically, this channel enables covert transmission of 32-bit messages per generation, which suffices for bootstrapping high-capacity exfiltration (e.g., streaming multiple outputs) or encoding authentication tokens, dataset pointers, or control flags. The natural occurrence of this channel in widely used stacks (llama.cpp, HuggingFace Transformers, vLLM) means that policy and deployment guidelines for LLMs must encompass randomness management—control and isolation of PRNG seeds are now security requirements, not implementation details.

Theoretically, these findings unify neural steganography and cryptographic subliminal channel paradigms, extending seed-based attacks previously demonstrated in diffusion models to the language domain.

Future Research

• Efficient search for higher-capacity (≥ 64-bit) seed spaces (e.g., meet-in-the-middle, lattice reduction techniques)
• Channel adaptation for non-constant-draw samplers (e.g., top-p < 1 deployment)
• Empirical analysis of robustness under perturbation (e.g., paraphrasing, token truncation)
• Systematic study of natural randomness consumption patterns in deployment-scale LLM inference

Conclusion

This work establishes that the deterministic structure of PRNG-based LLM decoding inherently constitutes a covert channel supporting high-reliability steganographic communication, requiring no modifications to the model or inference code. The channel is robust across architectures, tasks, and operational scenarios, and its existence directly challenges prevailing security assumptions in both LLM deployment and stegosystem design. Future theoretical and empirical work should focus on harder-to-control randomness consumption patterns, stronger adversarial models (including active modification), and mitigation strategies integral to model deployment pipelines.

---

Reference: "Steganography Without Modification: Hidden Communication via LLM Seeds" (2606.09135)