Efficient Provably Secure Linguistic Steganography via Range Coding

Abstract: Linguistic steganography involves embedding secret messages within seemingly innocuous texts to enable covert communication. Provable security, which is a long-standing goal and key motivation, has been extended to language-model-based steganography. Previous provably secure approaches have achieved perfect imperceptibility, measured by zero Kullback-Leibler (KL) divergence, but at the expense of embedding capacity. In this paper, we attempt to directly use a classic entropy coding method (range coding) to achieve secure steganography, and then propose an efficient and provably secure linguistic steganographic method with a rotation mechanism. Experiments across various LLMs show that our method achieves around 100% entropy utilization (embedding efficiency) for embedding capacity, outperforming the existing baseline methods. Moreover, it achieves high embedding speeds (up to 1554.66 bits/s on GPT-2). The code is available at github.com/ryehr/RRC_steganography.

• The paper introduces Rotation Range-Coding (RRC) that integrates cryptographic token rotation with range coding to ensure provable security and zero KL divergence.
• It demonstrates near-theoretical optimal embedding efficiency (approximately 100% entropy utilization) and high throughput, outperforming previous steganographic methods.
• The approach is model-agnostic and robust against both machine and human steganalysis, paving the way for secure and covert communication.

Efficient Provably Secure Linguistic Steganography via Range Coding

Introduction and Motivation

Linguistic steganography aims to enable covert communication by embedding secret messages in natural-language text such that the resulting output is statistically indistinguishable from normal text and robust to machine and human steganalysis. Historically, achieving both high embedding efficiency (maximal utilization of a cover’s entropy) and provable statistical security—commonly formalized as zero Kullback-Leibler (KL) divergence from the source distribution—has proven challenging. Prevailing approaches based on generative LMs either introduce detectable distributional artifacts or incur significant losses in capacity or speed due to redundancy, grouping, or coupling mechanisms.

This work investigates range coding (RC)—a classic entropy coding approach—as a foundation for linguistic steganography. The authors demonstrate that directly applying vanilla RC, while lossless in theory, induces probability distortions and randomness reuse, both leading to statistical vulnerabilities. They address these deficiencies by formulating Rotation Range-Coding (RRC) steganography, which incorporates a cryptographic rotation step at each generative token, ensuring provable indistinguishability and full entropy utilization while maintaining practical throughput.

Figure 1: The steganographic pipeline: Alice encodes using an LM, PRNG, and RC-based embedding, while Bob performs deterministic extraction given the same context and key.

Background: Security, Imperceptibility, and Coding

The security of a steganographic system is classically measured by the KL divergence $D_{KL}(P_c||P_s)$ between the cover distribution $P_c$ (natural LM output) and the stego distribution $P_s$. Zero divergence means perfect statistical concealment, making detection information-theoretically impossible. Practically, computational security is also formalized: no polynomial-time adversary can distinguish stegotexts from covers with non-negligible advantage, assuming secret keys remain secure.

Existing systems based on LM-guided entropy codes (e.g., arithmetic coding, grouping, distribution copies, sparse sampling) achieve statistical security only by restricting embedding capacity or incurring significant algorithmic overhead. For instance, Huffman or arithmetic coding stepwise construction can misalign message and distribution boundaries, producing tokens with shifted relative probability (see Figure 2).

Figure 2: Illustration of distribution distortion effects: Huffman-based encoding modulates token probabilities, producing detectable artifacts when the original distribution is highly skewed.

Further, attempts to maximize payloads in a statistically secure way suffer trade-offs between capacity and extraction uniqueness; grouping, coupling, or probabilistic-sparse selection often result in empirical utilization rates below the theoretical entropy bound.

Vanilla Range Coding for Steganography: Method and Limits

Vanilla RC proceeds by interpreting the secret message as a decimal or large integer, then iteratively narrowing the interval using the current LM token probabilities, deterministically selecting the narrowing that uniquely identifies the message. Both embedding and extraction are bijective, provided sufficient numerical precision and perfect replay of the LM token distributions.

Figure 3: Visualization of vanilla RC embedding: A 16-bit secret is mapped to a sequence via recursive interval partitioning, terminating when the interval contains only the message’s decimal value.

Yet, the authors demonstrate two major security flaws in this naïve approach:

• Probability distortion: Discrete intervals, especially for high-probability (low-entropy) tokens, cannot perfectly match the real-valued LM distribution; mapped frequencies of token selections systematically deviate, yielding positive KL divergence.
• Randomness reuse: Re-using the message-derived bits for multiple sampling steps generates statistical dependencies, increasing detectability in aggregate outputs.

These vulnerabilities undermine both information-theoretic and computational guarantees regardless of apparent high embedding rates.

Rotation Range-Coding Steganography: Design and Security Analysis

The core innovation is Rotation Range-Coding (RRC), which at each step randomly rotates the representation of the remaining message using key-driven fresh randomness (via PRNG), ensuring that:

• The mapping from possible messages to token intervals at each timestep is uniformly random and injects entropy independently.
• The marginal distribution of generated token sequences at each step is exactly the LM's original output distribution.
• Extraction remains unique, because the process is reversible given the key/seed and the token sequence.

This construction is sampling-equivalent to traditional LM-based text generation, but with hidden information injected in a computationally concealed manner. The security arguments rest on:

• Zero KL divergence: The formal proofs show that, due to uniform randomization and exact interval scaling, the probability of selecting any token at each LM step equals the original softmax probability, yielding $D_{KL} = 0$ for all $t$.
• Computational indistinguishability: Even a polynomial-time adversary observing the stegotext cannot distinguish RRC output from normal LM output, except with negligible probability; this follows from the cryptographic security of the PRNG and the information-theoretic arguments above.

Empirically, RRC demonstrates approximately 100% entropy utilization (bits/token), surpassing all previous provably secure baselines (e.g., Discop, iMEC, SparSamp) that reach only 92–97%, and achieves up to 1554.66 bits/s embedding speed on GPT-2.

Experimental Results: Capacity, Speed, and Imperceptibility

The authors rigorously evaluate RRC against established and state-of-the-art methods—AC, ADG, Meteor, iMEC, Discop, SparSamp—across multiple LM backbones (GPT-2, OPT-1.3b, Llama-2-7b). Key findings include:

• Embedding efficiency: RRC matches or exceeds the mean entropy of the generative process, achieving near-theoretical optimal bits/token embedding rates independently of message length (32–8192 bits).
• Statistical imperceptibility: Both machine learning steganalysis (RoBERTa, BERT) and human evaluators fail to classify stegotext and normal text above chance; median perplexities remain indistinguishable from normal LM outputs.
• Throughput scalability: RRC supports secret messages up to several thousand bits with constant utilization and strong extraction uniqueness, outperforming SparSamp and Discop especially for large payloads and longer generations.
• Model agnosticism: RRC is effective across LMs of vastly different scales and architectures.

Discussion: Practical Implications, Limitations, and Future Directions

The RRC framework reifies the theoretical limits of LM-based steganography, offering a training-free, model-agnostic, and computationally efficient approach with strong security properties. It alleviates the main open challenge in this domain—simultaneous maximization of capacity, speed, and undetectability.

Practical deployment must still address:

• Tokenization consistency: Detokenization or platform-level differences in LM preprocessing/decoding can induce mismatches between Alice and Bob, requiring additional coordination or tokenization-robust protocols.
• Hardware indeterminism: LM output distributions may vary across hardware implementations, introducing rare but potentially catastrophic extraction failures—future systems must account for hardware fingerprinting or execute on controlled environments.
• Policy and dual-use concerns: The generic, undetectable nature of RRC increases the potential for both privacy-preserving and malicious covert communication channels, highlighting the need for parallel research into robust, adversarially-sound steganalysis.

Theoretically, this work draws a sharp boundary: with reliable access to the LM predictions and PRNGs, the optimal, indistinguishable, high-capacity steganographic system is achievable via RRC. Directions for further inquiry include extending RRC to multimodal generation (images, audio), incorporating robustness-to-tokenization and hardware shifts into the framework, and formalizing resistance to adversarial perturbations or partial model access.

Conclusion

This work establishes that combining range coding with cryptographically-sound tokenwise rotation yields a provably secure, high-throughput, and statically imperceptible linguistic steganography system compatible with arbitrary generative LMs. RRC bridges the gap between information-theoretic optimality and practical deployment, setting a strong foundation for both steganography research and secure, private NLP applications (2604.08052).