ReTokSync: Self-Synchronizing Tokenization Disambiguation for Generative Linguistic Steganography

Abstract: Generative linguistic steganography (GLS) enables covert communication by embedding secret messages into the natural language generation process. In practical deployment, however, GLS is vulnerable to tokenization ambiguity: the same surface text may be re-tokenized into a different token sequence at the receiver, breaking the shared decoding state between the communicating parties so that a single local mismatch can propagate into complete extraction failure. Existing solutions either remove ambiguous tokens -- distorting the generation distribution and compromising security -- or preserve the distribution at the cost of substantially reduced embedding capacity or prohibitive runtime overhead. To address this issue, we propose ReTokSync (Re-Tokenization Synchronization), a self-synchronizing disambiguation framework that monitors the receiver-view tokenization during generation and triggers a corrective reset only when ambiguity actually occurs. By confining the effect of tokenization ambiguity to sparse residual bit errors rather than global desynchronization, ReTokSync leaves ambiguity-free positions entirely untouched and remains compatible with the underlying steganographic algorithm. Experiments on both English and Chinese settings show that ReTokSync stays closest to the steganographic baseline in distributional security (zero KL divergence), text quality, embedding capacity, and runtime, while achieving extraction accuracy above 99.7\%. Building on this property, we further develop a two-channel covert communication mechanism in which ReTokSync serves as the primary channel and a reliable auxiliary channel corrects the remaining errors, achieving 100\% end-to-end recovery across all evaluated configurations.

• The paper introduces ReTokSync, a framework that detects and corrects tokenization ambiguity via a self-synchronizing corrective method to prevent error propagation in GLS.
• It leverages online simulation of receiver tokenization to apply local resets, maintaining natural token distribution and minimizing runtime overhead.
• Experimental results demonstrate extraction accuracy above 99.7%, zero KL divergence, and minimal impact on throughput, confirming its practical deployment value.

ReTokSync: Self-Synchronizing Tokenization Disambiguation for Generative Linguistic Steganography

Introduction

Generative linguistic steganography (GLS) leverages the token-level generation process of LLMs to embed covert information in natural text. The primary barrier to reliable GLS in practical settings is tokenization ambiguity: alignment between sender and receiver fails when a generated string is re-tokenized into a different token sequence, causing the decoding states to desynchronize. This phenomenon results in catastrophic error propagation during message extraction and payload recovery. Prior solutions trade off distributional indistinguishability, embedding efficiency, or incur significant runtime penalties, but none eliminate the fundamental problem. This work introduces ReTokSync, a self-synchronizing framework that detects receiver-view tokenization ambiguity online and applies corrective synchronization only when necessary, thus containing ambiguity-induced errors to isolated, local bit flips rather than global extraction failure.

Figure 1: Impact of tokenization ambiguity in generative linguistic steganography. Although ambiguity is pervasive at the sample level, the tokens that trigger it are sparse. Once triggered, a single local mismatch can cause global extraction failure.

Background and Analysis of Tokenization Ambiguity

Modern LLMs employ subword tokenizers (e.g., BPE, SentencePiece) that admit multiple plausible tokenization paths for a given surface string due to vocabulary redundancy and prefix ambiguities. In GLS, any divergence between sender and receiver tokenization collapses synchronization, causing subsequent decoding to follow a trajectory that no longer matches the secret payload embedding plan.

The analysis demonstrates that, although most samples exhibit tokenization ambiguity at some point (i.e., sample-level ambiguity rate is high and increases with sequence length), the events that actually trigger ambiguity are sparse in the token stream—the fraction of tokens responsible for ambiguity remains consistently below 1% (see Figures 4 and 5).

Figure 2: Tokenization-ambiguity frequency analysis on IMDB using Llama-3.1-8B and Qwen3-8B.

Figure 3: Frequency analysis of ambiguity-triggering tokens on IMDB using Llama-3.1-8B and Qwen3-8B.

This sparsity motivates a paradigm shift from preventive, global intervention at each step to an event-driven, local corrective approach.

The ReTokSync Framework

ReTokSync introduces a real-time, sender-side ambiguity monitoring procedure. At each generation step, the algorithm simulates the receiver's tokenization by detokenizing and re-tokenizing the current output. If the receiver-view token sequence diverges from the expected direct concatenation, ambiguity is present. A corrective reset occurs: the sender aligns its message pointer and embedding state with the receiver’s decoding outcome, ensuring both sides remain synchronized for subsequent generation.

Figure 4: Illustration of ReTokSync. When tokenization ambiguity is detected, the sender performs corrective reset to prevent ambiguity-induced errors from propagating to subsequent positions; when no ambiguity occurs, no corrective reset is needed.

By ensuring context alignment, ReTokSync strictly preserves the original conditional generation distribution at every position, resulting in zero Kullback--Leibler divergence relative to the underlying model distribution.

Two-Channel Communication for Continuous Interaction

While ReTokSync effectively localizes ambiguity-induced errors, sparse residual local bit mismatches may persist. To guarantee full end-to-end payload integrity in practical interactive scenarios (such as multi-turn dialogue or messaging), the authors introduce a two-channel communication mechanism. In this design, ReTokSync serves as the high-throughput channel, while an auxiliary channel (based on Syncpool) periodically transmits compact correction messages describing any detected residual errors. This mechanism enables 100% reliable reconstruction with negligible overhead.

Figure 5: An example of covert communication under continuous interaction. ReTokSync is used as the main channel to transmit the primary secret payload, while Syncpool serves as an auxiliary reliable channel that periodically carries correction information in later interaction rounds, enabling accurate end-to-end recovery.

Experimental Evaluation

Comprehensive experiments are conducted on Llama-3.1-8B and Qwen3-8B across English and Chinese IMDB datasets, utilizing Discop as the primary steganographic baseline. Evaluation covers effectiveness (extraction accuracy), distributional security (average and maximum KL divergence, perplexity), and embedding efficiency (bits/token, entropy utilization, and runtime/overhead).

Key findings include:

• ReTokSync achieves extraction accuracy above 99.7% in the single-channel setting, and 100% after grouped correction via the auxiliary channel.
• It maintains zero KL divergence and perplexity indistinguishable from the baseline, indicating no detectable distortion of model statistics.
• Embedding capacity and runtime are virtually unchanged from the underlying scheme (e.g., <2% RTO compared to the baseline), with substantially higher efficiency than Syncpool or stepwise verification methods.
• The sparse local error rate enables correction overhead below 1% of total transmitted bits, regardless of group size or candidate set.

Evaluation against CNN and recurrent steganalysis detectors (FCN, BiLSTM-DENSE, RBILSTMC) on 20k-sample datasets confirms that detection accuracy remains at chance levels, preserving the imperceptibility of stegotext generated using ReTokSync.

Theoretical Implications

The paper provides a formal security proof: under the standard black-box threat model, the transcript distribution induced by ReTokSync is identical to natural model text, yielding exactly zero KL divergence on observable channels. The corrective reset procedure only updates hidden embedding states, without side effects on text distribution. Compositionality of the two-channel mechanism is also established, ensuring that even with auxiliary correction, there is no compromise of security if insertion schedules are payload-independent.

Practical Implications and Future Directions

ReTokSync enables the practical deployment of GLS systems by eliminating the core reliability bottleneck—tokenization-induced desynchronization—without introducing statistical artifacts or runtime penalties. The system is compatible with diverse steganographic algorithms (Discop, Meteor, AC-based, Shimmer, ADG, Framework-based approaches) and robust to the details of underlying tokenization schemes.

Potential lines for future research include:

• Application to broader model families featuring different or adaptive tokenizers, e.g., byte-based or hierarchical tokenization.
• Deployment in adversarial settings where attackers may control or mutate receiver-side tokenization conventions.
• Integration with non-distribution-preserving schemes and evaluation against advanced side-channel attacks.

Conclusion

This work demonstrates that tokenization ambiguity can be contained to sparse, recoverable local errors through online, sender-side ambiguity detection and targeted correction. ReTokSync achieves optimal trade-offs among security, reliability, embedding capacity, and efficiency—qualities unattainable by prior approaches. By providing a strictly distribution-preserving, practical, and extensible framework, this work advances GLS toward reliable real-world deployment and motivates further investigation of self-synchronizing protocols in covert generative modeling.