Token-Level Inference-Time Watermarking

• Token-level inference-time watermarking is a method for embedding secret, statistically-detectable signals into outputs of generative models during decoding.
• It leverages both white-box and black-box strategies—such as logit biasing and candidate resampling—to subtly modify token distributions for robust detection.
• These techniques balance strong detectability with minimal quality distortion, and are applied across language, vision-language, code, and diffusion models.

Token-level inference-time watermarking is a class of algorithmic techniques for embedding statistically detectable and secret-key-dependent signals into the outputs of generative models—most notably LLMs—directly at the stage of autoregressive or locally parallel decoding, without altering model parameters or retraining. These schemes operate by perturbing, selecting among, or compositing next-token outputs during generation, such that the resulting sequence distribution is only subtly modified but remains distinguishable from human or unwatermarked model text using efficient hypothesis tests. Inference-time watermarking encompasses a rich family of methods, including both white-box approaches (which require access to model logits or next-token probabilities) and black-box techniques (which function using only externally observed samples), and has been generalized to vision-LLMs, code generators, and discrete diffusion LMs, as well as robust or multi-bit settings. Key design goals are strong statistical detectability at negligible cost in quality (distortion), compatibility with black-box or API-access settings, and resilience against text-level post-processing or adversarial attacks.

1. Principles and Theoretical Foundations

Inference-time watermarking exploits the statistical structure of token generation to embed auxiliary information—typically via biasing, partitioning, or randomized selection within the next-token distribution. The theoretical analysis is often cast as a hypothesis testing problem: distinguishing between human- or baseline-LM-generated sequences (null, $H_0$) and watermarked sequences (alternate, $H_1$), with key metrics being false positive rate (Type I error), power (sensitivity), and induced distortion (e.g., KL-divergence or perplexity increment) (Bahri et al., 2024, Bai et al., 2024, He et al., 2024, Tsur et al., 13 May 2025, Tsur et al., 6 Jun 2025, Huang et al., 19 Feb 2026). The tradeoff between detection and distortion is often formalized via minimax or optimization-theoretic frameworks, with “distortion-free” methods preserving the original model’s marginal token distribution, while distortion-based approaches allow controlled perturbations for greater signal strength.

Statistically optimal watermark-detection tests are typically based on log-likelihood ratios or “z-scores” over watermark-correlated features (e.g., counts of partitioned [“green”] tokens, or cumulants of key-dependent pseudorandom functions), yielding ROC or AUC curves for empirical benchmarking. Recent development of e-value- and supermartingale-based “anytime-valid” tests allows for sample-efficient early stopping while retaining Type I control (Huang et al., 19 Feb 2026).

2. Core Algorithms and Embedding Procedures

Token-level inference-time watermarking methods can be categorized according to the watermarking mechanism and model access:

2.1 Black-Box Watermarking

Black-box algorithms work solely via output sampling, requiring no access to internal states:

• Candidate-resampling black-box watermark: At each generation step, multiple ($m$) candidate continuations are sampled from the model. Each unique candidate is scored via a key-seeded pseudorandom function applied to context $n$-grams, and a selection rule (maximizing a transformed sum-score) ensures both detection power and preservation of the underlying distribution ("distortion-free"). This approach allows recursive chaining and multi-key nesting, and can be deployed in API-only access scenarios (Bahri et al., 2024).

Setting	Model access	Mechanism	Distortion-free
Black-box	Sample only	PRF-based multi-resampling	Yes

2.2 White-Box and Distribution-Perturbation Watermarking

White-box and output-distribution approaches perturb internal probabilities or logits:

• Logit-bias watermarking (Red-Green, KGW): At each decoding step, the vocabulary is partitioned (often pseudo-randomly, via cryptographic hash of context + secret key) into "green" and "red" subsets. Logits in the green set receive a fixed bonus, and sampling is performed. This alters the marginal distribution (not distortion-free) but achieves strong detection through the fraction of green tokens across a sequence. Detection is via a binomial or $z$-test for green-token over-representation (Liu et al., 8 Jan 2026, Huo et al., 2024).
• Learnable watermarks: Lightweight networks generate context-adaptive token groupings and per-token logit biases, simultaneously optimizing for watermark detectability and semantic similarity to the base text using multi-objective optimization (Huo et al., 2024).
• Distribution shaping: Controlled, learnable perturbations are introduced into marginal token frequencies, aligning the output distribution with a noisy reference, and detection leverages KL-divergence statistics and a log-likelihood ratio test (Bai et al., 2024).
• Optimal mass-transport and coupling: Watermark embedding is formulated as an optimal coupling or optimal transport problem respecting per-token distortion constraints, with closed-form solutions achieving minimax optimal detection–distortion tradeoffs (Tsur et al., 13 May 2025, Tsur et al., 6 Jun 2025, He et al., 2024).

Setting	Model access	Mechanism	Distortion-free
White-box	Logits/prob	Logit-bias/red-green	No
White-box	Logits/prob	Distribution shaping (KL)	No (minimally)
Both	Surrogate	Gumbel-max/sampling coupling	Yes (in some)

2.3 Selection-, Feature-, and RL-based Schemes

• Rejection sampling & feature alignment: Watermarks are embedded by sampling $N$ candidate outputs per unit (e.g., sentences), extracting deterministic features (e.g., latent SAE activations) and selecting outputs whose feature statistics most closely match targets derived from the message and key (multi-bit support). Detection is via alignment of features to key-derived targets, with statistical $t$-tests (Yu et al., 11 Aug 2025).
• RL-based and adaptive methods: A policy model co-optimized (e.g., via GRPO) to choose when, where, and how to embed watermarks, balancing statistical signal with downstream task correctness, as in code generation under strict functional constraints (Guo et al., 16 Aug 2025).

3. Extensions to Specialized Modalities

3.1 Vision-Language and Image Generation

• Vision-grounded watermarking: Recent methods for large vision–LLMs deploy attention-guided or prefix-tuning-based schemes to adaptively assign watermark bias to visually-grounded tokens at each step, rather than indiscriminate pseudo-random partitioning. These dynamic, evidence-calibrated approaches reduce visual hallucinations while maintaining detection power (AUC > 96–99%) (Li et al., 10 Feb 2026, Zheng et al., 12 Jan 2026).
• Autoregressive image generation: Adapting token-level LLM watermarking to image-token VQ models faces a reverse cycle-consistency challenge ("RCC"). This is addressed via dedicated tokenizer–detokenizer finetuning, and by augmenting token watermarks with spatially localized pixel-domain synchronization marks to preserve detectability under neural rearrangement, compression, or geometric transformation (Jovanović et al., 19 Jun 2025).

3.2 Diffusion LLMs

• Discrete diffusion models: Seeding a distribution-preserving Gumbel-max sampler with per-token indices or hashed contexts allows embedding a distortion-free watermark across parallel, iterative denoising updates. Detection is via token-sequence-level statistics computable in an order-agnostic fashion (Bagchi et al., 3 Nov 2025, Raban et al., 18 Jan 2026).
• Order-agnostic left-right watermarking: For diffusion models updating tokens non-sequentially, logit biases are injected based on finalized left and right neighbors, enabling statistical detection via calibrated scoring but requiring text of sufficient length (variance scaling as $1/\sqrt{T}$) (Raban et al., 18 Jan 2026).

4. Performance Guarantees, Efficiency, and Tradeoffs

Watermarking schemes are benchmarked via:

• Detectability metrics: ROC-AUC, TPR at low FPR, sample complexity (tokens needed for reliable detection).
• Distortion metrics: Change in perplexity (PPL), BLEU/ROUGE/task accuracy, human evaluation of fluency/coherence.
• Overhead: Sampling overhead per token (e.g., $m$-fold for candidate-resampling), wall-clock latency increases (typically 0–15% for modern methods).

For example:

Scheme	PPL Change	AUC / Detection	Additional Latency	Reference
Black-box (m=16)	+3.2 ~ +3.9	97–98%	×16 sampling calls	(Bahri et al., 2024)
Adaptive RL (code)	−4.6 points	AUROC 79–83%	+few percent	(Guo et al., 16 Aug 2025)
Vision adaptive	−0.15 to −0.37	AUC >99%	+0.5–1.0 sec (200 img)	(Li et al., 10 Feb 2026)
Feature selection	~0–1%	F1 98–99%, multi-bit	1–2× (sampling N=10–50)	(Yu et al., 11 Aug 2025)

Trade-offs between detection strength and quality preservation can be tuned via logit bias magnitude, candidate pool size $m$, or noise parameters. In principle, increasing watermark strength increases detectability but at elevated risk of degraded fidelity or loss of functional correctness, especially in low-entropy settings or code (Tsur et al., 6 Jun 2025, Guo et al., 16 Aug 2025).

5. Robustness, Chaining, and Limitations

Robustness to text-level attacks (random edits, synonym substitution, paraphrasing, back-translation) varies by watermark type. Semantic- or visually-adaptive methods demonstrate higher resilience under substitution or synonym attacks but are still vulnerable to paraphrase-level obfuscation, where AUC may degrade substantially (e.g., from 99% to ~70–85%) (Li et al., 10 Feb 2026, Liu et al., 8 Jan 2026, Bahri et al., 2024).

Chaining and nesting watermarks via multiple secret keys is feasible (especially in black-box settings) and can marginalize detection decisions using methods such as Fisher’s combination (Bahri et al., 2024).

Known limitations include computational cost for candidate-based methods, strictly required model access for white-box schemes, and detectability degradation under strong text or code transformations. In the case of diffusion LMs, the detection statistic is tied to token-level context, limiting applicability under aggressive positional corruption or segment deletion (Bagchi et al., 3 Nov 2025, Raban et al., 18 Jan 2026). For low-entropy tasks (such as code), optimal transport-based watermarks (HeavyWater, SimplexWater) are recommended for maximal detection at minimal distortion (Tsur et al., 6 Jun 2025).

6. Practical Recommendations and Applications

Best practice selection of watermarking schemes is dictated by model access, quality requirements, and detection overhead constraints. For API-only (black-box) settings, distortion-free candidate-resampling or multi-bit selection methods are indicated. For vision or reasoning LLMs, adaptive, content-aware biasing preserves semantic and visual fidelity. In code-generation, RL-driven policy models address functional correctness. For high security or regulatory applications, chaining, e-value-based anytime-valid detectors, and robust OT-based coupling provide clear, theoretically grounded guarantees (He et al., 2024, Huang et al., 19 Feb 2026, Tsur et al., 13 May 2025).

Principal application domains include provenance detection, IP tracing, model extraction defense, trustworthy content moderation, and regulatory or forensic identification in high-stakes generative AI deployments.