---
title: Logit-Based Watermarking Overview
url: https://www.emergentmind.com/topics/logit-based-watermarking
type: topic
---

# Logit-Based Watermarking Overview

Searching arXiv for recent papers on logit-based watermarking and closely related LLM/model watermarking schemes.
Logit-based watermarking denotes a family of watermarking methods that treat logits—or probability distributions derived from logits—as the primary carrier of watermark information. In discriminative settings, this means embedding watermark signals into the geometry of the output logit vector rather than only into predicted labels; in autoregressive language models, it means modifying next-token logits in a secret, structured manner so that generated text carries a detectable statistical signature. Across recent work, the topic spans model ownership verification for classifiers, zero-bit provenance watermarking for large language models, and multi-bit payload embedding, with recurring concerns about detectability, imperceptibility, robustness, and the detectability–distortion trade-off [2409.06130], [2607.05694].

## 1. Definition, scope, and threat models

In classifier watermarking, logit-based watermarking is the idea of treating the full logit vector \(f(X)\)—not just the predicted class—as the carrier of watermark information, and designing both the embedding and verification procedures around this richer signal [2409.06130]. The formulation in "On the Weaknesses of Backdoor-based Model Watermarking: An Information-theoretic Perspective" explicitly distinguishes three parties: the **model owner**, who has training data \(D\) and a secret credit \(T\) and runs an embedding algorithm \(\mathcal{E}(D,T)\); the **authority**, which runs a verification algorithm \(\mathcal{V}(f,T)\); and the **adversary**, which attempts to steal or sanitize the model using an attack \(\mathcal{A}(f,D_{\text{aux}})\), where \(D_{\text{aux}}\) is auxiliary in-distribution data [2409.06130].

In large language models, logit-based watermarking operates at the next-token sampling stage. Given a context, the model outputs logits over the vocabulary, these logits are modified according to a keyed rule, and sampling proceeds from the modified distribution. The most common formulation partitions the vocabulary into keyed subsets such as green and red lists, then biases green-token logits upward by a fixed amount \(\delta\), producing a slightly higher rate of green tokens than would occur by chance [2607.05694]. This keyed, token-level perturbation differentiates logit-based watermarking from metadata-based provenance schemes and from post hoc stylometric detectors that do not rely on a secret key [2607.05694].

Threat models are correspondingly diverse. In classifier watermarking, attacks include knowledge distillation, fine-tuning, and pruning, all of which explicitly alter logits or the representations that generate them [2409.06130]. In LLM watermarking, the relevant perturbations include paraphrasing, re-generation, manual editing, truncation, and low-entropy decoding regimes such as code generation and mathematical reasoning, where next-token distributions are sharply peaked and small logit biases may have little effect [2604.22438], [2505.14112]. A recurring practical distinction is between **black-box** access, where only outputs can be queried, and **gray-box** or **white-box** settings, where logits or computation graphs are available for verification [2409.06130].

## 2. Core statistical mechanism: logits as watermark carriers

The central mechanism is a controlled perturbation of the model’s output geometry. In LLMs, if \(L_n \in \mathbb{R}^{|\mathcal{V}|}\) denotes the base logits and \(P_n=\mathrm{softmax}(L_n)\) the next-token distribution, then a canonical green-list watermark replaces \(L_n\) by
\[
\tilde{L}_{n,t} =
\begin{cases}
L_{n,t} + \delta, & t \in \mathcal{V}^g,\\
L_{n,t}, & t \in \mathcal{V}^r,
\end{cases}
\]
and samples from \(\tilde{P}_n=\mathrm{softmax}(\tilde{L}_n)\) [2604.22438]. If \(p_g\) is the pre-watermark probability mass assigned to the green set, then after biasing the green logits by \(\delta\), the green mass becomes
\[
\tilde{p}_g = \frac{e^{\delta} p_g}{e^{\delta} p_g + (1 - p_g)}.
\]
This dependence on \(p_g\) is fundamental: when \(p_g\) is near \(0\) or \(1\), the induced shift is small, whereas moderate \(p_g\) yields a stronger watermark signal [2604.22438].

For classifier models, the signal need not be tied to the argmax class at all. "On the Weaknesses of Backdoor-based Model Watermarking" formalizes the model as \(f:\mathbb{R}^N\to\mathbb{R}^K\) with logits \(f(X)=[f_1(X),\dots,f_K(X)]\), and then introduces **redundant logits**: if \(\mathcal{I}_{\text{top-}k}\) is the set of top-\(k\) logit indices, any \(f_j(X)\) with \(j\notin\mathcal{I}_{\text{top-}k}\) is redundant [2409.06130]. The paper states the information-theoretic approximation
\[
I\big(f_{\mathcal{I}_{\text{top-}k}}(X);Y\big) \approx I\big(f(X);Y\big),
\]
which motivates using non-top-\(k\) logits as watermark channels because manipulating them barely affects the main-task loss or prediction [2409.06130].

The same general idea appears in multibit LLM watermarking, where watermark scores are added to token logits or to quantities equivalent to logits. "Every Bit, Everywhere, All at Once" defines per-token multibit scores \(\tilde{G}_t(u)\) and uses transformations of the form
\[
q(\tilde{G}_t,p_t)_u \propto (p_t)_u \exp(\delta \tilde{G}_t(u)),
\]
or, in its main implementation, a Soft PPL rule that selects the token maximizing \(\tilde{G}'_t(v)+\lambda \log p_t(v)\) [2605.11653]. This suggests a broad unification: logit-based watermarking is less about any specific green/red construction than about embedding keyed structure into the model’s output score landscape.

## 3. Information-theoretic foundations and design principles

Several recent papers give logit-based watermarking an explicitly information-theoretic interpretation. In classifier watermarking, a standard backdoor-style embedding objective,
\[
\min_f \mathcal{L}_{\mathrm{CE}}(f(X),Y)+\delta\,\mathcal{L}_{\mathrm{CE}}(f(X_W),Y_W),
\]
is shown to be equivalent to maximizing
\[
I(s(X);Y)+\delta\,I(s(X_W);Y_W),
\]
where \(s(X)\) is an internal representation and \(X_W,Y_W\) denote watermark trigger samples and labels [2409.06130]. In this framing, robustness means that reducing \(I(s(X_W);Y_W)\) necessarily harms \(I(s(X);Y)\) enough to reduce utility, whereas vulnerability means the two can be decoupled [2409.06130].

This perspective yields an impossibility result for classical out-of-distribution trigger schemes. As
\[
KL[p(X)\,\|\,p(X_W)] \to \infty,
\]
there exist infinitely many functions \(g^*\) that are optimal for the main task yet satisfy \(I(g^*(X_W);Y_W)=0\), implying that an adversary can erase the watermark without sacrificing clean-task information [2409.06130]. The paper therefore argues that robust watermarking should use **in-distribution** triggers and shared representations rather than separated out-of-distribution trigger manifolds [2409.06130].

In LLM watermarking, "Beyond Heuristic Tuning: Power-Calibrated LLM Watermarking" characterizes a different but related trade-off between detection power and distortion. Under its assumptions, the effective green-token probability under watermarking is
\[
\gamma' = \frac{e^{\delta}\gamma}{1+\gamma(e^{\delta}-1)} + O(|V|^{-1/2}),
\]
and the per-token KL distortion is
\[
D_{\mathrm{KL}}(\gamma,\delta) = \delta\,\gamma'(\gamma,\delta) - \log\big(1+\gamma(e^{\delta}-1)\big).
\]
The resulting power approximation is
\[
\pi^*(\gamma,\delta) \approx \Phi\!\left(
\frac{\sqrt{n}(\gamma'-\gamma)-z_{1-\alpha}\sqrt{\gamma(1-\gamma)}}
{\sqrt{c\gamma'(1-\gamma')}}
\right),
\]
which turns hyperparameter selection into a distortion-constrained optimization problem rather than heuristic tuning [2607.05694].

A different information-theoretic direction appears in "Covert Multi-bit LLM Watermarking: An Information Theory and Coding Approach", which models the LLM cover distribution through state variables \(S_t\) representing next-token probability vectors and derives a covert multi-bit capacity
\[
C = \max_{p(u|s),\,W(x|u,s)} \big[ \mathbb{I}(U;X) - \mathbb{I}(U;S) \big]
\]
subject to a channel synthesis constraint preserving the marginal cover distribution [2605.16709]. A plausible implication is that, in sufficiently rich state-aware settings, multi-bit logit-based watermarking can be understood as a rate-distortion-constrained communication problem over the model’s own probability simplex.

## 4. Representative schemes and architectural patterns

A useful way to organize the literature is by the structure of the carrier signal and the decoding objective.

| Scheme family | Carrier structure | Detection target |
|---|---|---|
| Green/red zero-bit LLM watermarks | Keyed token subsets with logit bias \(\delta\) | Watermarked vs unwatermarked text |
| Redundant-logit classifier watermarks | Non-top-\(k\) classifier logits | Model ownership verification |
| End-to-end neural LLM watermarks | Learned context-dependent logit perturbations | Neural watermark classifier |
| Multibit LLM watermarks | Payload-dependent token scores across bits | Message recovery |
| Low-entropy-aware schemes | Entropy-conditioned selective logit bias | More reliable detection in code/math |

In classifier watermarking, the main concrete scheme in the provided material is **In-distribution Watermark Embedding (IWE)**. It constructs a trigger set from normal images and their augmentations,
\[
T=(X_W,Y_W)=\{(x,0)\}_{x\in D'}\cup\{(x_{\text{aug}},1)\}_{x\in D'},
\]
and computes a binary watermark task from partitioned averages of redundant logits using a secret partition key \(\mathcal{K}\subset\{1,\dots,K\}\) of size \(\lceil K/2\rceil\) [2409.06130]. Only logits outside the top-\(k\) set are used for watermark probabilities, which are then fed into a cross-entropy watermark loss [2409.06130].

In LLM watermarking, the canonical line is the **KGW-style** keyed vocabulary partition. "SSG: Logit-Balanced Vocabulary Partitioning for LLM Watermarking" keeps the same basic injection rule but redesigns the vocabulary partition itself: it sorts tokens by logits, groups the top-\(k\) tokens into pairs, and assigns one token from each pair to the green set and one to the red set [2604.22438]. This **Sort-then-Split by Groups** rule

Source: https://www.emergentmind.com/topics/logit-based-watermarking