SEEK: Sub-vocabulary Decomposed Texture Key

• The paper introduces SEEK, which decomposes the full vocabulary into disjoint sub-vocabularies to generate equivalent texture keys for enhanced watermark resilience.
• It retains a large watermark window to ensure spoofing resistance while leveraging intra-window redundancy to counter scrubbing attacks.
• Empirical results demonstrate significant improvements in TP@1% and reduced false positive rates under various scrubbing and spoofing scenarios without degrading text quality.

Sub-vocabulary Decomposed Equivalent tExture Key (SEEK) is an LLM watermarking scheme introduced to improve resilience against both scrubbing and spoofing attacks by exploiting redundancy from “equivalent texture keys” while preserving the spoofing resistance associated with larger watermark windows (Shen et al., 8 Jul 2025). It operates within h-gram watermarking, where token generation is biased from a secret signal derived from the previous $h$ tokens, but modifies the standard construction by decomposing the full vocabulary into disjoint sub-vocabularies and assigning each sub-vocabulary its own texture key. The resulting design is intended to break the usual trade-off in which small watermark windows are robust to scrubbing but easy to spoof, whereas large windows are difficult to spoof but fragile under localized edits.

1. Problem setting and motivation

In h-gram watermarking, each new token is biased according to a secret “texture key” derived from the previous $h$ tokens. The cited formulation identifies a window-size trade-off: a larger window $h$ increases sample sparsity $\sim |\mathcal V|^h$, which makes statistical spoofing exponentially harder, but it also makes the watermark more fragile under localized edits because disturbing one token in each window can break the chain; conversely, a small window $h \le 4$ is robust to paraphrasing and scrubbing, but easy to spoof via frequency-based attacks (Shen et al., 8 Jul 2025).

SEEK is proposed as a response to that trade-off. Its central observation is that collisions in the hash space can create redundancy: within a watermark window, multiple tokens may independently suffice to reconstruct the texture key. These tokens are termed “equivalent texture keys.” This redundancy increases scrubbing resilience because an attacker must disturb all texture-key tokens in the window to remove the watermark, while the use of a large $h$ preserves spoofing resistance. The paper characterizes this as a Pareto improvement rather than a simple shift along the existing trade-off frontier (Shen et al., 8 Jul 2025).

A common misunderstanding is to treat SEEK as a small-window watermarking variant. The construction does not reduce $h$ to gain robustness. Instead, it retains large-window behavior for spoofing resistance and introduces redundancy through collisions and sub-vocabulary decomposition. This suggests that the scheme’s novelty lies in decoupling two control axes that are often conflated in earlier h-gram designs: the watermark window $h$ and the collision structure induced by the hash-space size $d$.

2. Equivalent texture keys and the SEEK construction

The formulation begins from KGW-Min, where the texture key at generation step $n$ is

$h$0

with $h$1 a hash and $h$2 a secret scalar (Shen et al., 8 Jul 2025). If the hash-space size $h$3 is small, collisions become likely, and multiple tokens in the window may share the minimal hash value. SEEK formalizes the set of equivalent keys as

$h$4

with expected size

$h$5

The collision mechanism improves scrubbing resilience, but naïvely reducing $h$6 would shrink the number of distinct green-red partitions to $h$7, which degrades text quality. SEEK addresses this by splitting the full vocabulary $h$8 into $h$9 disjoint sub-vocabularies $h$0. Each sub-vocabulary uses its own texture key $h$1 derived from the same hash signature, and each yields a sub-green list $h$2. The final green list is the union

$h$3

Because the sub-vocabularies contribute independently, the partition diversity is restored to $h$4 while frequent equivalent texture keys remain available within each sub-vocabulary (Shen et al., 8 Jul 2025).

For each sub-vocabulary $h$5, the paper defines

$h$6

Each sub-green list $h$7 is then generated by seeding a pseudo-random partition function with $h$8 and biasing tokens in $h$9 by $\sim |\mathcal V|^h$0. The construction uses the same hash signature across sub-vocabularies but changes the keying logic so that membership in each hashed index contributes independently to the final watermark signal.

3. Embedding and detection procedures

During generation, SEEK takes as input a prompt $\sim |\mathcal V|^h$1, hash $\sim |\mathcal V|^h$2, key $\sim |\mathcal V|^h$3, window $\sim |\mathcal V|^h$4, sub-vocab count $\sim |\mathcal V|^h$5, bias $\sim |\mathcal V|^h$6, and LLM $\sim |\mathcal V|^h$7. For each step $\sim |\mathcal V|^h$8, it computes base logits $\sim |\mathcal V|^h$9, computes the hash signature $h \le 4$0, partitions $h \le 4$1 into uniform sub-vocabularies $h \le 4$2, derives $h \le 4$3 for each $h \le 4$4, forms the union green list $h \le 4$5, and biases logits according to

$h \le 4$6

The watermark is therefore embedded directly in the token sampling distribution rather than by post hoc sequence modification (Shen et al., 8 Jul 2025).

Detection re-derives the same sub-green lists from the shared secret logic. For each token, membership in $h \le 4$7 is checked to build a hit vector $h \le 4$8, after which one computes a green-hit count $h \le 4$9 over a sequence of length $h$0. The basic statistic under $h$1 is

$h$2

The paper also uses a sliding-window WinMax detector,

$h$3

A sequence is declared watermarked if $h$4, or both $h$5 and $h$6, exceed calibrated thresholds (Shen et al., 8 Jul 2025).

This embedding-detection pipeline preserves the operational structure of h-gram watermarking while changing the internal key schedule. A plausible implication is that SEEK can be understood as a structural modification of green-list generation rather than a replacement of the standard detector family.

4. Theoretical properties and parameterization

The paper gives a lower bound on the probability of at least one collision: $h$7 and uses this to motivate the dependence of redundancy on the hash-space size $h$8 (Shen et al., 8 Jul 2025). In the reported interpretation, $h$9 governs spoofing resistance because sample complexity scales as $h$0, whereas $h$1 governs scrubbing redundancy because smaller $h$2 induces more hash collisions, more equivalent keys, and higher post-scrub $h$3-scores.

The design therefore separates two parameter roles. Larger $h$4 lowers spoofing false-positive rate, while smaller $h$5 raises the number of equivalent keys. This decomposition is constrained by output quality, because partition diversity must remain large enough to avoid linguistic degradation. The practical recommendation reported in the paper is that $h$6–16 balances these goals. Two default settings are highlighted: SEEK$h$7 with $h$8 and SEEK$h$9 with $h$0 (Shen et al., 8 Jul 2025).

Appendix E reportedly proves a Pareto improvement guarantee: for any $h$1 and sufficiently large $h$2, the expected number of tokens an attacker must corrupt to erase SEEK’s watermark is strictly larger than for KGW-Min, while statistical spoofing complexity remains governed by $h$3 alone. Within the paper’s framework, this formalizes the claim that SEEK improves scrub resistance without surrendering the large-window protection against spoofing.

5. Empirical evaluation

The scrubbing evaluation uses WikiText, C4-RealNewsLike, and LFQA, with attackers DIPPER-I, DIPPER-II, ChatGPT 3.5, and DeepSeek v3. Reported metrics are AUROC, TP@1%, and TP@5% at FPR $h$4. Relative to KGW-Min$h$5 with $h$6, SEEK reports TP@1% gains of $h$7 on WikiText $h$8, $h$9 on C4 $d$0, and $d$1 on LFQA $d$2. The paper further states that SEEK maintains higher residual $d$3 under paraphrase and scales gracefully with sequence length (Shen et al., 8 Jul 2025).

The spoofing evaluation uses Dolly-CW, MMW-BookReports, MMW-FakeNews, and C4-Eval, under statistics-based attacks such as Steal [38] and distillation-based attacks [26]. Reported metrics include FPR@$d$4, FPR@$d$5, and $d$6-scores. For statistics-based spoofing, the reported gains over KGW-Min$d$7 at FPR@$d$8 are a reduction from $d$9 to $n$0 on Dolly-CW $n$1, from $n$2 to $n$3 on BookReports $n$4, and from $n$5 to $n$6 on FakeNews $n$7. Under distillation-based attacks, SEEK yields the lowest $n$8-scores and FPR@$n$9 across all four benchmarks (Shen et al., 8 Jul 2025).

Generation quality is assessed with perplexity using an OPT-2.7B oracle, P-SP paraphrase similarity, and log-diversity. The reported result is that SEEK remains within $h$00 of KGW baselines, indicating negligible linguistic degradation. Additional ablations in Appendix C vary $h$01 and $h$02, test cross-model generalization on LLaMA2-13B, Mistral-7B, and OPT-6.7B, and examine robustness under beam search, copy-paste attacks, and sliding-window detectors. Across these settings, the reported tendency is consistent with the theory: increasing $h$03 improves spoofing robustness, while decreasing $h$04 increases redundancy against scrubbing (Shen et al., 8 Jul 2025).

6. Interpretation, misconceptions, and terminological disambiguation

The most important conceptual clarification is that SEEK’s robustness does not derive from shrinking the watermark window. The paper explicitly frames small-window schemes as robust to scrubbing but susceptible to statistics-based spoofing. SEEK instead preserves the large-window regime and introduces intra-window redundancy through equivalent texture keys. This distinction is essential for interpreting the reported Pareto improvement: the scheme is not trading spoofing robustness for scrub robustness, but attempting to retain both within one parameterization (Shen et al., 8 Jul 2025).

A second clarification concerns the term “texture key.” In SEEK, the expression refers to a watermarking keying mechanism derived from hashed token histories. It is unrelated to the UV-domain texture vocabulary used in human-avatar modeling, such as TexVocab, which back-projects multi-view RGB videos to SMPL UV space, constructs pose-texture pairs, and synthesizes dynamic appearance through body-part-wise interpolation (Liu et al., 2024). The lexical overlap can be misleading, but the underlying objects are different: SEEK’s “texture key” is a sequence-conditioned watermark signal, whereas TexVocab’s “texture vocabulary” is an image-aligned representation for animatable avatars.

Within the watermarking literature represented by the cited paper, SEEK occupies a specific position. It remains an h-gram watermarking method; it continues to use green-list biasing and standard $h$05-score-style detection; and it modifies the scheme by decomposing the vocabulary into disjoint sub-vocabularies and exploiting collision-induced redundancy. This suggests that its main contribution is architectural rather than merely parametric: the watermark signal is factored across sub-vocabularies so that collision frequency and partition diversity can be tuned separately.