---
title: 'PMark: Semantic-Level Watermarking'
url: https://www.emergentmind.com/topics/pmark
type: topic
---

# PMark: Semantic-Level Watermarking

PMark is a semantic-level watermarking method for large language model output in which the sentence, rather than the token, is the fundamental watermark carrier. It was introduced to improve robustness against paraphrasing-style attacks while avoiding the distribution distortion associated with reject-sampling-based semantic watermarking. The method formalizes watermarking through a proxy function framework, uses median-based balanced partitions of sampled sentence candidates, and strengthens evidence by imposing multiple proxy-function constraints, called channels. In its online form, PMark is presented as distortion-free; the paper also introduces an empirically optimized offline variant that removes dynamic median estimation for better sampling efficiency [2509.21057].

## 1. Problem setting and conceptual scope

PMark is situated within semantic-level watermarking, where the next sentence is the unit of control and detection. Let \(\Sigma^*\) denote the set of semantically complete sentences, and let the model induce the next-sentence distribution
$$
P_M(s_{n+1}\mid s_1,\dots,s_n).
$$
The watermarking objective is zero-bit: given a private key \(k\), generate according to
$$
P_M^w(s_{n+1}\mid s_1,\dots,s_n;k),
$$
and later decide whether watermark evidence is present [2509.21057].

The motivation is that token-level watermarks are readily weakened by paraphrasing, whereas sentence-level methods operate in semantic space and are therefore better aligned with meaning-preserving rewrites. PMark’s starting point, however, is that prior semantic watermarking still had three weaknesses: weak or missing formal guarantees, fragility under edits because each sentence often carries sparse evidence, and distribution distortion caused by reject sampling over semantic regions. The paper places SemStamp, k-SemStamp, and SimMark in this class of earlier semantic methods and argues that a stronger theoretical and algorithmic framework is required [2509.21057].

A central formal notion is single-sentence distortion-free watermarking:
$$
P_M(s_{n+1}\mid \bm{\pi})=\sum_{k\in K} P_K(k)\cdot P_M^w(s_{n+1}\mid \bm{\pi};k),
$$
where \(\bm{\pi}=[s_1,\dots,s_n]\). This definition requires that, after averaging over keys, the watermarked distribution match the base model’s next-sentence distribution exactly [2509.21057].

## 2. Proxy functions and semantic channels

PMark’s main abstraction is the proxy function, a scalar-valued map on sentence space:
$$
\mathcal F:\Sigma^* \rightarrow \mathbb R.
$$
This abstraction is used to unify semantic watermarking schemes. In the paper’s analysis, SemStamp uses a locality-sensitive hashing map of the sentence embedding, k-SemStamp uses nearest-cluster assignment, and SimMark uses similarity to the previous sentence. PMark makes the proxy-function view explicit and then builds watermarking and detection around it [2509.21057].

Its own proxy function is defined from a sentence encoder \(\mathcal T(s)\in \mathbb R^d\) and a pivot vector \(v\in \mathbb R^d\):
$$
\mathcal F_v(s)=\langle v,\mathcal T(s)\rangle
= \frac{v\cdot \mathcal T(s)}{\|v\|\cdot \|\mathcal T(s)\|}.
$$
Thus each sentence is projected onto a semantic direction, and the sign or relative position of that projection becomes the watermark carrier [2509.21057].

A channel is one such proxy function defined by one pivot vector. With \(b\) orthogonal pivots \(v^{(1)},\dots,v^{(b)}\), PMark uses
$$
\mathcal F_j(s)=\langle v^{(j)},\mathcal T(s)\rangle,
$$
for \(j=1,\dots,b\). The significance of multiple channels is that each sentence can contribute several pieces of watermark evidence rather than one. The paper interprets this as increasing evidence density, which in turn improves robustness under paraphrasing, word deletion, and synonym substitution [2509.21057].

The paper also uses high-dimensional orthogonality as a practical prior for the offline variant. For random unit vectors \(x,y\in S^{d-1}\), the angle \(\theta\) has density
$$
p_d(\theta)=
\frac{\Gamma\left(\frac d2\right)}
{\Gamma\left(\frac{d-1}2\right)\sqrt{\pi}}
\sin^{d-2}\theta,
\qquad \theta\in[0,\pi].
$$
This is used to motivate the empirical observation that \(\mathcal F_v(s)\) is often concentrated near \(0\), so the proxy-function median is frequently close to \(0\) [2509.21057].

## 3. Generation procedure and the distortion-free theorem

PMark’s online generation algorithm begins by sampling a fixed candidate pool from the model’s natural next-sentence distribution. For a given sentence position, let
$$
W=\{x_1,\dots,x_N\}, \qquad x_i \sim P_M(s\mid \bm{\pi}),
$$
with even \(N\). The method computes proxy-function scores \(f_i=\mathcal F_v(x_i)\), estimates their median \(m_v\), and partitions the candidates into balanced halves:
$$
F_{\text{upper}}=\{x_i\mid f_i\ge m_v\}, \qquad
F_{\text{lower}}=\{x_i\mid f_i<m_v\}.
$$
A secret random bit chooses one half, and the output is then sampled uniformly from that half [2509.21057].

This balanced partition is the key distinction from reject-sampling semantic watermarking. The paper proves the following single-channel result: if \(W\) is sampled i.i.d. from the base model, the candidates are split into equal halves by the median rule, a key bit \(k\sim\mathrm{Unif}\{0,1\}\) selects one half, and the final sentence \(Y\) is chosen uniformly within that half, then
$$
P_M^w(s\mid \bm{\pi})=P_M(s\mid \bm{\pi}).
$$
The intuition given is that, conditioned on \(W\), each candidate has probability \(1/N\) after averaging over the secret bit, so the original distribution is recovered exactly [2509.21057].

Multi-channel PMark applies this halving sequentially. Starting from \(V_0=W\), it sets
$$
V_i=\mathcal D(V_{i-1}\mid k_i,\mathcal F_i), \qquad i=1,\dots,b,
$$
and then samples
$$
Y\sim \mathrm{Un}(V_b).
$$
The paper remarks that multi-channel PMark remains distortion-free because every candidate survives each channel with probability \(1/2\), and the final selection is again uniform over the originally sampled set [2509.21057].

The same section provides a formal critique of earlier semantic methods. For a finite-range proxy function \(\mathcal F:\Sigma^*\to U\), let
$$
q(u)\triangleq \mu_M(\mathcal F^{-1}(u))
= \sum_{t\in \mathcal F^{-1}(u)} P_M(t\mid \boldsymbol{\pi}),
\qquad
\sum_{u\in U} q(u)=1.
$$
The paper shows that SemStamp-style rejection produces the base distribution conditioned on the chosen green set and derives a closed-form watermarked PMF. The resulting corollary is that such methods are distortion-free if and only if
$$
q(u)=\frac{1}{M}\qquad \forall u\in U,
$$
that is, only when all proxy-function bins have equal natural mass. PMark is designed specifically to avoid this requirement [2509.21057].

The paper also presents an optimized offline PMark. This version removes dynamic median estimation, uses the prior threshold \(0\), and scores candidates by accumulated channel evidence. The authors describe it as an empirical optimization rather than the fully proved distortion-free construction.

| Variant | Core rule | Stated property |
|---|---|---|
| Online PMark | Dynamic median estimation from sampled candidates | Distortion-free |
| Offline PMark | Fixed threshold \(0\) instead of dynamic medians | Better sampling efficiency |

## 4. Detection and robustness guarantees

Detection for online PMark reconstructs the semantic test channel by channel. For each observed sentence \(s^{(t)}\) and channel \(j\), the detector resamples a candidate set from the same model context, computes proxy scores, and estimates the median using the Harrell–Davis median estimator:
$$
\widehat m_{(t,j)} \gets \operatorname{HDMedian}\big(\{\mathcal F_j(x^{(i)})\}_{i=1}^N\big).
$$
It then compares the actual sentence score \(\mathcal F_j(s^{(t)})\) with the reconstructed threshold and the secret seed pattern [2509.21057].

To tolerate mismatch between the generator’s and detector’s median estimates, PMark uses a soft evidence score. For channel \(j\) at sentence \(t\),
$$
c_{(t,j)}=
\begin{cases}
1, & \text{if } (r_{(t,j)}=1 \wedge \mathcal F_j(s_t)>m'_{(t,j)}-\delta)\vee (r_{(t,j)}=0 \wedge \mathcal F_j(s_t)<m'_{(t,j)}+\delta),\\
e^{-K|\mathcal F_j(s_t)-m'_{(t,j)}|}, & \text{otherwise}.
\end{cases}
$$
Evidence is aggregated as
$$
N_g=\sum_{t=1}^{T}\sum_{j=1}^{b} c_{(t,j)}, \qquad
N_{\text{total}}=bT.
$$
The detector then applies a soft-count \(z\)-test:
$$
z=
\frac{\left|N_g-0.5\,N_{\text{total}}\right|}
{\sqrt{0.25\,N_{\text{total}}}},
$$
and declares watermark presence when \(z>z_\alpha\) [2509.21057].

The theoretical robustness claim is phrased in terms of bounded semantic perturbation. If an attacker \(\mathcal A\) changes a watermarked sentence \(s\) so that
$$
1-\langle \mathcal T(\mathcal A(s)),\mathcal T(s)\rangle \le d,
$$
then the probability of removing the channel evidence is bounded by
$$
P_{\mathrm{rm}}
\le
\mu_M\!\left(
s\in\Sigma^*
\;\middle|\;
\mathcal F_v(s)\in [m_v-\sqrt{2d},\,m_v+\sqrt{2d}]
\right).
$$
The interpretation is that a paraphrase can flip the watermark signal only when the original sentence score lies close to the median boundary. This gives PMark a formal semantic robustness statement absent from earlier sentence-level schemes [2509.21057].

A common misconception addressed implicitly by the paper is that any sentence-level watermark is automatically both robust and distortion-free. PMark’s analysis rejects that view: semantic-level watermarking may still distort the output distribution under reject sampling, and robustness depends on how densely evidence is distributed across sentences and channels, not merely on operating at the sentence level [2509.21057].

## 5. Empirical evaluation, efficiency, and ablations

The empirical study uses 500 samples each from C4 and BOOKSUM, with OPT-1.3B and Mistral-7B-v0.1 as backbone models. Baselines include token-level methods such as KGW, UPV, MorphMark, SIR, EXP, EXPGumbel, and SynthID, and semantic-level methods such as SemStamp, k-SemStamp, and SimMark. PMark uses all-mpnet-base-v2 embeddings without fine-tuning, \(b=4\) channels, sample budget \(N=64\), \(K=150\), and \(\delta=0.001\) [2509.21057].

Without attack, online PMark is reported as essentially perfect, often reaching \(99.8\%-100.0\%\) TPR@1% FPR with ROC-AUC near \(99.9\%\). Under GPT paraphrasing on Mistral-7B, the paper highlights the following results. On BOOKSUM, SemStamp reaches \(76.0\) TP@FP1%, SIR \(41.2\), PMark online \(96.8\), and PMark offline \(94.4\). On C4, SemStamp reaches \(69.2\), SIR \(63.4\), PMark online \(95.2\), and PMark offline \(92.0\). The paper summarizes these as gains up to \(20.8\%\) and \(26.0\%\) over the best semantic baseline SemStamp on Mistral-7B under GPT paraphrasing for BOOKSUM and C4 [2509.21057].

The reported text-quality figures also favor PMark. The paper gives perplexity values of \(4.37\) to \(4.71\) for PMark and interprets them as lower than those of earlier semantic methods and, in some cases, better than the strongest token-level baselines. This is presented as empirical support for the claim that avoiding reject-sampling distortion improves text quality [2509.21057].

The robustness study extends beyond sentence-level paraphrasing. Under 15% word deletion and 15% synonym substitution on BOOKSUM with Mistral-7B, online PMark reaches \(98.2\%\) and \(99.0\%\) TP@FP1%, while offline PMark reaches \(95.2\%\) and \(98.6\%\). These results are used to support the paper’s argument that multiple channels increase evidence density and thereby make the watermark resilient not only to paraphrasing but also to token-level edits [2509.21057].

Efficiency is reported in terms of average token consumption per sentence. The paper gives: k-SemStamp \(246.9\), SimMark \(186.7\), SemStamp \(1694.4\), PMark online \(315.8\), and PMark offline \(239.7\). Online PMark is therefore substantially cheaper than SemStamp, while offline PMark is cheaper than online PMark and slightly cheaper than k-SemStamp [2509.21057].

The ablations are consistent with the paper’s theoretical narrative. On OPT-1.3B and BOOKSUM, online PMark reaches \(98\%\) TP@FP1% even with \(N=8\) and \(b=3\), and approaches \(100\%\) as channel count and budget increase. The soft-\(z\) test is also validated empirically: relative to a hard-threshold \(z\)-test, it improves attacked TP@FP1% by \(11.9\%\) on BOOKSUM and \(6.4\%\) on C4 [2509.21057].

## 6. Nomenclature, related uses, and scope boundaries

In the literature represented here, “PMark” refers exactly to the semantic-level watermarking method introduced in “PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraints” [2509.21057]. It should not be conflated with “PromptMark,” the method proposed in “PromptMark: A Prompt-Guided Iterative-Feedback Framework for Source Code Watermarking.” That paper does not use the exact string “PMark,” does not introduce “PromptMark (PMark)” as an alias, and refers to its method as PromptMark only [2606.20835].

The label is also distinct from unrelated uses in other areas. One separate line of work uses “PMark” in connection with “Parallel Markov Chain Monte Carlo,” where the term denotes a partition-and-recombination strategy for parallelizing MCMC across asynchronous processors rather than a language-model watermarking method [1312.7479]. A further potential source of confusion is “PEMark,” which denotes a position-encoding watermark for API responses and is likewise unrelated in mechanism and problem setting [2605.21865].

Within watermarking research specifically, PMark’s scope is semantic-level watermarking of LLM-generated text, with an online construction that is claimed to be distortion-free and an offline approximation optimized for efficiency. The paper is explicit that the offline variant lacks the same strict proof as the online construction, that sentence segmentation consistency remains an assumption, and that stronger notions such as sequence-level or \(n\)-shot undetectability remain open problems [2509.21057].

Source: https://www.emergentmind.com/topics/pmark