ForgetMark: Fingerprinting via Targeted Unlearning

• The paper introduces ForgetMark, a framework that replaces fixed trigger backdoors with targeted unlearning to embed ownership in language models.
• It uses a two-stage, uncertainty-driven key-value selection process, where highly confident responses are identified and adapted via lightweight LoRA adapters.
• Ownership is verified through a probabilistic forgetting trace that detects significant likelihood suppression, ensuring stealth and limited performance degradation.

Searching arXiv for the primary paper and closely related watermarking/fingerprinting work. ForgetMark is a fingerprinting framework for LLMs that encodes provenance through targeted unlearning rather than through a fixed trigger–response backdoor. Introduced in "ForgetMark: Stealthy Fingerprint Embedding via Targeted Unlearning in LLMs" (Xu et al., 13 Jan 2026), it constructs a compact set of human-readable Key$\to$Value pairs, identifies those pairs on which the base model is maximally confident, and then trains lightweight LoRA adapters to suppress the original values on their keys while preserving general capabilities. Ownership is subsequently verified from a probabilistic forgetting trace: for the selected pairs, the fingerprinted model assigns very low likelihood to the original values or generates semantically different completions, under gray-box or black-box access.

1. Conceptual position within model provenance

ForgetMark is explicitly framed against traditional invasive fingerprinting based on backdoors. In the backdoor formulation summarized for the method, a rare trigger sequence $T$ is implanted so that $M(T)$ deterministically yields a response $R$. The reported weaknesses of that design are threefold: the triggers are high-perplexity and thus easy to filter, the responses are fixed and therefore exposed by heuristic detectors, and the triggers can spuriously activate on benign inputs (Xu et al., 13 Jan 2026). ForgetMark replaces this template with selective forgetting of naturally phrased prompts that the base model originally answers with high confidence.

The mechanism is therefore not response synthesis but response suppression. A small set of Key$\to$Value pairs is first chosen so that the target model already exhibits low-variance, high-likelihood behavior on them. A lightweight adaptation step then reduces $P(v\mid k)$ for those pairs while leaving the rest of the distribution largely intact. This makes provenance detectable not by observing an anomalous answer pattern, but by measuring whether a suspect model has retained a stable negative trace on a carefully chosen prompt set.

This design suggests a different stealth regime from classical trigger-based fingerprints. The trigger surface is human-readable rather than synthetically rare, and the ownership signal is probabilistic rather than deterministic. A plausible implication is that detection methods tuned to high-perplexity triggers or fixed outputs are structurally mismatched to the signal that ForgetMark embeds.

2. Construction of the fingerprint set

ForgetMark begins with a two-stage selection procedure for the fingerprint set. An auxiliary assistant model, specified as GPT-4, is prompted to generate 500 human-readable, single-turn prompts of length 10–40 tokens that elicit specific factual answers. After manual filtering for safety and diversity, these form the candidate key set

$K=\{k_1,\dots,k_K\}, \qquad K=500.$

For each candidate key $k_i$, the target model $M_{\text{target}}$ is queried to sample $M$ independent continuations,

$T$0

with $T$1 in the main experiments, while recording token-level generation probabilities $T$2. ForgetMark then ranks keys by a predictive-entropy proxy

$T$3

Small $T$4 indicates that the model is highly confident and low-variance on key $T$5. Keys are sorted by ascending $T$6, and the top $T$7 are retained, with $T$8 in the main configuration. For each selected key, the retained value is the sampled continuation with highest total likelihood,

$T$9

The resulting fingerprint set is

$M(T)$0

The reported ablation makes the role of this uncertainty-driven selection explicit. On non-unlearned Qwen2.5 and LLaMA3 models, random selection yields average $M(T)$1 and ROUGE-L $M(T)$2, whereas uncertainty-driven selection yields $M(T)$3 and ROUGE-L $M(T)$4 (Xu et al., 13 Jan 2026). This indicates that entropy-based selection concentrates the fingerprint on behaviors that are already stable in the base model, thereby enlarging the pre/post-unlearning gap and reducing false alarms on untouched models.

3. Targeted unlearning as the embedding mechanism

The embedding stage freezes the base parameters $M(T)$5 and inserts a low-rank adapter

$M(T)$6

so that the adapted model defines the next-token distribution

$M(T)$7

The training objective is a signed-likelihood loss that simultaneously increases the NLL on the fingerprint pairs and decreases the NLL on a retention set:

$M(T)$8

In the reported setup, $M(T)$9.

The retention set $R$0 is drawn from Alpaca and sized so that $R$1. The optimization updates only $R$2 and $R$3, with reported hyperparameters including rank $R$4, AdamW, learning rate $R$5, and approximately $R$6 training steps (Xu et al., 13 Jan 2026). The operational intent is selective forgetting: the model should sharply suppress the original values $R$7 on keys $R$8 while maintaining general utility on ordinary instructions.

This formulation distinguishes ForgetMark from fingerprints that directly encode a recognizably artificial behavior. The provenance signal is stored as a localized likelihood depression over naturally phrased prompts. That choice also explains why the framework emphasizes retention data: without an explicit counterweight, an unlearning-based fingerprint could collapse into general degradation rather than targeted suppression.

4. Probabilistic forgetting trace and verification

After adaptation, ForgetMark defines ownership through what it calls a probabilistic forgetting trace. For every fingerprint pair $R$9, the adapted model is expected to assign dramatically reduced probability to the original continuation:

$\to$0

Verification is supported under two access regimes. In the gray-box protocol, the verifier queries each key $\to$1 and directly reads out $\to$2 from the suspect model $\to$3. In the black-box protocol, the verifier samples a continuation $\to$4 from $\to$5 and computes $\to$6. These signals are aggregated into the Fingerprint Success Rate:

$\to$7

The reported thresholds are $\to$8 and $\to$9, calibrated to yield approximately zero false positives. Ownership is declared if $P(v\mid k)$0, with $P(v\mid k)$1 given as an example (Xu et al., 13 Jan 2026).

The gray-box and black-box variants are also reported separately as $P(v\mid k)$2 and $P(v\mid k)$3. The central point is that verification does not require the suspect model to emit a specific string. It is sufficient that the model consistently fails, in either likelihood or semantics, to reproduce the original high-confidence continuation on the selected keys.

5. Reported empirical properties

The main experiments use Mistral-7B-v0.3, LLaMA-3-8B, and Qwen2.5-7B as target models; GPT-4 as the assistant model; $P(v\mid k)$4 candidate keys; $P(v\mid k)$5 continuations per key; $P(v\mid k)$6 final fingerprint pairs; and an Alpaca retention set of size $P(v\mid k)$7 (Xu et al., 13 Jan 2026). Evaluation spans effectiveness, harmlessness, input-level stealth, output stealth, and robustness under model merging and incremental fine-tuning.

Aspect	ForgetMark result	Context
Clean fingerprinted models	$P(v\mid k)$8, $P(v\mid k)$9	Compared with IF-SFT and Chain·Hash
Qwen2.5-7B harmlessness	$K=\{k_1,\dots,k_K\}, \qquad K=500.$0	Original model: $K=\{k_1,\dots,k_K\}, \qquad K=500.$1
Input stealth, GPT-2 PPL	$K=\{k_1,\dots,k_K\}, \qquad K=500.$2	IF-SFT: $K=\{k_1,\dots,k_K\}, \qquad K=500.$3, Chain·Hash: $K=\{k_1,\dots,k_K\}, \qquad K=500.$4
Input stealth, LLaMA3 estimator PPL	$K=\{k_1,\dots,k_K\}, \qquad K=500.$5	IF-SFT: $K=\{k_1,\dots,k_K\}, \qquad K=500.$6, Chain·Hash: $K=\{k_1,\dots,k_K\}, \qquad K=500.$7
Output stealth	Token-Forcing detection rate $K=\{k_1,\dots,k_K\}, \qquad K=500.$8	IF-SFT: $K=\{k_1,\dots,k_K\}, \qquad K=500.$9, Chain·Hash: $k_i$0

On clean fingerprinted models, ForgetMark reports $k_i$1 and $k_i$2, matching IF-SFT on raw success rate and exceeding Chain·Hash, which reports $k_i$3 on both metrics. On Qwen2.5-7B, the zero-shot accuracy averaged over 18 benchmarks declines from $k_i$4 for the original model to $k_i$5 after ForgetMark embedding. The benchmarks listed are ANLI, ARC, OpenBookQA, Winogrande, QC, BoolQ, CB, RTE, WiC, WSC, CoPA, MultiRC, and LAMBADA within the reported 18-benchmark average (Xu et al., 13 Jan 2026).

The stealth metrics are particularly central to the method’s positioning. Input-level stealth, measured by external perplexity estimators, yields $k_i$6 under GPT-2 and $k_i$7 under LLaMA3-Instruct, compared with IF-SFT at $k_i$8 and $k_i$9, and Chain·Hash at $M_{\text{target}}$0 and $M_{\text{target}}$1. Output-level stealth, measured by Token-Forcing detection, is reported as $M_{\text{target}}$2 for ForgetMark, compared with $M_{\text{target}}$3 for IF-SFT and $M_{\text{target}}$4 for Chain·Hash. This supports the claim that the forgetting trace is substantially less exposed to detector heuristics aimed at trigger-based fingerprints.

Robustness under model merging is reported with MergeKit under the Task merge strategy. For $M_{\text{target}}$5, $M_{\text{target}}$6, and $M_{\text{target}}$7, ForgetMark retains $M_{\text{target}}$8; at $M_{\text{target}}$9 it reports $M$0; and at $M$1 it reports $M$2. The accompanying summary states that ForgetMark sustains near-perfect FSR down to $M$3 (Xu et al., 13 Jan 2026). Under incremental fine-tuning on UltraChat using Mistral-7B, the reported trajectory is: at step $M$4, $M$5; at $M$6, $M$7; at $M$8, $M$9; and at $T$00, $T$01.

A second ablation studies the key-set size $T$02 on Qwen2.5. The reported values are:

• $T$03: harmlessness $T$04, average $T$05, ROUGE-L $T$06
• $T$07: harmlessness $T$08, average $T$09, ROUGE-L $T$10
• $T$11: harmlessness $T$12, average $T$13, ROUGE-L $T$14
• $T$15: harmlessness $T$16, average $T$17, ROUGE-L $T$18

The authors’ interpretation is that $T$19 offers a clear verification margin with only negligible utility drop. This suggests that the framework’s operating point depends not only on adapter capacity but also on the concentration of the selected behavioral manifold.

6. Relation to watermarking, removal, and terminological ambiguity

ForgetMark belongs to the broader provenance literature for generative models, but it occupies a distinct niche. In the language-model setting, it differs from statistical watermarking schemes such as the green-token/red-token formulation associated with "A Watermark for LLMs" (Kirchenbauer et al., 2023), and it also differs from recovery or stealing attacks such as "Watermark Stealing in LLMs" (Jovanović et al., 2024). A particularly direct contrast is "De-mark: Watermark Removal in LLMs" (Chen et al., 2024), which targets n-gram watermarks by estimating the red list, green list, watermark strength, and prefix length through random selection probing, then inverting the green-token boost to form a removal distribution. ForgetMark does not remove a watermark; it embeds ownership via targeted unlearning.

There is also a terminological ambiguity in contemporaneous image-provenance literature. In the MarkCleaner description, “ForgetMark” is used as a summary label for a watermark-removal capability that exploits micro-geometric perturbations to break the phase alignment of semantic watermarks (Kong et al., 2 Feb 2026). In the MarkSweep description, “ForgetMark” similarly denotes the capability to erase invisible image watermarks through edge-aware noise intensification and frequency-aware denoising (Cao et al., 17 Feb 2026). These usages are conceptually separate from the arXiv work titled "ForgetMark" (Xu et al., 13 Jan 2026). One concerns fingerprint embedding in LMs; the others concern watermark erasure in AI-generated images.

The principal limitations reported for ForgetMark are also different from those in watermark-removal systems. Its trace degrades under sufficient downstream fine-tuning, as shown by the UltraChat experiment, and its durability under extreme merge ratios is finite. The stated future directions are anti-recovery regularizers and rotating or refreshable fingerprints (Xu et al., 13 Jan 2026). This suggests that ForgetMark should be understood less as an immutable watermark and more as a stealth-oriented ownership signal whose persistence depends on post-deployment adaptation.