EverTracer: Stealthy Fingerprinting for LLM Ownership

• EverTracer is a gray-box fingerprinting framework for verifying LLM ownership by embedding natural-language memorization signals into the model.
• It uses calibrated probability variation based on semantically perturbed inputs to detect stealthy, statistically significant ownership signals.
• The framework demonstrates robust performance against fine-tuning, pruning, and model merging, making it effective for forensic attribution in LLM theft scenarios.

EverTracer is a gray-box fingerprinting framework for LLM ownership verification that aims to trace model provenance after theft, unauthorized redistribution, or derivative reuse. It embeds ownership signals through memorization of ordinary natural-language data rather than through explicit trigger-output overfitting, and it verifies those signals by testing whether a suspect model preserves a calibrated local probability signature on the owner’s fingerprint set. Introduced in “EverTracer: Hunting Stolen LLMs via Stealthy and Robust Probabilistic Fingerprint” by Zhenhua Xu, Meng Han, and Wenpeng Xing of Zhejiang University and GenTel.io, the framework is presented as the first method to repurpose Membership Inference Attacks for defensive use in LLM fingerprinting (Xu et al., 3 Sep 2025).

1. Problem formulation and threat model

EverTracer addresses LLM ownership verification under a setting in which a model owner needs to determine whether a suspect model was stolen from, or derived from, a protected model. The motivating concerns are model theft, license violations, and the practical difficulty of proving provenance after a thief has fine-tuned, pruned, or merged the stolen checkpoint. The paper argues that existing ownership-verification methods are limited either because they require white-box access to parameters or internal representations, or because they rely on trigger-based mechanisms that introduce detectable artifacts and can be weakened by simple defenses (Xu et al., 3 Sep 2025).

The framework assumes a gray-box verification setting. The verifier can query the suspect model and obtain generated texts together with token-level logits, loss values, or log-probabilities, but cannot inspect internal parameters or activations. This is weaker than white-box verification and stronger than strict text-only black-box access. The suspect model, denoted $\theta_U$, may be an original stolen model or a derived descendant modified by pruning, incremental fine-tuning, or model merging. The adversary may also deploy input-level defenses such as perplexity-based filtering or adversarial query perturbation. EverTracer therefore defines its target as a stealthy and robust probabilistic fingerprint: stealthy because it should not create obvious detectable artifacts, robust because it should remain verifiable after realistic model-level and input-level modification, and probabilistic because the evidentiary signal is statistical rather than a deterministic trigger-response behavior (Xu et al., 3 Sep 2025).

2. Fingerprint injection through natural-language memorization

EverTracer divides a fingerprint corpus into two disjoint subsets,

$D_f = D_{tr} \cup D_{ref},$

where $D_{tr}$ is used to inject the fingerprint into the protected model and $D_{ref}$ is used to train a matched reference model. The paper emphasizes that the owner may use any open-source or privately curated natural-language corpus; the reported experiments use AG News and XSum. This design is central to the framework’s stealth claim, because the injected data are ordinary language samples rather than handcrafted triggers, rare-token prompts, or forced target strings (Xu et al., 3 Sep 2025).

Injection itself is deliberately conventional. The protected model is fine-tuned on $D_{tr}$, while a separate reference model is fine-tuned on $D_{ref}$, both using LoRA and the standard causal language-model negative log-likelihood,

$$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$

No custom watermarking loss, hidden-state regularizer, or explicit backdoor objective is introduced. The method instead relies on the empirical fact that fine-tuning can induce memorization without visibly abnormal behavior, and later treats that memorization as an ownership signal. The default reported configuration uses $|D_{tr}| = 100$ and $|D_{ref}| = 1000$; Mistral is trained for 10 epochs, other models for 20 epochs, and reference models for 4 epochs. The paper states that all runs fit in about 16GB GPU memory under FP16 with LoRA, and that each fine-tuning run completes in under 30 minutes (Xu et al., 3 Sep 2025).

This construction differentiates EverTracer from backdoor-based fingerprinting. Prior trigger-oriented methods create a visible activation interface between a prompt pattern and a prescribed output. EverTracer instead makes ownership evidence latent in the next-token probability geometry of normal text. A plausible implication is that the framework shifts provenance verification from output behavior to memorization behavior, which is why it can remain compatible with ordinary language-model training objectives while still supporting later forensic testing (Xu et al., 3 Sep 2025).

3. Verification by calibrated probability variation

Verification begins with a fingerprint sample $\boldsymbol{x} \in D_{tr}$. For each such sample, EverTracer generates $D_f = D_{tr} \cup D_{ref},$0 semantically perturbed neighbors,

$D_f = D_{tr} \cup D_{ref},$1

In the reported implementation, $D_f = D_{tr} \cup D_{ref},$2, 30% of tokens are randomly selected, and T5-Base is used to generate positive and negative semantic replacements. The core intuition is that a memorized sample tends to lie in a sharper local likelihood basin than nearby paraphrastic alternatives (Xu et al., 3 Sep 2025).

For the suspect model $D_f = D_{tr} \cup D_{ref},$3, EverTracer computes the probability variation signal

$D_f = D_{tr} \cup D_{ref},$4

It then calibrates that signal using the reference model $D_f = D_{tr} \cup D_{ref},$5: $D_f = D_{tr} \cup D_{ref},$6 A sample is declared memorized if $D_f = D_{tr} \cup D_{ref},$7 for a threshold $D_f = D_{tr} \cup D_{ref},$8. The true positive and false positive rates are then computed over $D_f = D_{tr} \cup D_{ref},$9 and an unseen set $D_{tr}$0, and the paper reports both AUC and Fingerprint Success Rate, where FSR is the TPR at the largest threshold satisfying $D_{tr}$1 (Xu et al., 3 Sep 2025).

The reference-model subtraction is not a minor refinement but a central calibration device. The paper reports a LLaMA2 ablation after incremental fine-tuning in which Alpaca yields FSR $D_{tr}$2 without the reference model and $D_{tr}$3 with it, ShareGPT yields $D_{tr}$4 without and $D_{tr}$5 with it, and Dolly yields $D_{tr}$6 without and $D_{tr}$7 with it; the corresponding AUC values rise from $D_{tr}$8 to $D_{tr}$9 (Xu et al., 3 Sep 2025). This indicates that calibrated memorization, rather than raw memorization alone, is the operative forensic signal.

4. Stealthiness and utility preservation

EverTracer’s central claim of stealthiness is evaluated through input naturalness. The paper compares the perplexity of fingerprint queries under GPT-2 and LLaMA3-8B-Instruct. EverTracer-AG reports GPT-2 perplexity $D_{ref}$0 and LLaMA3-Instruct perplexity $D_{ref}$1; the XSum-based variant reports $D_{ref}$2 and $D_{ref}$3. These values are substantially lower than IF, which reports $D_{ref}$4 and $D_{ref}$5, and vastly lower than ProFlingo, which reports $D_{ref}$6 and $D_{ref}$7 on one line and $D_{ref}$8 and $D_{ref}$9 on another. HashChain is closer to normal language than IF or ProFlingo, but still above EverTracer on the reported GPT-2 metric. On the paper’s definition, lower perplexity corresponds to less detectable fingerprint queries (Xu et al., 3 Sep 2025).

The framework also evaluates harmfulness to general task performance across 16 zero-shot tasks. The results show that the effect depends on the fingerprint dataset. XSum-based EverTracer remains close to base-model averages: for example, LLaMA2 moves from $D_{tr}$0 before fingerprinting to $D_{tr}$1 with EverTracer-XSum, while LLaMA3 moves from $D_{tr}$2 to $D_{tr}$3. AG News-based EverTracer can reduce average scores more noticeably, such as Mistral from $D_{tr}$4 to $D_{tr}$5. The paper therefore does not claim universal harmlessness; rather, it argues that performance retention depends on dataset choice and hyperparameter configuration, and reports appendix ablations showing strong verification performance across multiple data sizes and epoch counts (Xu et al., 3 Sep 2025).

A common misconception is that stealthiness here means invisibility in a cryptographic sense. The paper does not make that claim. Its narrower claim is that natural-language fingerprint data avoid the obvious statistical anomalies produced by trigger-based schemes, especially under perplexity-based filtering (Xu et al., 3 Sep 2025).

5. Effectiveness and robustness under attack

On unmodified fingerprinted models, EverTracer reports near-perfect verification strength across Falcon-7B, LLaMA-2-7B, Mistral-7B-v0.3, and LLaMA3-8B. EverTracer-AG reports $D_{tr}$6 on Falcon, $D_{tr}$7 on LLaMA2, and $D_{tr}$8 on both Mistral and LLaMA3; EverTracer-XSum reports $D_{tr}$9 on Falcon and $D_{ref}$0 on the other three architectures. The paper presents these values as FSR@AUC, showing that EverTracer matches trigger-based methods in base effectiveness while avoiding explicit trigger behavior (Xu et al., 3 Sep 2025).

The more consequential results concern post-theft modification. Under incremental fine-tuning on Alpaca, ShareGPT, and Dolly, EverTracer remains consistently strong, especially in the XSum variant. For LLaMA2, EverTracer-XSum reports $D_{ref}$1 FSR after Alpaca, $D_{ref}$2 after ShareGPT, and $D_{ref}$3 after Dolly, with AUC values $D_{ref}$4. For Falcon, EverTracer-XSum reports $D_{ref}$5 across the same datasets. For Mistral and LLaMA3, the AG variant can be weaker, but the paper notes that AUC often remains high even when FSR drops, implying residual forensic separation (Xu et al., 3 Sep 2025).

Under pruning, the strongest reported EverTracer result appears under Taylor pruning on LLaMA2, where it achieves $D_{ref}$6. Under model merging, the framework remains effective even when the fingerprinted component’s contribution weight is small: for example, the paper reports EverTracer values such as $D_{ref}$7 at merge ratio $D_{ref}$8, $D_{ref}$9 or $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$0 at $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$1, and $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$2 or $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$3 at $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$4, depending on the merging variant (Xu et al., 3 Sep 2025). The paper summarizes this as robustness to model fusion and argues that such post-hoc transformations are realistic in license-violation scenarios.

Input-level perturbation produces more architecture-dependent outcomes. Under Remove-Perturbation on LLaMA3, EverTracer-AG reports $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$5 at RP-5% and $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$6 at RP-10%. Under the same test on LLaMA2, the results are $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$7 and $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$8. The paper attributes this difference to stronger models preserving semantic association more reliably under noisy perturbations (Xu et al., 3 Sep 2025). This is an important caveat: robustness is strong overall, but not uniform across model families.

6. Position in the fingerprinting literature, limitations, and implications

EverTracer situates itself against three major prior categories: white-box or intrinsic fingerprints, optimization-based gray-box or black-box fingerprints, and backdoor-based fingerprints. Its primary conceptual contribution is to convert membership inference from a privacy attack into a provenance mechanism. In that sense, the framework belongs as much to the calibrated MIA literature as to the watermarking literature. The paper explicitly argues that backdoor methods face a stealthiness-robustness paradox: robust triggers are detectable, while natural triggers are often fragile. EverTracer’s answer is to abandon trigger activation entirely and verify ownership through calibrated memorization instead (Xu et al., 3 Sep 2025).

The framework’s limitations are also explicit. It requires gray-box access to token-level probabilities, logits, or loss values, and therefore does not operate under strict text-only black-box APIs. It is evaluated only with LoRA rather than a broad range of fine-tuning regimes. Its attack suite, while broader than many prior fingerprinting studies, does not exhaust the space of post-theft defenses, and the paper specifically notes that resilience against explicit memory-erasure methods such as MEraser is unknown. The method also provides probabilistic forensic evidence rather than a cryptographic proof, and the paper does not offer a formal legal-standard false-accusation guarantee beyond reporting AUC and FSR at $$\mathcal{L} = -\sum_{\boldsymbol{x} \in D} \sum_{i=1}^n \log p(x^i|\boldsymbol{x}^{<i}).$$9 (Xu et al., 3 Sep 2025).

These limitations do not negate the paper’s significance. EverTracer shows that ownership verification for LLMs need not rely on unnatural prompt triggers or white-box parameter access. Instead, it can be built around an ordinary fine-tuning pipeline, a matched reference model, and a statistically calibrated test of local probability geometry. This suggests a broader shift in model-provenance research: from explicit watermark behavior toward latent memorization signatures, and from attack-oriented membership inference toward defensive forensic attribution (Xu et al., 3 Sep 2025).