FSFT: Fingerprint Subspace-aware Fine-Tuning

• The paper introduces FSFT, a method that secures LLM fingerprints by constraining updates in a specialized parameter subspace.
• FSFT recasts fingerprint injection as a knowledge editing task, offering a lightweight alternative to conventional fine-tuning approaches.
• Empirical results demonstrate that FSFT improves fingerprint persistence during model modifications, including pruning, quantization, and further tuning.

Fingerprint Subspace-aware Fine-Tuning (FSFT) is a defense method for preserving persistent edit-based fingerprints in LLMs. It is introduced in "From Evaluation to Defense: Constructing Persistent Edit-Based Fingerprints for LLMs" as part of a broader framework for LLM intellectual property protection that treats fingerprint injection as a knowledge editing problem rather than as ordinary instruction tuning (Li et al., 3 Sep 2025). In this formulation, a model is trained to respond to a special private key with a public response, but the central requirement is not merely effectiveness at injection time; it is persistence under subsequent model modification. FSFT addresses the specific failure mode in which later large-scale fine-tuning degrades or removes the fingerprint by constraining updates in the fingerprint-relevant parameter subspace.

1. Problem setting and motivation

The paper studies LLM fingerprinting for IP protection, with emphasis on the persistence problem: fingerprints may be present immediately after injection, yet become weakened or lost after pruning, quantization, or downstream fine-tuning (Li et al., 3 Sep 2025). In the setting considered, a fingerprinting scheme trains a model to map a special private key $x$ to a public response $y$. This mechanism is intended to support model ownership verification without substantially changing ordinary behavior.

The paper argues that persistence is the decisive requirement. Ordinary fine-tuning often damages persistence, especially when later training touches many parameters or large parts of the model. This is presented not as an incidental implementation issue but as a structural challenge for fingerprinting schemes that rely on parameter changes that are not protected against subsequent optimization. A common misconception is that successful injection alone is sufficient for IP protection; the paper’s analysis instead treats persistence under realistic post-release modification as a primary criterion.

This framing also motivates the shift from evaluation to defense. Rather than only measuring how existing fingerprints fail under model modification, the paper proposes a mechanism specifically designed to preserve the parameter-space structure associated with the injected fingerprint. FSFT is the defense mechanism that emerges from that analysis.

2. Fingerprint injection as knowledge editing

A major claim of the paper is that knowledge editing is a better lightweight alternative to ordinary instruction tuning for fingerprint injection (Li et al., 3 Sep 2025). The justification is an analogy between knowledge editing and fingerprint injection: both seek to alter model behavior on a narrow set of inputs while preserving the original model’s behavior elsewhere.

For a model $f_\theta : \mathcal{X} \to \mathcal{Y}$, and editing data

$D_e = \{(x_i^e, y_i^e)\}_{i=1}^n,$

the goal is to obtain parameters $\theta^e$ such that

$$f_{\theta^e}(x) = \begin{cases} y_i^e & \text{for all } (x_i^e, y_i^e) \in D_e, \ f_\theta(x) & \text{otherwise}. \end{cases}$$

The paper states that fingerprint injection can be cast in almost exactly the same form. Given fingerprint data

$D_{fp} = \{(x_i^{fp}, y^{fp})\}_{i=1}^n,$

the target is to map $x \in \mathcal{X}_{fp}$ to $y^{fp}$ while retaining $f_\theta(x)$ for $y$0. This reformulation is consequential because editing is described as lighter-weight, more targeted, and more efficient than full or LoRA-style fine-tuning.

The paper evaluates several editing families. The locate-then-edit methods are R-ROME, EMMET, AlphaEdit, and UltraEdit. The hypernetwork-based methods are Malmen and RLEdit. The memory-based methods are DEFER and WISE. These are compared with fine-tuning-based baselines including LoRA and FT-M. The reported finding is that edit-based fingerprints generally outperform fine-tuning-based fingerprints across effectiveness, harmlessness, efficiency, and often persistence. This suggests that the main contribution of FSFT sits within a larger reorientation of fingerprinting from general adaptation to targeted editing.

3. Fingerprint subspace and the FSFT objective

The paper’s central defense insight is that fingerprint degradation under later fine-tuning is not random: fine-tuning disrupts a specific fingerprint-relevant subspace (Li et al., 3 Sep 2025). Empirically, as fine-tuning proceeds, token-level fingerprint success rate $y$1 decreases steadily, while the norm of the update projected onto the fingerprint subspace increases. The interpretation is that downstream optimization increasingly overlaps with the parameter directions that carry fingerprint information.

FSFT defines that subspace from the difference between edited and original weights. If fingerprint injection modifies module $y$2 from $y$3 to $y$4, the fingerprint direction or matrix is

$y$5

This parameter displacement is treated as the signature of the fingerprint injection in module $y$6. The associated projection matrix is

$y$7

The paper also mentions a cheaper approximation used in related work: $y$8

During downstream fine-tuning, if the update on module $y$9 is $f_\theta : \mathcal{X} \to \mathcal{Y}$0, then for LoRA

$f_\theta : \mathcal{X} \to \mathcal{Y}$1

The magnitude of the update projected into the fingerprint subspace is measured by

$f_\theta : \mathcal{X} \to \mathcal{Y}$2

FSFT adds this quantity as a regularizer to the downstream task objective: $f_\theta : \mathcal{X} \to \mathcal{Y}$3 where $f_\theta : \mathcal{X} \to \mathcal{Y}$4 controls the penalty strength and the sum is taken over modules. The method therefore does not freeze fingerprint modules. Instead, it allows fine-tuning everywhere while penalizing updates that strongly overlap with the fingerprint subspace.

The distinction from related baselines is explicit. Standard fine-tuning updates parameters without regard to fingerprint directions. FreezeFT freezes modules that contain fingerprint information. FSFT regularizes the update geometry so that the fingerprint subspace is preserved. The paper further argues that freezing modules can be counterproductive because fingerprint information is not stored in isolation and must also be transmitted across modules. This directly addresses the misconception that preservation requires immobility of the edited module; the proposed alternative is selective geometric regularization rather than blanket freezing.

4. Experimental configuration

The main experiments use two open-source LLMs: Llama-3.2-3B-Instruct and Qwen-3-8B (Li et al., 3 Sep 2025). The paper compares 10 fingerprinting methods: LoRA, FT-M, R-ROME, EMMET, AlphaEdit, UltraEdit, Malmen, RLEdit, DEFER, and WISE.

Fingerprint construction is based on 20 fingerprint QA pairs built from classical Chinese, Pokémon names written in Japanese, and arbitrary model vocabulary tokens, with a unified public key. The 20 examples are split evenly for effectiveness and robustness. This design is intended to isolate the behavior of the trigger-response mechanism while also testing confusion with similar inputs.

Evaluation is organized along five dimensions. Effectiveness is measured by Fingerprint Success Rate (FSR), defined as

$f_\theta : \mathcal{X} \to \mathcal{Y}$5

Robustness is evaluated such that lower FSR on similar inputs is better. Harmlessness is measured using zero-shot accuracy on BoolQ, RTE, ARC-Challenge, and TinyMMLU, together with WikiText2 perplexity. Efficiency is assessed by time and memory overhead. Persistence is measured under pruning, quantization, and further fine-tuning.

For fine-tuning persistence, the paper also uses token-level fingerprint retention: $f_\theta : \mathcal{X} \to \mathcal{Y}$6 Stress tests include pruning at 30\%, 40\%, and 50\%; quantization at 8-bit, 4-bit, and 3-bit; and further fine-tuning on Alpaca and MathInstruct. This protocol makes persistence a comparative property across multiple modification regimes rather than a single-point post hoc measurement.

5. Empirical behavior of FSFT and edit-based fingerprints

The paper reports that most knowledge-editing methods can achieve 100% effectiveness FSR, often with little or no degradation in harmlessness, and with lower time and memory cost than LoRA (Li et al., 3 Sep 2025). Across pruning, quantization, and fine-tuning, knowledge-editing-based fingerprints are described as much more persistent than LoRA-based ones. This places FSFT within a broader empirical picture in which edit-based injection is already strong before any downstream defense is applied.

The specific contribution of FSFT is improvement in fingerprint persistence under downstream fine-tuning. In the Llama-3.2 experiments, FSFT achieves at least a 10% improvement in effectiveness over ordinary fine-tuning in the worst case. On Qwen3, similar behavior is reported, with FSFT giving +10% effectiveness on Alpaca and +20% effectiveness on MathInstruct, while keeping harmlessness essentially stable.

The paper also reports that FreezeFT can perform worse than ordinary FT, especially on larger models. The stated reason is that freezing the fingerprint module can block useful inter-module transmission. This is important because it changes the interpretation of persistence preservation: the relevant issue is not merely whether the locally edited parameters are protected, but whether the network can continue to propagate the fingerprint behavior through the model after downstream adaptation. A plausible implication is that FSFT’s advantage derives from preserving fingerprint-carrying directions while still permitting the broader network reconfiguration needed for task adaptation.

6. Robustness limits and implications

A major observation of the paper is that fingerprint-injected models often cannot reliably distinguish fingerprints from similar scrambled or garbled texts, even for the better methods (Li et al., 3 Sep 2025). The authors visualize latent representations and find that fingerprint keys and similar altered keys are very close in feature space, that both are far from ordinary data like Alpaca, and that the intra-group variation among scrambled inputs is small.

The paper interprets this as evidence that the model’s representation is not fine-grained enough. The fingerprint triggers are too coarse in the sense that inputs that look similarly nonsensical or token-patterned can activate similar internal features. As a result, robustness remains weak because the model learns a broad “weird-string” region rather than a precise fingerprint identity.

This observation limits how FSFT should be understood. FSFT is a persistence defense, not a complete solution to fingerprint robustness. It preserves the parameter subspace associated with the injected fingerprint during downstream fine-tuning, but it does not by itself make the trigger representation more discriminative. The paper therefore concludes that future fingerprinting methods need to be more discriminative, more fine-grained, and better at separating true fingerprint keys from similar-looking distractors. The broader message is that edit-based fingerprints are a strong alternative to conventional fingerprint injection, while robustness to near-trigger confusion remains a major open problem.