LoRA-Leak: Privacy Evaluation of LoRA Fine-Tuning

• LoRA-Leak is a comprehensive evaluation framework that exposes membership inference risks in LoRA fine-tuning by leveraging a public pre-trained model as a reference.
• The framework employs a novel calibration method that computes loss differences between the fine-tuned and pre-trained models to detect private data inclusion.
• Experimental results indicate that even conservative LoRA settings bear significant privacy risks, though defenses like dropout and selective layer exclusion can mitigate these risks.

LoRA-Leak is a membership-inference evaluation framework for Low-Rank Adaptation (LoRA) fine-tuned LLMs in the standard pre-training plus fine-tuning regime. It studies the case in which a public pre-trained causal LLM is adapted to a private downstream dataset through LoRA and the resulting fine-tuned model is released together with its adapters. The framework’s central finding is that parameter-efficient fine-tuning does not eliminate privacy risk: even when only low-rank adapter parameters are trained, fine-tuning-set membership remains inferable, and the publicly available pre-trained model can itself be used as a reference that strengthens the attack surface (Ran et al., 24 Jul 2025).

1. Conceptual basis and formal setting

LoRA-Leak is grounded in the observation that modern language-model deployment commonly separates a universal pre-trained model, denoted $\mathcal M_{pt}$, from a downstream-specialized model, denoted $\mathcal M_{ft}$, obtained by LoRA fine-tuning. In LoRA, the original weights are frozen and each adapted linear layer replaces the original parameter matrix with a low-rank update:

$$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$

Because $r$ is small, LoRA adds only a few million parameters on top of billion-parameter backbones, reducing memory and GPU requirements by $5$–$10\times$. This efficiency helped drive widespread adoption, with over 12 000 LoRA models on Hugging Face by end-2023 (Ran et al., 24 Jul 2025).

A key misconception addressed by LoRA-Leak is the idea that training only a small side adapter should make fine-tuning data effectively invulnerable to membership inference. The framework rejects that premise. It argues that privacy analysis under LoRA cannot ignore the original public checkpoint, because the pre-trained model encodes a baseline familiarity with general text and provides a direct comparison point for measuring how fine-tuning changed the model’s response to candidate samples (Ran et al., 24 Jul 2025).

The threat model is white-box. The adversary knows the public pre-trained checkpoint, has access to the released fine-tuned model and LoRA adapters, and can execute forward and backward passes through both $\mathcal M_{pt}$ and $\mathcal M_{ft}$. The adversary’s objective is standard membership inference: for a candidate sample $x$, decide whether $x\in\mathcal D_{ft}$, where $\mathcal M_{ft}$0 is the private fine-tuning corpus (Ran et al., 24 Jul 2025).

2. Membership inference objective and pre-trained reference calibration

LoRA-Leak formulates membership inference around a score function applied to a candidate sequence. For a prefix $\mathcal M_{ft}$1, the framework uses the sequence-level cross-entropy

$\mathcal M_{ft}$2

and then defines attack-specific scores $\mathcal M_{ft}$3, together with a threshold $\mathcal M_{ft}$4, to predict membership when the score crosses the threshold (Ran et al., 24 Jul 2025).

The distinctive technical contribution is the pre-trained-reference calibration. Given any attack score $\mathcal M_{ft}$5, LoRA-Leak defines

$\mathcal M_{ft}$6

This construction uses the public pre-trained model as a reference rather than as a neutral background artifact. Intuitively, the score difference isolates the effect of fine-tuning: if a sample becomes unusually favorable under $\mathcal M_{ft}$7 relative to $\mathcal M_{ft}$8, that increase is treated as evidence of membership (Ran et al., 24 Jul 2025).

This calibration matters because it changes the privacy interpretation of LoRA. The released adapter alone is not the full privacy boundary. Under the pre-training and fine-tuning paradigm, the combination of a known base model and a released adapted model creates a two-model comparison channel. A plausible implication is that privacy risk in parameter-efficient fine-tuning is partly relational rather than purely local to the adapter weights.

3. Attack inventory and scoring families

LoRA-Leak is described as a holistic evaluation framework because it integrates fifteen membership inference attacks: ten existing attacks and five improved attacks that explicitly leverage the pre-trained model as a reference (Ran et al., 24 Jul 2025).

The ten non-referenced baselines span several signal families. Loss-based attacks include LOSS and zlib. Local geometry and perturbation methods include Neighborhood, SPV, and MoPe. Token-centric scoring includes Min-K % and Min-K %++. Gradient-based signals include GradNorm$\mathcal M_{ft}$9 and GradNorm$$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$0. The framework also treats a pure shadow-model LiRA-style attack for comparison, even though it was not originally evaluated on LoRA (Ran et al., 24 Jul 2025).

The five enhanced attacks apply pre-trained-reference calibration to existing scores. In the framework, LOSS becomes LiRA-style calibration against $$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$1, and analogous reference-subtraction variants are defined for Neighborhood, MoPe, Min-K % and Min-K %++, and GradNorm$$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$2 (Ran et al., 24 Jul 2025).

Attack family	Representative methods
Loss and compression	LOSS, zlib
Neighborhood and perturbation	Neighborhood, SPV, MoPe
Token-based	Min-K %, Min-K %++
Gradient-based	GradNorm$$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$3, GradNorm$$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$4
Pre-trained referenced	LiRA-style LOSS calibration, Neighborhood+Pre, MoPe+Pre, Min-K %+Pre, Min-K %+++Pre, GradNorm$$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$5+Pre

The framework’s attack taxonomy is significant because it shifts evaluation away from any single leakage proxy. Rather than assuming that membership is visible only through sequence loss, LoRA-Leak evaluates multiple internal-state signatures—loss, local sharpness, perturbation sensitivity, token-probability tails, and gradient norms—and then shows that nearly all of them become stronger once the public pre-trained model is introduced as a reference (Ran et al., 24 Jul 2025).

4. Experimental design and empirical findings

The experimental study covers three pre-trained backbones—GPT-2 XL (1.5 B), Pythia-2.8B, and Llama-2-7B—across three downstream tasks: AG News, OAsst TOP-1, and MedQA. Each dataset is subsampled to approximately 10 K training examples and 1 K validation examples, and membership evaluation uses 512 members versus 512 non-members. The default LoRA setting is deliberately conservative: rank $$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$6, scaling $$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$7, batch size $$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$8, dropout $$W_i' = W_i^0 + A_iB_i,\quad A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).$$9, AdamW with learning rate $r$0, no weight decay, $r$1 epochs, and maximum length $r$2 tokens (Ran et al., 24 Jul 2025).

Under this conservative configuration, LoRA-Leak still reports substantial privacy risk. The best attack AUCs reach $r$3–$r$4 on GPT-2 and Pythia, and $r$5–$r$6 on Llama-2, with MedQA reaching $r$7. Pre-trained-model calibration consistently improves attack strength by $r$8–$r$9 AUC points; one reported example is Min-K %++ on Llama-2 plus MedQA, which rises from $5$0 to $5$1 when the pre-trained reference is used (Ran et al., 24 Jul 2025).

The framework also maps how privacy changes across fine-tuning conditions. As epochs increase from $5$2 to $5$3, the generalization gap grows and attack AUCs rise toward approximately $5$4, indicating a strong positive relationship between overfitting and membership inference success. At high overfitting, some non-referenced attacks approach or surpass referenced ones because both score families saturate (Ran et al., 24 Jul 2025).

Module selection within the transformer matters materially. LoRA can be attached to attention projections, output projections, MLP up and down projections, and gating layers. Excluding the MLP up-projection layer alone reduces the best AUC by $5$5 on AG News, and more generally layer exclusion can reduce the best AUC from $5$6 to $5$7 with only minor perplexity change. When the “u” and “g” layers are removed simultaneously, the best AUC drops from $5$8 to $5$9 on AG News, from $10\times$0 to $10\times$1 on OAsst, and from $10\times$2 to $10\times$3 on MedQA, while PPL@val increases by only $10\times$4–$10\times$5 (Ran et al., 24 Jul 2025).

LoRA variants and quantization alter the privacy-utility profile but do not remove the threat. DoRA slightly increases leakage, with AUC gains of $10\times$6–$10\times$7 at minor utility cost. qLoRA Int8 reduces AUC by $10\times$8–$10\times$9, and FP4 reduces it by a further $\mathcal M_{pt}$0–$\mathcal M_{pt}$1, but perplexity worsens. Across pre-trained model scales, experiments on Pythia $\mathcal M_{pt}$2M to $\mathcal M_{pt}$3B and GPT-2 $\mathcal M_{pt}$4M to $\mathcal M_{pt}$5B show AUC peaking around the $\mathcal M_{pt}$6B scale and then declining slightly, while pre-trained-reference attacks remain stronger than their non-referenced counterparts (Ran et al., 24 Jul 2025).

5. Defenses and the utility–privacy trade-off

LoRA-Leak evaluates four defenses: dropout, weight decay, differential privacy, and excluding specific layers from LoRA adaptation. The results are uneven, and the framework concludes that only dropout and selective layer exclusion effectively mitigate membership inference while maintaining model utility (Ran et al., 24 Jul 2025).

Dropout is the most practical mitigation in the study. Increasing LoRA fine-tuning dropout from $\mathcal M_{pt}$7 to $\mathcal M_{pt}$8 reduces the best AUC from $\mathcal M_{pt}$9 to $\mathcal M_{ft}$0. The reported mechanism is that dropout injects noise during adapter training and thereby reduces memorization of individual samples. Utility remains relatively stable: PPL@val stays within $\mathcal M_{ft}$1 up to dropout $\mathcal M_{ft}$2, whereas dropout above $\mathcal M_{ft}$3 harms generalization (Ran et al., 24 Jul 2025).

Weight decay is largely ineffective. AdamW weight decay over $\mathcal M_{ft}$4 changes AUC by only about $\mathcal M_{ft}$5. Differential privacy is effective in a narrow technical sense but costly in practice. DPLoRA with privacy budgets $\mathcal M_{ft}$6 reduces attack AUC to approximately $\mathcal M_{ft}$7 for all attacks, but PPL@val rises by $\mathcal M_{ft}$8–$\mathcal M_{ft}$9 and runtime increases by $x$0 to $x$1 (Ran et al., 24 Jul 2025).

Selective layer exclusion is the other lightweight defense favored by the paper. The operational recommendation is to omit LoRA adapters on the MLP up-projection and, where appropriate, the gating layers. Because the frozen backbone remains untouched, this mitigation targets the adapter insertion pattern rather than the base model or the optimization pipeline (Ran et al., 24 Jul 2025).

Taken together, the defense results define a specific utility–privacy frontier for LoRA fine-tuning. LoRA-Leak does not claim that LoRA is uniquely privacy-destructive; rather, it shows that practical LoRA configurations expose measurable membership risk, and that moderate regularization plus architecture-aware adapter placement can reduce that risk without the severe degradation associated with differential privacy.

6. Relation to adjacent privacy and security work

LoRA-Leak belongs to a broader line of work showing that LoRA is not intrinsically privacy-preserving, but it targets a specific threat model: membership inference against the fine-tuning corpus of LLMs. Adjacent work extends the threat surface to direct extraction, generative memorization, and privacy amplification under further fine-tuning (Ran et al., 24 Jul 2025).

In diffusion models, “Shake to Leak” shows that LoRA-based fine-tuning on a synthetic private set can amplify pre-existing generative privacy risk. On Stable Diffusion v1-1, S2L plus LoRA raises MIA AUC to $x$2 with $x$3 and yields $x$4 extracted samples at $x$5 precision; combining LoRA with DreamBooth further raises AUC to $x$6 and yields $x$7 samples extracted at $x$8 precision (Li et al., 2024). This establishes that LoRA can interact with memorization mechanisms even when the fine-tuning data are synthetic.

In LLMs, Marinelli and Eckhoff fine-tune OPT-1.3B on customer-support data interleaved with $x$9 RockYou passwords and exactly recover $x\in\mathcal D_{ft}$0 of the injected passwords through a simple prompt. They then use causal tracing to localize the leakage mainly to block 21’s “fc1” sub-layer and apply ROME with $x\in\mathcal D_{ft}$1, reducing recoverable passwords from $x\in\mathcal D_{ft}$2 to $x\in\mathcal D_{ft}$3 (Marinelli et al., 29 Mar 2025). This is a direct extraction result rather than membership inference, but it reinforces the broader claim that low-rank adaptation does not by itself prevent sensitive-string memorization.

At the same time, “Leaner Training, Lower Leakage” reports that LoRA significantly reduces memorization risks relative to full fine-tuning under plagiarism-based and similarity-based metrics, with LoRA and head-only fine-tuning staying near $x\in\mathcal D_{ft}$4 plagiarism across scale and keeping fewer than $x\in\mathcal D_{ft}$5 of samples above cosine similarity $x\in\mathcal D_{ft}$6 in key settings (Wang et al., 25 Jun 2025). This suggests that LoRA-Leak and exact-match or similarity-based memorization studies are measuring different privacy phenomena. A plausible implication is that low direct-copy leakage and elevated membership inference can coexist, especially when the attacker can compare a fine-tuned model against its public pre-trained ancestor.

Within this literature, LoRA-Leak’s distinctive contribution is methodological. It reframes the public base model from background infrastructure into an explicit inference resource and, in doing so, argues that privacy assessment for parameter-efficient fine-tuning must be conducted at the level of the full deployment stack—pre-trained checkpoint, adapter configuration, optimization regime, and release practice—rather than at the adapter alone (Ran et al., 24 Jul 2025).