---
title: 'LoRA-Leak: Privacy Evaluation of LoRA Fine-Tuning'
url: https://www.emergentmind.com/topics/lora-leak
type: topic
---

# LoRA-Leak: Privacy Evaluation of LoRA Fine-Tuning

LoRA-Leak is a membership-inference evaluation framework for Low-Rank Adaptation (LoRA) fine-tuned language models in the standard pre-training plus fine-tuning regime. It studies the case in which a public pre-trained causal language model is adapted to a private downstream dataset through LoRA and the resulting fine-tuned model is released together with its adapters. The framework’s central finding is that parameter-efficient fine-tuning does not eliminate privacy risk: even when only low-rank adapter parameters are trained, fine-tuning-set membership remains inferable, and the publicly available pre-trained model can itself be used as a reference that strengthens the attack surface [2507.18302].

## 1. Conceptual basis and formal setting

LoRA-Leak is grounded in the observation that modern language-model deployment commonly separates a universal pre-trained model, denoted $\mathcal M_{pt}$, from a downstream-specialized model, denoted $\mathcal M_{ft}$, obtained by LoRA fine-tuning. In LoRA, the original weights are frozen and each adapted linear layer replaces the original parameter matrix with a low-rank update:
$$
W_i' = W_i^0 + A_iB_i,\quad
A_i\in\mathbb R^{d\times r},\; B_i\in\mathbb R^{r\times k},\; r\ll\min(d,k).
$$
Because $r$ is small, LoRA adds only a few million parameters on top of billion-parameter backbones, reducing memory and GPU requirements by $5$–$10\times$. This efficiency helped drive widespread adoption, with over 12 000 LoRA models on Hugging Face by end-2023 [2507.18302].

A key misconception addressed by LoRA-Leak is the idea that training only a small side adapter should make fine-tuning data effectively invulnerable to membership inference. The framework rejects that premise. It argues that privacy analysis under LoRA cannot ignore the original public checkpoint, because the pre-trained model encodes a baseline familiarity with general text and provides a direct comparison point for measuring how fine-tuning changed the model’s response to candidate samples [2507.18302].

The threat model is white-box. The adversary knows the public pre-trained checkpoint, has access to the released fine-tuned model and LoRA adapters, and can execute forward and backward passes through both $\mathcal M_{pt}$ and $\mathcal M_{ft}$. The adversary’s objective is standard membership inference: for a candidate sample $x$, decide whether $x\in\mathcal D_{ft}$, where $\mathcal D_{ft}$ is the private fine-tuning corpus [2507.18302].

## 2. Membership inference objective and pre-trained reference calibration

LoRA-Leak formulates membership inference around a score function applied to a candidate sequence. For a prefix $x_{1:n}$, the framework uses the sequence-level cross-entropy
$$
\mathcal L(x;\mathcal M)=
-\frac1{n-1}\sum_{i=1}^{n-1}\log\Pr(x_{i+1}\mid x_{1:i};\mathcal M),
$$
and then defines attack-specific scores $S(x;\mathcal M)$, together with a threshold $\tau$, to predict membership when the score crosses the threshold [2507.18302].

The distinctive technical contribution is the pre-trained-reference calibration. Given any attack score $S$, LoRA-Leak defines
$$
S_{\mathrm{pt\text{-}ref}}(x;\mathcal M_{ft})
=
S(x;\mathcal M_{pt})-S(x;\mathcal M_{ft}).
$$
This construction uses the public pre-trained model as a reference rather than as a neutral background artifact. Intuitively, the score difference isolates the effect of fine-tuning: if a sample becomes unusually favorable under $\mathcal M_{ft}$ relative to $\mathcal M_{pt}$, that increase is treated as evidence of membership [2507.18302].

This calibration matters because it changes the privacy interpretation of LoRA. The released adapter alone is not the full privacy boundary. Under the pre-training and fine-tuning paradigm, the combination of a known base model and a released adapted model creates a two-model comparison channel. A plausible implication is that privacy risk in parameter-efficient fine-tuning is partly relational rather than purely local to the adapter weights.

## 3. Attack inventory and scoring families

LoRA-Leak is described as a holistic evaluation framework because it integrates fifteen membership inference attacks: ten existing attacks and five improved attacks that explicitly leverage the pre-trained model as a reference [2507.18302].

The ten non-referenced baselines span several signal families. Loss-based attacks include LOSS and zlib. Local geometry and perturbation methods include Neighborhood, SPV, and MoPe. Token-centric scoring includes Min-K % and Min-K %++. Gradient-based signals include GradNorm$_\theta$ and GradNorm$_x$. The framework also treats a pure shadow-model LiRA-style attack for comparison, even though it was not originally evaluated on LoRA [2507.18302].

The five enhanced attacks apply pre-trained-reference calibration to existing scores. In the framework, LOSS becomes LiRA-style calibration against $\mathcal M_{pt}$, and analogous reference-subtraction variants are defined for Neighborhood, MoPe, Min-K % and Min-K %++, and GradNorm$_x$ [2507.18302].

| Attack family | Representative methods |
|---|---|
| Loss and compression | LOSS, zlib |
| Neighborhood and perturbation | Neighborhood, SPV, MoPe |
| Token-based | Min-K %, Min-K %++ |
| Gradient-based | GradNorm$_\theta$, GradNorm$_x$ |
| Pre-trained referenced | LiRA-style LOSS calibration, Neighborhood+Pre, MoPe+Pre, Min-K %+Pre, Min-K %+++Pre, GradNorm$_x$+Pre |

The framework’s attack taxonomy is significant because it shifts evaluation away from any single leakage proxy. Rather than assuming that membership is visible only through sequence loss, LoRA-Leak evaluates multiple internal-state signatures—loss, local sharpness, perturbation sensitivity, token-probability tails, and gradient norms—and then shows that nearly all of them become stronger once the public pre-trained model is introduced as a reference [2507.18302].

## 4. Experimental design and empirical findings

The experimental study covers three pre-trained backbones—GPT-2 XL (1.5 B), Pythia-2.8B, and Llama-2-7B—across three downstream tasks: AG News, OAsst TOP-1, and MedQA. Each dataset is subsampled to approximately 10 K training examples and 1 K validation examples, and membership evaluation uses 512 members versus 512 non-members. The default LoRA setting is deliberately conservative: rank $r=4$, scaling $\alpha=2r$, batch size $16$, dropout $5\%$, AdamW with learning rate $10^{-4}$, no weight decay, $3$ epochs, and maximum length $1\,024$ tokens [2507.18302].

Under this conservative configuration, LoRA-Leak still reports substantial privacy risk. The best attack AUCs reach $0.68$–$0.72$ on GPT-2 and Pythia, and $0.73$–$0.77$ on Llama-2, with MedQA reaching $0.775$. Pre-trained-model calibration consistently improves attack strength by $3$–$8$ AUC points; one reported example is Min-K %++ on Llama-2 plus MedQA, which rises from $0.689$ to $0.775$ when the pre-trained reference is used [2507.18302].

The framework also maps how privacy changes across fine-tuning conditions. As epochs increase from $1$ to $10$, the generalization gap grows and attack AUCs rise toward approximately $1.0$, indicating a strong positive relationship between overfitting and membership inference success. At high overfitting, some non-referenced attacks approach or surpass referenced ones because both score families saturate [2507.18302].

Module selection within the transformer matters materially. LoRA can be attached to attention projections, output projections, MLP up and down projections, and gating layers. Excluding the MLP up-projection layer alone reduces the best AUC by $0.15$ on AG News, and more generally layer exclusion can reduce the best AUC from $0.765$ to $0.601$ with only minor perplexity change. When the “u” and “g” layers are removed simultaneously, the best AUC drops from $0.765$ to $0.620$ on AG News, from $0.721$ to $0.672$ on OAsst, and from $0.775$ to $0.710$ on MedQA, while PPL@val increases by only $0.0$–$0.02$ [2507.18302].

LoRA variants and quantization alter the privacy-utility profile but do not remove the threat. DoRA slightly increases leakage, with AUC gains of $0.004$–$0.008$ at minor utility cost. qLoRA Int8 reduces AUC by $0.01$–$0.03$, and FP4 reduces it by a further $0.02$–$0.03$, but perplexity worsens. Across pre-trained model scales, experiments on Pythia $160$M to $12$B and GPT-2 $124$M to $1.5$B show AUC peaking around the $1$B scale and then declining slightly, while pre-trained-reference attacks remain stronger than their non-referenced counterparts [2507.18302].

## 5. Defenses and the utility–privacy trade-off

LoRA-Leak evaluates four defenses: dropout, weight decay, differential privacy, and excluding specific layers from LoRA adaptation. The results are uneven, and the framework concludes that only dropout and selective layer exclusion effectively mitigate membership inference while maintaining model utility [2507.18302].

Dropout is the most practical mitigation in the study. Increasing LoRA fine-tuning dropout from $0.05$ to $0.99$ reduces the best AUC from $0.765$ to $0.553$. The reported mechanism is that dropout injects noise during adapter training and thereby reduces memorization of individual samples. Utility remains relatively stable: PPL@val stays within $\pm 5\%$ up to dropout $0.85$, whereas dropout above $0.95$ harms generalization [2507.18302].

Weight decay is largely ineffective. AdamW weight decay over $\lambda\in\{10^{-4},\ldots,10^{-1}\}$ changes AUC by only about $\pm 0.01$. Differential privacy is effective in a narrow technical sense but costly in practice. DPLoRA with privacy budgets $\epsilon\in\{0.1,1,10\}$ reduces attack AUC to approximately $0.50$ for all attacks, but PPL@val rises by $20$–$80\%$ and runtime increases by $7\times$ to $31\times$ [2507.18302].

Selective layer exclusion is the other lightweight defense favored by the paper. The operational recommendation is to omit LoRA adapters on the MLP up-projection and, where appropriate, the gating layers. Because the frozen backbone remains untouched, this mitigation targets the adapter insertion pattern rather than the base model or the optimization pipeline [2507.18302].

Taken together, the defense results define a specific utility–privacy frontier for LoRA fine-tuning. LoRA-Leak does not claim that LoRA is uniquely privacy-destructive; rather, it shows that practical LoRA configurations expose measurable membership risk, and that moderate regularization plus architecture-aware adapter placement can reduce that risk without the severe degradation associated with differential privacy.

## 6. Relation to adjacent privacy and security work

LoRA-Leak belongs to a broader line of work showing that LoRA is not intrinsically privacy-preserving, but it targets a specific threat model: membership inference against the fine-tuning corpus of language models. Adjacent work extends the threat surface to direct extraction, generative memorization, and privacy amplification under further fine-tuning [2507.18302].

In diffusion models, “Shake to Leak” shows that LoRA-based fine-tuning on a synthetic private set can amplify pre-existing generative privacy risk. On Stable Diffusion v1-1, S2L plus LoRA raises MIA AUC to $0.745$ with $\Delta\mathrm{AUC}=+0.033$ and yields $13.1$ extracted samples at $86.8\%$ precision; combining LoRA with DreamBooth further raises AUC to $0.766$ and yields $15.8$ samples extracted at $88.7\%$ precision [2403.09450]. This establishes that LoRA can interact with memorization mechanisms even when the fine-tuning data are synthetic.

In language models, Marinelli and Eckhoff fine-tune OPT-1.3B on customer-support data interleaved with $200$ RockYou passwords and exactly recover $37$ of the injected passwords through a simple prompt. They then use causal tracing to localize the leakage mainly to block 21’s “fc1” sub-layer and apply ROME with $\beta=0.1$, reducing recoverable passwords from $37$ to $0$ [2504.00031]. This is a direct extraction result rather than membership inference, but it reinforces the broader claim that low-rank adaptation does not by itself prevent sensitive-string memorization.

At the same time, “Leaner Training, Lower Leakage” reports that LoRA significantly reduces memorization risks relative to full fine-tuning under plagiarism-based and similarity-based metrics, with LoRA and head-only fine-tuning staying near $0\%$ plagiarism across scale and keeping fewer than $1\%$ of samples above cosine similarity $0.8$ in key settings [2506.20856]. This suggests that LoRA-Leak and exact-match or similarity-based memorization studies are measuring different privacy phenomena. A plausible implication is that low direct-copy leakage and elevated membership inference can coexist, especially when the attacker can compare a fine-tuned model against its public pre-trained ancestor.

Within this literature, LoRA-Leak’s distinctive contribution is methodological. It reframes the public base model from background infrastructure into an explicit inference resource and, in doing so, argues that privacy assessment for parameter-efficient fine-tuning must be conducted at the level of the full deployment stack—pre-trained checkpoint, adapter configuration, optimization regime, and release practice—rather than at the adapter alone [2507.18302].

Source: https://www.emergentmind.com/topics/lora-leak