RecLiRA: Interaction-Level Privacy Attack

• RecLiRA is an interaction-level membership inference attack for recommender systems that quantifies membership privacy risks using calibrated shadow models.
• It adapts LiRA methodology for models like NCF and LightGCN by estimating TPR and FPR in the low false-positive regime critical for differential privacy auditing.
• Empirical evaluations on datasets such as MovieLens and Amazon show RecLiRA’s superior AUC and precise interaction-level risk scoring for privacy auditing.

Searching arXiv for the primary RecLiRA source and closely related membership-inference work. RecLiRA is an interaction-level membership inference attack for recommender systems that was introduced as the critical attack component of the RecPS privacy risk scoring framework. In that setting, a recommender model $f_\theta$ exposes scores or probabilities for user–item pairs $(u,i)$, and the adversarial task is to test $H_0:(u,i)\notin D$ versus $H_1:(u,i)\in D$ using the model output for that pair. RecLiRA adapts LiRA to recommender models with probability outputs, especially NCF and LightGCN, in order to obtain high-quality estimates of $\mathrm{TPR}$ and $\mathrm{FPR}$ in the low-FPR regime required by DP-style privacy auditing. Those estimates are then used by RecPS to compute interaction-level and user-level privacy scores (He et al., 24 Jul 2025).

1. Position within recommender-system privacy auditing

Modern recommender systems are trained on sensitive user–item interaction data, including explicit ratings and implicit clicks. In the formulation used by RecPS, the relevant privacy question is not merely whether a model performs well, but whether an adversary with black-box API access to scores or probabilities for $(u,i)$ can infer that a specific interaction, or a specific user, appeared in the training data. RecLiRA is designed for precisely this interaction-level inference problem (He et al., 24 Jul 2025).

The motivation for RecLiRA follows from two observations. First, heuristics such as “high prediction score implies membership” or thresholding raw loss or gradients are described as neither calibrated nor theoretically grounded; they also ignore the trade-off between true and false positives and do not connect naturally to differential privacy. Second, under the DP hypothesis testing view, privacy risk is governed by how well an attacker can distinguish neighboring datasets that differ in one record. RecPS therefore treats a high-quality membership inference attack as the appropriate route to quantitative privacy scoring, and RecLiRA is the attack that supplies those measurements (He et al., 24 Jul 2025).

A common source of confusion is nomenclature. The string “LiRA” also appears in unrelated contexts, including audiovisual self-supervision in “LiRA: Learning Visual Speech Representations from Audio through Self-supervision” (Ma et al., 2021), and the label “RecLiRA” is also associated in one language-identification setting with a reciprocal-rank classifier (Widdows et al., 2021). In the recommender-systems privacy literature considered here, however, RecLiRA denotes the recommender-specific membership inference attack used inside RecPS (He et al., 24 Jul 2025).

2. Differential-privacy linkage and scoring objective

RecLiRA is not presented as an isolated attack, but as the mechanism that makes the RecPS scoring rule operational. The RecPS interaction-level score is motivated by the relaxed $(\epsilon,\delta)$-DP condition

$$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$

together with the auditing observation that the per-record bound $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$ links attack operating characteristics to privacy loss (He et al., 24 Jul 2025).

For a non-DP modeling method $(u,i)$0 and adjacent datasets $(u,i)$1 differing by the interaction $(u,i)$2, the interaction-level privacy score is defined as

$(u,i)$3

RecPS then estimates this quantity through an interaction-level attack $(u,i)$4 by

$(u,i)$5

with $(u,i)$6. Improving the attack therefore tightens the estimate, which is why RecLiRA’s low-FPR performance is central rather than incidental (He et al., 24 Jul 2025).

At the user level, if $(u,i)$7 is the interaction set of user $(u,i)$8, the paper states the bound

$(u,i)$9

To avoid over-penalizing active users, RecPS uses the mean aggregation

$H_0:(u,i)\notin D$0

and analogously

$H_0:(u,i)\notin D$1

This aggregation choice is also tied to the paper’s interpretation of popularity and sparsity effects: users with many interactions do not automatically receive larger risk solely due to volume; instead, risk reflects how sensitive their interactions are on average (He et al., 24 Jul 2025).

3. Attack construction and statistical mechanism

RecLiRA adapts LiRA to interaction-level membership inference for recommenders with probability outputs. Its offline stage trains $H_0:(u,i)\notin D$2 shadow models $H_0:(u,i)\notin D$3 with identical architecture and training regimen as the target model. For each shadow dataset $H_0:(u,i)\notin D$4, each interaction in $H_0:(u,i)\notin D$5 is sampled with probability $H_0:(u,i)\notin D$6, so that an interaction is labeled IN for a shadow model if $H_0:(u,i)\notin D$7 and OUT otherwise. The paper recommends $H_0:(u,i)\notin D$8 shadow models for stable $H_0:(u,i)\notin D$9 estimation (He et al., 24 Jul 2025).

The attack feature begins by treating the recommender as a binary classifier with confidence vector $H_1:(u,i)\in D$0. RecLiRA then uses the margin-like confidence

$H_1:(u,i)\in D$1

followed by the logit transform

$H_1:(u,i)\in D$2

Empirically, $H_1:(u,i)\in D$3 for OUT samples is approximately Gaussian, and this transformed quantity is the attack feature. The offline stage estimates the OUT distribution parameters $H_1:(u,i)\in D$4 of $H_1:(u,i)\in D$5 using OUT samples from shadow models, with a single OUT distribution shared across shadow models to reduce cost (He et al., 24 Jul 2025).

For a shadow model $H_1:(u,i)\in D$6 and interaction $H_1:(u,i)\in D$7, RecLiRA defines the one-sided score

$H_1:(u,i)\in D$8

where $H_1:(u,i)\in D$9 in the notation of the paper. Large $\mathrm{TPR}$0 indicates IN-like behavior under the OUT distribution’s tail. Rather than imposing a fixed global threshold such as $\mathrm{TPR}$1, RecLiRA performs interaction-specific calibration: it sweeps candidate thresholds drawn from OUT models’ $\mathrm{TPR}$2 values and selects the threshold that maximizes $\mathrm{TPR}$3 for that interaction. This threshold search is the step that yields the empirical $\mathrm{TPR}$4 used by RecPS (He et al., 24 Jul 2025).

The paper explicitly contrasts this implementation with classical LiRA formulations that use the log-likelihood ratio

$\mathrm{TPR}$5

and the posterior

$\mathrm{TPR}$6

RecLiRA, as implemented in RecPS, does not require modeling $\mathrm{TPR}$7 explicitly and does not require choosing a prior $\mathrm{TPR}$8 (He et al., 24 Jul 2025).

4. Algorithms, complexity, and operational settings

The paper specifies an offline preparation stage and an online scoring stage. In the offline stage, the inputs are the training dataset $\mathrm{TPR}$9 and the number of shadow models $\mathrm{FPR}$0. For each $\mathrm{FPR}$1, it samples $\mathrm{FPR}$2 by including each interaction with probability $\mathrm{FPR}$3, trains a shadow model $\mathrm{FPR}$4 with the same architecture and optimization as the target, and collects $\mathrm{FPR}$5 for a subset of OUT interactions $\mathrm{FPR}$6 to estimate the Gaussian $\mathrm{FPR}$7 with parameters $\mathrm{FPR}$8 (He et al., 24 Jul 2025).

In the online ScoreQuery stage, the inputs are $\mathrm{FPR}$9, $(u,i)$0, $(u,i)$1, $(u,i)$2, and a user’s interaction set $(u,i)$3. For each interaction $(u,i)$4, the procedure computes $(u,i)$5 for each shadow model, records the ground-truth membership label $(u,i)$6, constructs candidate thresholds from OUT shadow models’ $(u,i)$7 values, and for each candidate threshold computes $(u,i)$8 and $(u,i)$9 over the shadow models. The interaction score is the maximum $(\epsilon,\delta)$0 over those thresholds, and the user-level score is the mean over the user’s interactions (He et al., 24 Jul 2025).

The stated complexity is $(\epsilon,\delta)$1 offline, where $(\epsilon,\delta)$2 is the cost to train one shadow model, $(\epsilon,\delta)$3 is the cost to apply a model to one interaction, and $(\epsilon,\delta)$4 is the number of OUT samples used to fit $(\epsilon,\delta)$5. Online cost per interaction is $(\epsilon,\delta)$6. The paper further recommends $(\epsilon,\delta)$7 OUT samples to fit the Gaussian parameters reliably. For target and shadow training, the architecture and hyperparameters are shared, with SGD, batch size $(\epsilon,\delta)$8, learning rate $(\epsilon,\delta)$9, up to $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$0 epochs with early stopping, and $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$1 negative sampling (He et al., 24 Jul 2025).

These operational details are significant because RecLiRA depends on close target–shadow alignment. The paper states that target and shadow models share architecture, that user splits ensure no overlap between target and shadow users, and that RecLiRA requires probability outputs, which is why the experiments focus on NCF and LightGCN rather than classical MF or ALS (He et al., 24 Jul 2025).

5. Empirical evaluation and observed behavior

The empirical evaluation in RecPS uses MovieLens-1M, Amazon Digital Music, and Amazon Beauty, after preprocessing that keeps users with more than 20 interactions, applies a chronological split with the last interaction as test and the second-last as validation, and uses the remaining interactions for training with $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$2 negative sampling. The target models are NCF and LightGCN, with embedding dimension $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$3 and, for LightGCN, $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$4 GCN layers. Attack quality is measured by MIA AUC and TPR at low FPR, while recommender utility is measured by HitRate@100 (He et al., 24 Jul 2025).

The paper compares RecLiRA with MINER, an interaction-level baseline adapted to NCF and LightGCN. The reported AUC values are as follows.

Dataset	RecLiRA AUC	MINER AUC
MovieLens-1M, LightGCN	0.9430	0.8310
MovieLens-1M, NCF	0.9169	0.8727
Amazon Digital Music, LightGCN	0.9988	0.8449
Amazon Digital Music, NCF	0.9984	0.4888
Amazon Beauty, LightGCN	0.9971	0.9226
Amazon Beauty, NCF	0.9984	0.6786

Across datasets, the paper states that RecLiRA achieves near-perfect AUC on the Amazon datasets and substantially higher TPR in the low-FPR region than MINER, which is precisely the regime that drives large $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$5 values and therefore precise RecPS scoring (He et al., 24 Jul 2025).

At the user level, for randomly selected 60% of users, $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$6 lies roughly in $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$7, with the top $$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$8–$$\Pr(M(D_0)\in O)\le e^\epsilon \Pr(M(D_1)\in O)+\delta,$$9 higher, the bottom $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$0–$\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$1 lower, and a middle band clustered. LightGCN typically yields higher scores than NCF, which the paper describes as consistent with stronger HR@100. The paper also notes popularity effects: because scoring is per interaction and then averaged, popular items or users that yield less distinctive margins tend to receive lower interaction-level scores, whereas rare or highly memorized interactions receive higher scores (He et al., 24 Jul 2025).

6. Unlearning use cases, limitations, and interpretive boundaries

RecLiRA’s practical value in RecPS is tied to privacy-aware removal and unlearning. The paper reports that removing entire top-$\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$2 sensitive users by $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$3 significantly degrades recommender utility. On Amazon Digital Music, removing the top 5% users reduces HR@100 by 10.79% for LightGCN and 37.05% for NCF. By contrast, score-guided partial interaction removal preserves utility better: removing the top 70% sensitive interactions of the top 5% users demotes 100% of those users below a cutoff $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$4, defined as the minimum score among the original top 5% users, while reducing HR@100 by 9.71% for LightGCN and 15.56% for NCF. The paper further states that random interaction removal achieves far less risk reduction for comparable utility drops (He et al., 24 Jul 2025).

The threat model is explicitly black-box: the adversary has access to the deployed recommendation API and can query scores or probabilities for $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$5, but has no access to offline training. RecPS uses RecLiRA to simulate such a powerful adversary, so that model owners can quantify risk and contributors can request removal or unlearning of high-risk interactions. In that sense, RecLiRA is an auditing and prioritization mechanism rather than a privacy defense by itself (He et al., 24 Jul 2025).

Several limitations are stated. RecLiRA applies to recommender models that output probabilities, such as NCF and LightGCN, and does not directly apply to pure scoring or ranking models that cannot be cast as binary classification, including classical MF and ALS. Its OUT distribution model for $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$6 is Gaussian; this is described as an empirical assumption standard in LiRA and effective in the reported experiments. The method relies on many shadow models, approximately $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$7, to stabilize $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$8 estimates. It also provides no formal $\ln(\mathrm{TPR}/\mathrm{FPR})\le \epsilon$9 guarantee for non-DP training: the scores are lower bounds on per-interaction distinguishability expressed as $(u,i)$00, not certified privacy parameters (He et al., 24 Jul 2025).

The paper also discusses dynamic effects after removal. Deleting the top 5% sensitive users slightly changes remaining users’ scores, with differences within $(u,i)$01; roughly 10–20% of users increase slightly, while most decrease or remain unchanged. Interaction-level removal weakens this “privacy onion effect,” reducing the fraction of score-increased users by approximately 1–7%. This suggests that RecLiRA-enabled auditing is not a one-shot procedure: risk should be recomputed after selective deletion if privacy reduction and utility preservation are both operational requirements (He et al., 24 Jul 2025).