Partial Feature Membership Inference

• PFMI is a technique that infers model training data membership from incomplete observations such as masked features or unordered text fragments.
• It employs methods like MRAD for white-box gradient-based reconstruction and likelihood ratio tests in black-box LLM settings to detect memorization.
• PFMI broadens traditional membership inference by addressing multiple modalities including images, tabular data, and text to enhance privacy analysis.

Searching arXiv for the specified PFMI papers and closely related work. Partial Feature Membership Inference (PFMI) denotes an inference setting in which an adversary observes only partial information about a sample and seeks to determine whether the observed subset was present in a model’s training data. In the classifier setting, PFMI is defined over masked inputs and asks whether there exists a training sample whose observed coordinates match the available partial observation; in the fine-tuned LLM setting, PFMI appears as fragment-level inference from unordered partial evidence, where an adversary uses a set of observed fragments to infer an additional fragment and, implicitly, whether the model’s behavior reflects memorization or membership-like leakage (Wang et al., 8 Aug 2025, Rosenblatt et al., 20 May 2025). PFMI therefore extends classical membership inference beyond full-sample access and ordered-prefix assumptions, and shifts the attack surface to masked images, incomplete tabular records, and fragmentary textual evidence.

1. Formal definitions and attack objectives

In the classifier formalization, PFMI is posed for a trained $K$-class classifier $f_\theta : X \to \Delta^K$ with parameters $\theta$ learned by training algorithm $T$ on a dataset $D_{\text{train}} \subset X \times \{1,\dots,K\}$ drawn from distribution $\pi$. Let $x \in X$ be a target input with label $y$. A binary mask $M \in \{0,1\}^d$ partitions the $d$-dimensional input into observed and missing features, so that $f_\theta : X \to \Delta^K$0, where $f_\theta : X \to \Delta^K$1 and $f_\theta : X \to \Delta^K$2. The adversary receives $f_\theta : X \to \Delta^K$3 and aims to infer whether the observed subset $f_\theta : X \to \Delta^K$4 “appears in” the training set. The associated membership indicator is

$f_\theta : X \to \Delta^K$5

This definition is explicitly distinct from full-sample membership inference, which asks whether $f_\theta : X \to \Delta^K$6 (Wang et al., 8 Aug 2025).

In the LLM instantiation, the adversary observes an unordered set of fragments $f_\theta : X \to \Delta^K$7 extracted from a single individual’s sample $f_\theta : X \to \Delta^K$8 and wishes to infer whether a sensitive fragment $f_\theta : X \to \Delta^K$9 appears in $\theta$0. The decision variable is $\theta$1, where $\theta$2 indicates presence of $\theta$3 in $\theta$4. The attack also seeks evidence that the fine-tuned model’s conditional probability $\theta$5 reflects training-set membership of $\theta$6, linking fragment inference to memorization and membership-like leakage (Rosenblatt et al., 20 May 2025).

Setting	Observed information	Inference target
Classifier PFMI	Masked feature subset $\theta$7 under mask $\theta$8	Whether there exists $\theta$9 with $T$0
LLM fragment-level PFMI	Unordered fragment set $T$1	Whether an additional fragment $T$2 appears in $T$3, and whether $T$4 carries membership-like signal

These formulations share the same structural change relative to standard MIA: the adversary does not control or observe a full sample. A plausible implication is that PFMI broadens privacy analysis from exact-example exposure to partial-pattern exposure, which is often more realistic in deployed systems.

2. White-box PFMI for classifiers: MRAD

The principal attack framework for classifier PFMI is MRAD, short for Memory-guided Reconstruction and Anomaly Detection. The paper instantiates PFMI in a white-box setting in which the adversary has white-box access to $T$5, the architecture, the training loss $T$6, gradients $T$7, internal representations, and any layer features used by a detector; knows $T$8 up to sampling and can draw auxiliary data $T$9; can obtain at least one valid class label $D_{\text{train}} \subset X \times \{1,\dots,K\}$0 compatible with $D_{\text{train}} \subset X \times \{1,\dots,K\}$1; and knows the arbitrary missingness mask $D_{\text{train}} \subset X \times \{1,\dots,K\}$2 (Wang et al., 8 Aug 2025).

MRAD proceeds in two stages. In Stage 1, the adversary reconstructs the missing features by minimizing the training loss with respect to the unobserved coordinates:

$D_{\text{train}} \subset X \times \{1,\dots,K\}$3

For cross-entropy classification, if $D_{\text{train}} \subset X \times \{1,\dots,K\}$4 and $D_{\text{train}} \subset X \times \{1,\dots,K\}$5, the objective becomes

$D_{\text{train}} \subset X \times \{1,\dots,K\}$6

Optimization is performed by initializing $D_{\text{train}} \subset X \times \{1,\dots,K\}$7, then updating only the missing coordinates via projected gradient steps,

$D_{\text{train}} \subset X \times \{1,\dots,K\}$8

where $D_{\text{train}} \subset X \times \{1,\dots,K\}$9 is the step length and $\pi$0 enforces domain constraints such as pixel bounds or feature ranges. An optional regularizer $\pi$1 may be added, although the reported experiments rely on the model’s training loss without explicit priors.

The operational intuition is that overfitted models create loss valleys around training samples. If $\pi$2 is drawn from a member-consistent partial observation, gradient descent on the missing coordinates tends to “snap” the reconstruction toward a memorized mode of the training distribution. If $\pi$3 is non-member, optimization more often converges to an off-distribution local minimum. The attack therefore uses the model’s loss landscape as memory.

In Stage 2, MRAD scores the reconstructed sample $\pi$4 for deviation from the training distribution. The default detector computes class centroids and robust dispersion using Median Absolute Deviation (MAD) from $\pi$5. For class $\pi$6, it defines a centroid $\pi$7, median distance $\pi$8, and $\pi$9 from Euclidean distances to the centroid. The reconstructed target is assigned a standardized deviation

$x \in X$0

A shadow model $x \in X$1 trained on data drawn from $x \in X$2 provides a reference reconstruction $x \in X$3 and corresponding deviation $x \in X$4. The final membership score is the ratio

$x \in X$5

where $x \in X$6 is the non-member space excluding samples consistent with $x \in X$7. The decision rule predicts member if $x \in X$8, with $x \in X$9 calibrated to control false positives on samples from $y$0.

MRAD is modular in the anomaly detector. The paper explicitly supports centroid/MAD in input or feature space, Mahalanobis distance, CADE, MSAD, and NCI. This modularity matters because PFMI is not reduced to a single detector design; the reconstruction stage is the invariant component, and the detector quantifies distributional alignment after reconstruction.

3. Black-box fragment-level PFMI for fine-tuned LLMs

For fine-tuned LLMs, PFMI is instantiated as fragment inference under black-box access to log-probabilities. The adversary has no access to weights or training samples, no access to ordered prefixes of the target sample, and only an unordered fragment set $y$1, with $y$2 typically in $y$3. The attack queries three models: the fine-tuned target model $y$4, a shadow or reference model $y$5 not fine-tuned on the same data, and a “world” ensemble $y$6 of LLMs not fine-tuned on either dataset (Rosenblatt et al., 20 May 2025).

The conditional probability of a multi-token target fragment $y$7 under an autoregressive model is

$y$8

with analogous definitions for $y$9 and $M \in \{0,1\}^d$0. Prompt templates encode unordered fragments into a task-consistent statement. The medical template is: “Consider a patient whose medical summary contains: $M \in \{0,1\}^d$1. That individual’s medical summary also includes: $M \in \{0,1\}^d$2.” The legal template uses the same pattern for legal document summaries.

The first attack is a likelihood ratio test. It defines

$M \in \{0,1\}^d$3

and predicts $M \in \{0,1\}^d$4 or, equivalently, thresholds the log-likelihood ratio. The raw conditional probability over the sequence is used, equivalently the sum of token-level log-probabilities for numerical stability. No generation is required; only logprob APIs are needed.

The second attack, PRISM, regularizes LR by introducing a world prior. Its stated motivation is that LR can report high scores for general associations that reflect population-level correlations rather than memorization of a specific individual’s sample. The LR-based membership posterior is

$M \in \{0,1\}^d$5

where $M \in \{0,1\}^d$6 is a prior. The PRISM score is then

$M \in \{0,1\}^d$7

Thresholding $M \in \{0,1\}^d$8 yields the final decision. The practical workflow therefore consists of prompt construction, log-probability evaluation under $M \in \{0,1\}^d$9, $d$0, and $d$1, LR or PRISM scoring, and threshold sweeps to obtain ROC curves and low-FPR operating points.

This fragment-level PFMI sits between attribute inference and classical membership inference. The attacker does not infer full-sample membership directly; instead, the attack infers a missing fragment conditioned on partial fragments while interpreting the resulting probability shift as evidence of data-specific memorization or membership-like leakage.

4. Empirical behavior across modalities

In the classifier setting, PFMI is evaluated on CIFAR-10, Fashion-MNIST, STL-10, and Epsilon, with AlexNet used for images and an MLP for Epsilon. Known-feature percentage varies from 10% to 90% in 10% increments, plus a leave-one-out case with one missing feature. The principal metrics are area under the ROC curve (AUC) and $d$2. The reported findings show that MRAD begins to produce meaningful signal at 50% known features and improves with more known features; on STL-10 and Epsilon, MRAD achieves AUC $d$3 with 40% missing features, and on STL-10 the attack achieves an AUC of around 0.6 even with 40% of the missing features (Wang et al., 8 Aug 2025).

The classifier experiments also report several sensitivity studies. With