--- title: Partial Feature Membership Inference url: https://www.emergentmind.com/topics/partial-feature-membership-inference-pfmi type: topic --- # Partial Feature Membership Inference Searching arXiv for the specified PFMI papers and closely related work. Partial Feature Membership Inference (PFMI) denotes an inference setting in which an adversary observes only partial information about a sample and seeks to determine whether the observed subset was present in a model’s training data. In the classifier setting, PFMI is defined over masked inputs and asks whether there exists a training sample whose observed coordinates match the available partial observation; in the fine-tuned LLM setting, PFMI appears as fragment-level inference from unordered partial evidence, where an adversary uses a set of observed fragments to infer an additional fragment and, implicitly, whether the model’s behavior reflects memorization or membership-like leakage [2508.06244] [2505.13819]. PFMI therefore extends classical membership inference beyond full-sample access and ordered-prefix assumptions, and shifts the attack surface to masked images, incomplete tabular records, and fragmentary textual evidence. ## 1. Formal definitions and attack objectives In the classifier formalization, PFMI is posed for a trained $K$-class classifier $f_\theta : X \to \Delta^K$ with parameters $\theta$ learned by training algorithm $T$ on a dataset $D_{\text{train}} \subset X \times \{1,\dots,K\}$ drawn from distribution $\pi$. Let $x \in X$ be a target input with label $y$. A binary mask $M \in \{0,1\}^d$ partitions the $d$-dimensional input into observed and missing features, so that $x = (x_O, x_M)$, where $x_O = x \odot M$ and $x_M = x \odot (1-M)$. The adversary receives $x_O$ and aims to infer whether the observed subset $x_O$ “appears in” the training set. The associated membership indicator is $$ m = \begin{cases} 1 & \text{if } \exists\, x' \in D_{\text{train}} \text{ with } x' \odot M = x \odot M,\\ 0 & \text{otherwise.} \end{cases} $$ This definition is explicitly distinct from full-sample membership inference, which asks whether $x \in D_{\text{train}}$ [2508.06244]. In the LLM instantiation, the adversary observes an unordered set of fragments $F=\{f_i\}$ extracted from a single individual’s sample $s \in D$ and wishes to infer whether a sensitive fragment $t$ appears in $s$. The decision variable is $y \in \{0,1\}$, where $y=1$ indicates presence of $t$ in $s$. The attack also seeks evidence that the fine-tuned model’s conditional probability $p_M(t\mid F)$ reflects training-set membership of $s$, linking fragment inference to memorization and membership-like leakage [2505.13819]. | Setting | Observed information | Inference target | |---|---|---| | Classifier PFMI | Masked feature subset $x_O$ under mask $M$ | Whether there exists $x' \in D_{\text{train}}$ with $x' \odot M = x_O$ | | LLM fragment-level PFMI | Unordered fragment set $F$ | Whether an additional fragment $t$ appears in $s$, and whether $p_M(t\mid F)$ carries membership-like signal | These formulations share the same structural change relative to standard MIA: the adversary does not control or observe a full sample. A plausible implication is that PFMI broadens privacy analysis from exact-example exposure to partial-pattern exposure, which is often more realistic in deployed systems. ## 2. White-box PFMI for classifiers: MRAD The principal attack framework for classifier PFMI is MRAD, short for Memory-guided Reconstruction and Anomaly Detection. The paper instantiates PFMI in a white-box setting in which the adversary has white-box access to $\theta$, the architecture, the training loss $L$, gradients $\nabla_x L_\theta(x,y)$, internal representations, and any layer features used by a detector; knows $\pi$ up to sampling and can draw auxiliary data $D_{\text{aux}}$; can obtain at least one valid class label $y$ compatible with $x_O$; and knows the arbitrary missingness mask $M$ [2508.06244]. MRAD proceeds in two stages. In Stage 1, the adversary reconstructs the missing features by minimizing the training loss with respect to the unobserved coordinates: $$ \min_{x_M} L_\theta(x_O \oplus x_M, y). $$ For cross-entropy classification, if $f_\theta(x)=(f_1(x),\dots,f_K(x)) \in \Delta^K$ and $L_\theta(x,y)=-\log f_y(x)$, the objective becomes $$ \min_{x_M \in X_M} -\log f_y(x_O \oplus x_M). $$ Optimization is performed by initializing $x_M=0$, then updating only the missing coordinates via projected gradient steps, $$ x_M \leftarrow \Pi_{X_M}\!\left[x_M - \eta \cdot (\nabla_x L_\theta(x_O \oplus x_M, y))_M\right], $$ where $\eta$ is the step length and $\Pi_{X_M}$ enforces domain constraints such as pixel bounds or feature ranges. An optional regularizer $R(x_M\mid x_O)$ may be added, although the reported experiments rely on the model’s training loss without explicit priors. The operational intuition is that overfitted models create loss valleys around training samples. If $x_O$ is drawn from a member-consistent partial observation, gradient descent on the missing coordinates tends to “snap” the reconstruction toward a memorized mode of the training distribution. If $x_O$ is non-member, optimization more often converges to an off-distribution local minimum. The attack therefore uses the model’s loss landscape as memory. In Stage 2, MRAD scores the reconstructed sample $\hat{x}=x_O \oplus \hat{x}_M$ for deviation from the training distribution. The default detector computes class centroids and robust dispersion using Median Absolute Deviation (MAD) from $D_{\text{aux}}$. For class $k$, it defines a centroid $c_k$, median distance $\tilde d_k$, and $\text{MAD}_k$ from Euclidean distances to the centroid. The reconstructed target is assigned a standardized deviation $$ \delta = \frac{\left|\hat d - \tilde d_k\right|}{\text{MAD}_k}, \qquad \hat d = \|\hat{x}-c_k\|_2. $$ A shadow model $\theta_s$ trained on data drawn from $\pi$ provides a reference reconstruction $\hat{x}_s$ and corresponding deviation $\delta_s$. The final membership score is the ratio $$ \operatorname{MRAD}(x_O;\theta,\pi')=\frac{\delta_s}{\delta}, $$ where $\pi'$ is the non-member space excluding samples consistent with $x_O$. The decision rule predicts member if $\operatorname{MRAD}(x_O;\theta,\pi')>\tau$, with $\tau$ calibrated to control false positives on samples from $\pi'$. MRAD is modular in the anomaly detector. The paper explicitly supports centroid/MAD in input or feature space, Mahalanobis distance, CADE, MSAD, and NCI. This modularity matters because PFMI is not reduced to a single detector design; the reconstruction stage is the invariant component, and the detector quantifies distributional alignment after reconstruction. ## 3. Black-box fragment-level PFMI for fine-tuned LLMs For fine-tuned LLMs, PFMI is instantiated as fragment inference under black-box access to log-probabilities. The adversary has no access to weights or training samples, no access to ordered prefixes of the target sample, and only an unordered fragment set $F$, with $|F|$ typically in $[4,30]$. The attack queries three models: the fine-tuned target model $M$, a shadow or reference model $B$ not fine-tuned on the same data, and a “world” ensemble $W$ of LLMs not fine-tuned on either dataset [2505.13819]. The conditional probability of a multi-token target fragment $t=(\tau_1,\dots,\tau_L)$ under an autoregressive model is $$ p_M(t \mid F)=\prod_{j=1}^{L} p_M\!\big(\tau_j \mid \text{Prompt}(F), \tau_{\tau]$ or, equivalently, thresholds the log-likelihood ratio. The raw conditional probability over the sequence is used, equivalently the sum of token-level log-probabilities for numerical stability. No generation is required; only logprob APIs are needed. The second attack, PRISM, regularizes LR by introducing a world prior. Its stated motivation is that LR can report high scores for general associations that reflect population-level correlations rather than memorization of a specific individual’s sample. The LR-based membership posterior is $$ \widehat{\Pr}(M \mid \hat{\ell}) = \frac{\beta \hat{\ell}^{2}}{\beta \hat{\ell}^{2} + (1-\beta)}, \qquad \hat{\ell}=\frac{p_M(t\mid F)}{p_B(t\mid F)}, $$ where $\beta=\Pr(M)$ is a prior. The PRISM score is then $$ \operatorname{PRISM}(F,t) = \frac{p_W(t\mid F) - p_B(t\mid F)\big(1-\widehat{\Pr}(M \mid \hat{\ell})\big)} {\widehat{\Pr}(M \mid \hat{\ell})}. $$ Thresholding $\operatorname{PRISM}(F,t)$ yields the final decision. The practical workflow therefore consists of prompt construction, log-probability evaluation under $M$, $B$, and $W$, LR or PRISM scoring, and threshold sweeps to obtain ROC curves and low-FPR operating points. This fragment-level PFMI sits between attribute inference and classical membership inference. The attacker does not infer full-sample membership directly; instead, the attack infers a missing fragment conditioned on partial fragments while interpreting the resulting probability shift as evidence of data-specific memorization or membership-like leakage. ## 4. Empirical behavior across modalities In the classifier setting, PFMI is evaluated on CIFAR-10, Fashion-MNIST, STL-10, and Epsilon, with AlexNet used for images and an MLP for Epsilon. Known-feature percentage varies from 10% to 90% in 10% increments, plus a leave-one-out case with one missing feature. The principal metrics are area under the ROC curve (AUC) and $\text{TPR@FPR}=0.1$. The reported findings show that MRAD begins to produce meaningful signal at 50% known features and improves with more known features; on STL-10 and Epsilon, MRAD achieves AUC $\approx 0.6$ with 40% missing features, and on STL-10 the attack achieves an AUC of around 0.6 even with 40% of the missing features [2508.06244]. The classifier experiments also report several sensitivity studies. With Source: https://www.emergentmind.com/topics/partial-feature-membership-inference-pfmi