---
title: Imitative Membership Inference Attack
url: https://www.emergentmind.com/topics/imitative-membership-inference-attack-imia
type: topic
---

# Imitative Membership Inference Attack

Imitative Membership Inference Attack (IMIA) is a black-box membership inference paradigm in which the adversary replaces target-agnostic shadow training with target-informed imitative training, constructing a small ensemble of models that mimic a target classifier’s behavior and then comparing the target’s observed score on a query to the score distributions induced by imitative “out” and “in” models [2509.06796]. In the formulation introduced in "Imitative Membership Inference Attack" [2509.06796], the target is a classifier \(f_T : \mathcal{X} \to \Delta^c\) queried through softmax posteriors, auxiliary data \(D_{\mathrm{adv}}\) are sampled from the same data distribution, and membership is inferred from a non-parametric discrepancy score built from scaled-confidence margins. The acronym IMIA is not uniform across the literature: it has also been used for "Iterations for Membership Inference Attack," which bases inference on adversarial-generation iteration counts [2506.02711], and related wireless-security work employs an imitative surrogate stage to compensate for over-the-air channel mismatch [2107.12173]. Within current usage, however, IMIA most commonly denotes the target-informed imitative-training attack of 2025 [2509.06796].

## 1. Threat model and conceptual basis

The goal of IMIA is the standard membership inference task: given a target classifier \(f_T\) trained on \(D_{\mathrm{train}} \subset \mathbb{D}\), decide for a queried labeled instance \((x,y)\) whether \(x \in D_{\mathrm{train}}\) or \(x \notin D_{\mathrm{train}}\) [2509.06796]. The attack assumes black-box access to the target’s softmax probabilities with default temperature \(1\); where available, pre-softmax logits improve performance but are not strictly required. Label-only access is not the focus of the method. As in standard MIAs, the adversary may sample auxiliary data from the same distribution in order to construct \(D_{\mathrm{adv}}\) [2509.06796].

The method is defined for two operational settings. In the non-adaptive, or offline, setting, imitative models are trained before the query set \(D_{\mathrm{query}}\) is observed and are then reused across queries. In the adaptive, or online, setting, imitative models may be trained after observing \(D_{\mathrm{query}}\) and may depend on each query instance. The formulation follows the membership inference security game of Du et al. as summarized in the paper [2509.06796].

IMIA is motivated by a limitation of conventional shadow-model attacks. Traditional shadow MIAs train many target-agnostic shadows on random subsets of \(D_{\mathrm{adv}}\), and the resulting membership signals can exhibit high variance across models and instances, especially in difficult cases. Strong attacks such as LiRA and PMIA therefore rely on hundreds of shadows, which induces substantial computational overhead. IMIA conditions model construction on target outputs in order to reduce this variance. The paper makes this intuition explicit through the Law of Total Variance:
\[
\mathbb{E}_{o_T}\big[\operatorname{Var}(z(x)\mid o_T)\big] \le \operatorname{Var}[z(x)],
\]
where \(o_T = \mathcal{Q}(f_T)\) denotes observable outputs of the target. The intended implication is that conditioning on target outputs stabilizes the membership signal \(z(x)\) [2509.06796].

This conditioning is operationalized by training imitative out models to match target behavior on non-members and then fine-tuning those same models on carefully selected pivot instances so as to produce imitative in models. The paper reports that with \(N \approx 10\) imitative models, IMIA can obtain stable in/out score distributions while using less than \(5\%\) of the computational cost of state-of-the-art shadow-based approaches [2509.06796].

## 2. Mathematical formulation and imitative training

The central scalar statistic in IMIA is the scaled-confidence margin. For posterior vector \(p = f(x)\) and true class \(y\), the method defines
\[
\phi(f(x)) = \log f(x)_y - \log\!\Big(\max_{y' \ne y} f(x)_{y'}\Big).
\]
When logits \(z\) are available, the numerically stable form is
\[
\phi(f(x)) = z_y - \max_{y' \ne y} z_{y'}.
\]
The paper treats this quantity as more membership-informative than raw loss and uses it throughout both model construction and final inference [2509.06796].

The first training phase constructs imitative out models \(f_\psi\) by minimizing a weighted mean-squared error between the log probabilities of the imitative model and the target:
\[
\mathcal{L}_{\mathrm{imitate}}(x,y; f_\psi, f_T)
=
\sum_{i=1}^c
w_i(c)\,
\Big(\log f_\psi(x)_i - \log f_T(x)_i\Big)^2.
\]
The weighting scheme emphasizes the ground-truth class \(y\) and the target’s most likely wrong class \(i^* = \arg\max_{y' \ne y} f_T(x)_{y'}\), while the remaining classes receive weight \(1/(c + 2\sqrt{c})\). The paper identifies this weighting as preferable to standard KL-based distillation because it concentrates on the logits viewed as most membership-indicative [2509.06796].

The second phase produces imitative in models by continuing training on a pivot dataset \(D_{\mathrm{pivot}}\) with cross-entropy:
\[
\mathcal{L}_{\mathrm{ce}}(x,y; f_\psi) = -\log f_\psi(x)_y.
\]
This phase is intended to preserve the target-informed behavior learned during imitation while encoding behavior associated with training inclusion [2509.06796].

Pivot selection is itself target-informed. The paper selects, for each class, the \(k\) auxiliary instances with lowest target loss:
\[
D_{\mathrm{pivot}}
=
\bigcup_{y=1}^c
\mathrm{Top}\text{-}k
\Big\{(u,y)\in D_{\mathrm{adv}}
\ \text{by}\ 
-\log f_T(u)_y\Big\},
\]
with default \(k = 100\). For a query \((x,y)\), proxies are all members of \(D_{\mathrm{pivot}}\) with the same class label \(y\) [2509.06796].

The resulting two-phase construction is what distinguishes IMIA from ordinary shadow training and from distillation-only MIA variants. The first phase mimics target behavior on auxiliary data, and the second injects an explicit “in” signal through cross-entropy on carefully chosen pivots. The paper argues that distillation-only methods such as Attack-D, SeqMIA, and GLiRA lack this dual behavior modeling and therefore perform poorly in the low-FPR regime relative to IMIA [2509.06796].

## 3. Membership score, decision rule, and operational variants

For a query \((x,y)\), IMIA collects the target’s observed scaled-confidence score
\[
s_{\mathrm{obs}} = \phi(f_T(x)).
\]
It then evaluates the query against the ensemble of imitative out models and the corresponding imitative in models. The out scores are
\[
S_{\mathrm{out}} = \{\phi(f_{\mathrm{out}}^j(x))\}_{j=1}^N,
\qquad
\bar{s}_{\mathrm{out}} = \frac{1}{N}\sum_{j=1}^N \phi(f_{\mathrm{out}}^j(x)).
\]
The in scores are aggregated across the imitative in models and the class-matched proxies in \(D_{\mathrm{pivot}}\), yielding a mean \(\bar{s}_{\mathrm{in}}\). Membership is then scored with the non-parametric squared-distance difference
\[
\Lambda(x)
=
\big(s_{\mathrm{obs}} - \bar{s}_{\mathrm{out}}\big)^2
-
\big(s_{\mathrm{obs}} - \bar{s}_{\mathrm{in}}\big)^2.
\]
The default decision rule is to predict member if \(\Lambda(x) > 0\), though in practice the paper also calibrates a threshold \(\tau\) to meet a target FPR and declares member iff \(\Lambda(x) > \tau\) [2509.06796].

In the non-adaptive setting, the attack prepares the imitative ensemble once. The adversary samples \(D_{\mathrm{imitate}} \subset D_{\mathrm{adv}}\), queries the target on \(D_{\mathrm{adv}}\), selects \(D_{\mathrm{pivot}}\), and for each \(j \in \{1,\dots,N\}\) trains an imitative out model for \(T_1\) epochs on \(\mathcal{L}_{\mathrm{imitate}}\), then continues training for \(T_2\) epochs on \(\mathcal{L}_{\mathrm{ce}}\) over \(D_{\mathrm{pivot}}\) to obtain an imitative in model. The paper’s default hyperparameters are \(N=10\), \(T_1=100\), \(T_2=20\), \(k=100\), and temperature \(1\) [2509.06796].

In the adaptive setting, the attack is query-dependent. For each query \((x,y)\), the adversary samples \(D_{\mathrm{tmp}} \subset D_{\mathrm{adv}}\), defines \(D_{\mathrm{out}} = D_{\mathrm{tmp}} \setminus \{(x,y)\}\) and \(D_{\mathrm{in}} = D_{\mathrm{tmp}} \cup \{(x,y)\}\), trains \(f_{\mathrm{out}}^j\) and \(f_{\mathrm{in}}^j\) with the same two-phase procedure, and uses the resulting in/out ensembles to score the query [2509.06796].

The default IMIA scorer is non-parametric and does not require Gaussian fitting of score distributions. The paper nonetheless describes an “IMIA-LiRA” variant in which LiRA-style Gaussian likelihood-ratio modeling is applied to imitative scores when more imitative models are available, yielding additional gains at increased computational cost [2509.06796].

## 4. Empirical behavior, low-FPR performance, and efficiency

The empirical evaluation covers image datasets MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 with ResNet-50, VGG-16, DenseNet-121, and MobileNet-V2 targets, as well as the tabular datasets Purchase-100 and Texas-100 with MLP targets. Training uses SGD with learning rate \(0.1\), momentum \(0.9\), weight decay \(5 \times 10^{-4}\), cosine learning-rate scheduling, and standard image augmentation [2509.06796]. The primary metrics are \(\mathrm{TPR}@0.001\%\mathrm{FPR}\), \(\mathrm{TPR}@0.1\%\mathrm{FPR}\), and balanced accuracy.

In the non-adaptive ResNet-50 setting, IMIA outperforms PMIA, LiRA, Canary, GLiRA, RMIA, and RAPID across the datasets reported in the paper. On MNIST, IMIA attains \(\mathrm{TPR}@0.001\%\mathrm{FPR} = 1.01\%\) versus \(0.30\%\) for PMIA, \(\mathrm{TPR}@0.1\%\mathrm{FPR} = 1.86\%\) versus \(0.77\%\), and balanced accuracy \(54.14\%\) versus \(53.23\%\). On Fashion-MNIST, the corresponding values are \(2.52\%\) versus \(0.25\%\), \(5.08\%\) versus \(3.51\%\), and \(61.22\%\) versus \(60.51\%\). On CIFAR-10, IMIA reports \(1.45\%\) versus \(0.62\%\), \(3.42\%\) versus \(1.84\%\), and \(61.08\%\) versus \(60.05\%\). On CIFAR-100, it reports \(2.10\%\) versus \(0.89\%\), \(7.32\%\) versus \(5.01\%\), and \(79.52\%\) versus \(77.64\%\) [2509.06796].

The adaptive ResNet-50 results preserve this ordering. On MNIST, IMIA yields \(\mathrm{TPR}@0.001\%\mathrm{FPR} = 1.33\%\) versus \(0.80\%\) for LiRA and \(\mathrm{TPR}@0.1\%\mathrm{FPR} = 2.35\%\) versus \(2.01\%\). On Fashion-MNIST, the values are \(4.62\%\) versus \(3.85\%\) and \(7.10\%\) versus \(6.03\%\). On CIFAR-10, they are \(2.33\%\) versus \(1.31\%\) and \(3.61\%\) versus \(2.63\%\). On CIFAR-100, they are \(8.52\%\) versus \(5.48\%\) and \(19.82\%\) versus \(18.32\%\) [2509.06796].

The non-image results are more pronounced. On Purchase-100 in the non-adaptive setting, IMIA reports \(\mathrm{TPR}@0.001\%\mathrm{FPR} = 0.51\%\) versus \(0.05\%\) for PMIA, \(\mathrm{TPR}@0.1\%\mathrm{FPR} = 9.54\%\) versus \(2.28\%\), and balanced accuracy \(82.45\%\) versus \(78.38\%\). On Texas-100, it reports \(0.82\%\) versus \(0.42\%\), \(10.61\%\) versus \(5.72\%\), and \(89.90\%\) versus \(87.10\%\) [2509.06796].

A notable feature of IMIA is reduced variance across random targets. The paper reports, for example, that on MNIST in the non-adaptive setting, IMIA achieves \(\mathrm{TPR}@0.001\%\mathrm{FPR} = 1.01 \pm 0.04\), compared with \(0.30 \pm 0.11\) for PMIA and \(0.16 \pm 0.10\) for LiRA. Residual analyses further indicate that normalized residuals under Gaussian fits follow \(N(0,1)\) more closely for imitative models than for shadow models, and Wasserstein distances between in/out distributions are substantially larger for imitative models, including a reported \(0.82\) versus \(0.21\) on CIFAR-100 [2509.06796].

The computational results are central to the method’s positioning. On a single A100 GPU in the non-adaptive setting, IMIA runs in \(0.64\) h on MNIST, \(0.65\) h on Fashion-MNIST, \(6.72\) h on CIFAR-10, \(13.43\) h on CIFAR-100, \(0.12\) h on Purchase, and \(0.13\) h on Texas, compared with \(143\)–\(287\) h for PMIA and LiRA. In the adaptive setting, the corresponding runtimes are \(0.72\), \(0.73\), \(6.96\), \(14.08\), \(0.14\), and \(0.16\) h, versus \(142\)–\(288\) h for LiRA. The paper summarizes these reductions as approximately \(95\%\) or greater across datasets [2509.06796].

## 5. Nomenclature, related constructions, and interpretation across domains

The acronym IMIA has been used for distinct attack families, and distinguishing them is important for correct interpretation. In "Privacy Leaks by Adversaries: Adversarial Iterations for Membership Inference Attack" [2506.02711], IMIA stands for **Iterations for Membership Inference Attack**, not imitative training. There, the attack infers membership from the minimal number of adversarial-attack iterations \(T(x)\) required to generate a successful adversarial example, with decision rule
\[
I_{\mathrm{IMIA}}(f_\theta,(x,y)) = 1\{T(x) \ge \tau\}.
\]
That method operates in score-based black-box, decision-based black-box, and white-box settings using SimBA, HSJA, and PGD respectively, and it does not train shadow models or imitative models. Its empirical signal is therefore tied to adversarial-generation dynamics rather than to target-informed imitation [2506.02711].

A different use of imitation appears in wireless signal classification. "Membership Inference Attack and Defense for Wireless Signal Classifiers with Deep Learning" [2107.12173] studies an over-the-air scenario in which the adversary’s received signal differs from the service provider’s view because of channel mismatch. The adversary therefore trains a surrogate classifier \(\hat C\) on its own received features and labels inferred from observed admissions so as to imitate the target on the adversary’s input domain. Membership inference is then performed by a learned binary model \(m(\cdot)\) over \((X,\hat Y)\) or \((X,s)\), with decision threshold \(0.5\). This is an imitative pipeline, but it is structurally distinct from the target-informed imitative-training ensemble of [2509.06796] [2107.12173].

The question of how IMIA transfers to large language models is addressed indirectly in "CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models" [2606.17464]. That benchmark does not explicitly define, implement, or evaluate IMIA. Instead, it constructs an i.i.d. evaluation protocol by partitioning shuffled pre-training data around an intermediate checkpoint, so that member and non-member samples are drawn from the same distribution \(\mathcal{P}\). The attack interface is the standard binary MIA setting: draw \(z \sim D_{\mathrm{mem}}\) or \(z \sim D_{\mathrm{nonmem}}\), compute a score \(p\), threshold at \(\tau\), and report ROC, AUC, and \(\mathrm{TPR}_{1\%}\) [2606.17464].

Within that framework, IMIA would have to be added as a new attack module. The paper explicitly states that IMIA is not “blind,” since it requires model outputs to train the imitator, and it notes that a black-box IMIA using next-token probabilities would be compatible with the benchmark. It also reports that on cleaned Pythia and OLMo splits, existing likelihood-based, calibration-based, and white-box attacks remain close to chance, with AUCs typically near \(0.5\) and very low \(\mathrm{TPR}_{1\%}\). This suggests that an LLM adaptation of IMIA would be evaluated under much stricter controls against distribution shift than many earlier MIA benchmarks provided [2606.17464].

## 6. Ablations, defenses, limitations, and open problems

The ablation studies in [2509.06796] isolate several design choices. Replacing the imitation loss with KL-based distillation or replacing imitative training with conventional shadow training significantly degrades low-FPR performance. On MNIST in the non-adaptive setting, \(\mathrm{TPR}@0.001\%\mathrm{FPR}\) is \(1.01\%\) for IMIA, compared with \(0.83\%\) for distillation and \(0.28\%\) for shadow training; on CIFAR-10, the corresponding values are \(1.45\%\), \(0.96\%\), and \(0.54\%\). Using soft targets at temperature \(1\) outperforms hard labels at temperature \(0\), while remaining robust for \(T \in [0.5,2]\). Per-class lowest-loss pivots outperform random pivots and remain robust for \(k = 50\) to \(1000\). Emphasizing the true class and top competitor in \(\mathcal{L}_{\mathrm{imitate}}\) is important, and square-root weighting is reported as the default best choice. Too small a pivot fine-tuning budget, such as \(T_2 = 10\), underfits; too many epochs can overfit the CE phase and erode target-informed behavior. Pre-softmax logits are preferred for \(\phi\), while pure loss is inferior as a membership signal [2509.06796].

The method has explicit assumptions. It relies on access to posterior probabilities or logits and on auxiliary data from the same distribution as the target’s training data. It does not directly target label-only interfaces. As with other MIAs, distribution shift between \(D_{\mathrm{adv}}\) and \(D_{\mathrm{train}}\) degrades performance, especially in very low-FPR regimes. The paper reports, for example, that when the target is trained on CIFAR-10 but the adversary uses CINIC-ImageNet, low-FPR performance drops notably across attacks, including IMIA [2509.06796].

Defenses reduce IMIA along with competing attacks. Under strong DP-SGD on CIFAR-10 in the non-adaptive setting, with \(\sigma = 1.0\) and \(\epsilon \approx 4\), the target accuracy drops to approximately \(32\%\) and IMIA’s \(\mathrm{TPR}@0.1\%\mathrm{FPR}\) falls to approximately \(0.12\%\). Adaptive comparisons at \(\sigma \in \{0.2, 0.5\}\) still show IMIA leading, but the absolute leakage is much smaller; for example, at \(\sigma = 0.2\), the reported \(\mathrm{TPR}@0.1\%\mathrm{FPR}\) is \(0.34\%\) for IMIA versus \(0.17\%\) for LiRA and \(0.19\%\) for Canary [2509.06796].

For large language models, the principal open problem is not only whether an imitative attack can be implemented, but whether it retains measurable power once evaluation artifacts are removed. CheckMIABench shows that blind baselines collapse to AUCs near \(0.5\) on cleaned splits and that published MIAs also struggle under i.i.d. pre-/post-checkpoint construction [2606.17464]. A plausible implication is that any LLM instantiation of IMIA would need unusually clear access assumptions, carefully controlled query budgets, and stronger evidence of statistical significance than earlier distribution-shifted benchmarks required.

In current usage, therefore, IMIA denotes a target-informed, low-variance alternative to large shadow ensembles: it models target behavior through weighted imitation, refines “in” behavior through pivot-based fine-tuning, and infers membership from a non-parametric comparison of scaled-confidence margins [2509.06796]. Its empirical profile is strongest in low-FPR auditing settings with posterior access and matched auxiliary data, while its main constraints are dependence on target outputs, sensitivity to distribution shift, and diminishing absolute power under strong privacy defenses [2509.06796].

Source: https://www.emergentmind.com/topics/imitative-membership-inference-attack-imia