Biologically-Informed Hybrid MIA (biHMIA)

• The paper introduces a hybrid membership inference attack (biHMIA) that integrates genomic metrics with black-box methods for improved detection of training data membership.
• It leverages biological features such as allele frequency deviation, mutation rate, and genotype frequencies to exploit data artifacts even under differential privacy.
• Experimental results show biHMIA boosts AUC by 6–8 points over traditional approaches, highlighting the need for domain-aware defenses in privacy-preserving genomic models.

A biologically-informed hybrid membership inference attack (biHMIA) is an adversarial methodology designed to determine whether a target individual’s genotype profile was part of the training data for generative genomic models. The approach augments traditional black-box membership inference attacks (MIAs) by integrating genomic domain-specific metrics such as allele frequency deviation, mutation rate, and genotype frequency, thereby increasing the adversary’s ability to distinguish between member and non-member data points. This hybrid technique is particularly pertinent to assessing privacy vulnerabilities in transformer-based LLMs trained on genomic datasets, such as those derived from the 1000 Genomes Project, even when these models are trained with differential privacy.

1. Membership Inference in Generative Genomic Models

Membership inference attacks in genomics present unique challenges due to both the sensitivity of genetic data and the statistical structure of the data itself. The adversarial setting involves a challenger choosing a genotype profile $x$ either from the model’s training set ($\mathcal{D}_{\mathrm{train}}$) or a disjoint holdout set ($\mathcal{D}_{\mathrm{hold}}$). The adversary, with black-box access to a generator $f$—such as a GPT-like model producing synthetic mutation profiles—aims to infer the membership status of $x$.

The privacy threat is measured using the advantage $\mathrm{Adv}_{\mathrm{MIA}}$: $$\mathrm{Adv}_{\mathrm{MIA}}(\mathcal{A}) = \big|\Pr[\hat b=1 \mid b=1] - \Pr[\hat b=1 \mid b=0]\big|$$ This quantifies the adversary's ability to distinguish members from non-members, where $b$ indicates true membership and $\hat b$ is the adversarial guess.

When generators are trained with differential privacy—specifically, $(\varepsilon,\delta)$-DP mechanisms such as DP-SGD—membership inference is theoretically mitigated. However, empirical findings indicate that genomics-specific information embedded in model outputs can still be exploited.

2. Traditional Black-Box Membership Inference Attacks

Classical MIAs on generative models rely solely on model-based signals. The most prevalent approaches include:

• Confidence Score and Perplexity: The generator's loss or perplexity on a specific genotype sequence indicates potential membership, with lower values suggestive of training data inclusion. Perplexity is defined as:

$$\mathrm{PPL}(x) = \exp\left(-\frac{1}{|x|}\sum_{i}\log p(x_{i} \mid x_{<i})\right)$$

A threshold-based attack signals membership when the score falls below a chosen threshold $\tau$.

• Likelihood Ratio: More sophisticated MIAs leverage shadow models to estimate the likelihood ratio between training and holdout sets:

$$\Lambda(x) = \frac{p_{\mathrm{train}}(s_{\mathrm{bb}}(x))}{p_{\mathrm{hold}}(s_{\mathrm{bb}}(x))}$$

Membership is declared when $\Lambda(x) \ge \eta$ for some threshold $\eta$.

While effective in general settings, these black-box attacks may overlook idiosyncratic signals latent in biological data.

3. Biologically-Informed Metrics for Genomic Data

biHMIA introduces genomic features derived from the biological structure of synthetic variant profiles generated by $f$ upon being prompted with the target profile $x$. Key biologically-informed metrics include:

• Mutation Rate (MR):

$\mathrm{MR}(S) = \frac{|S|}{L}$

where $S$ is the synthetic variant set and $L$ the number of loci.

• Genotype Frequencies ($f_g$):

$f_g(S) = \frac{|\{v \in S : g_v = g\}|}{|S|}$

for each genotype $g$.

• Variation Type Frequencies ($f_t$):

$$f_t(S) = \frac{|\{v \in S : \mathrm{type}(v)=t\}|}{|S|}$$

with types $t$ including deletion, insertion, and substitution.

• Allele Frequency Deviation (AFD):

$$\mathrm{AFD}(S) = \sum_{a} |\hat{p}_{S}(a) - p_{\mathrm{pop}}(a)|$$

where $\hat{p}_{S}(a)$ is the empirical allele frequency in $S$, and $p_{\mathrm{pop}}(a)$ comes from population reference data.

Optionally, additional metrics such as pairwise linkage disequilibrium and external annotation-derived impact scores enhance attack power.

4. biHMIA Algorithmic Pipeline

biHMIA combines model-based and biologically-informed metrics in a weighted ensemble to arrive at a membership decision. The standard pipeline consists of:

Algorithm Steps:

1. Model-Based Score: Compute loss or perplexity of $f$ on $x$.
2. Synthetic Generation: Prompt $f$ with a prefix of $x$ to produce synthetic variant sequence $S$.
3. Biological Feature Extraction: Calculate the feature vector $$\mathbf{g} = [\mathrm{MR}(S),\,\{f_g(S)\},\,\{f_t(S)\},\,\mathrm{AFD}(S)]$$.
4. Domain-Specific Score: Form a weighted sum $s_{\mathrm{bio}} = w^\top \mathbf{g}$ or use a classifier trained on biological features.
5. Score Combination: Compute the hybrid score:

$$s_{\mathrm{hybrid}} = \alpha s_{\mathrm{bb}} + (1-\alpha)s_{\mathrm{bio}}$$

where $\alpha \in [0,1]$ tunes the reliance on each modality.

1. Membership Decision: Apply a threshold or a classifier $\mathcal{C}$ (e.g., logistic regression, random forest, k-nearest neighbors) to $(s_{\mathrm{bb}}, \mathbf{g})$.

Parameter Selection: The ensemble weight $\alpha$ is tuned to maximize area under the receiver operating characteristic (AUC) on a validation set. Threshold $\tau$ is chosen to balance false-positive and false-negative rates or maximize attack advantage.

5. Experimental Evaluation

The original evaluation employs genotypes of 2504 individuals from the 1000 Genomes Project, focusing on chromosome 22 VCF data. Two categories of transformer-based LMs serve as generative models: GPT-2 small (124M parameters, fine-tuned with and without DP) and MinGPT (~12M parameters, trained from scratch with a custom BPE tokenizer, with and without DP).

Differential Privacy Training:

• Gaussian DP-SGD with a clipping norm $C$ and noise multiplier to target $\varepsilon=1$, $\delta=10^{-5}$.

Attack Metrics:

• True Positive Rate (TPR)
• False Positive Rate (FPR)
• Area Under ROC (AUC)
• Attack advantage ($\mathrm{TPR} - \mathrm{FPR}$)

Results summary:

Model	Baseline MIA AUC	biHMIA AUC	Δ
MinGPT (no DP)	0.60	0.68	+0.08
MinGPT-DP ($\varepsilon=1$)	0.54	0.61	+0.07
GPT-2 (no DP)	0.72	0.78	+0.06
GPT-2-DP ($\varepsilon=1$)	0.59	0.66	+0.07

Across all model configurations, biHMIA increases AUC by 6–8 points relative to the black-box baseline.

6. Implications for Privacy and Genomic Model Design

The enhanced attack power of biHMIA demonstrates that incorporating biological metrics provides adversaries with a substantial advantage in inferring membership, even when differential privacy mechanisms are employed. Notably:

• DP Mitigation: While DP-SGD reduces overall vulnerability (e.g., MinGPT AUC decreases from 0.60 to 0.54 under DP for the baseline attack), biHMIA maintains non-trivial attack advantage, illustrating that DP alone is insufficient if biological artifacts persist.
• Model Size Trade-off: Smaller models (e.g., MinGPT) exhibit enhanced innate privacy at some expense to data utility.
• Design Recommendations: Genomic data generators should be evaluated against both classical and biologically-informed MIAs. Achieving strict DP guarantees ($\varepsilon \leq 1$) substantially degrades both attack types. Additional post-generation filters specifically targeting allele frequency and mutation pattern signals may offer extra protection.

A plausible implication is that genomic privacy research must prioritize comprehensive privacy auditing using hybrid, domain-aware metrics rather than relying solely on model-agnostic attacks or theoretical DP guarantees.

7. Future Directions and Considerations

The integration of biologically-informed feature extraction into membership inference attacks signals the necessity for evolving defense techniques that neutralize not only statistical signals but also genomics-contextual cues in synthetic data. Future research on privacy-preserving genomic models should investigate methods to mask biological patterns, such as allele frequencies and genotype combinations, potentially via post-processing filters or advanced privacy-preservation schemes. Evaluations must account for hybrid attacks that exploit domain knowledge, not merely classical black-box MIAs.

A plausible implication is that as generative models continue to be deployed for sensitive genomic data synthesis, the landscape of privacy auditing will need to incorporate domain-specific adversarial perspectives to ensure robust privacy assurances.