VLALeaks: Membership Inference in VLA Models

• VLALeaks is a membership inference attack framework for Vision-Language-Action models that leverages subtle discrepancies in multi-modal attention patterns.
• It extracts comprehensive features from per-layer, cross-layer, and action attention blocks to determine if an image, text, and action triple was used during training.
• Empirical evaluations show that VLALeaks outperforms baselines on simulated and real-world benchmarks, highlighting significant privacy and intellectual-property risks.

VLALeaks is a membership inference attack (MIA) framework for Vision-Language-Action (VLA) models that uses attention discrepancies to determine whether a given image/text/action triple was used during training. In the VLA setting, end-to-end robot policies take a camera image or video sequence, a language instruction, and, during training, an expert action trajectory, then produce a sequence of motor commands. The framework introduced in "VLALeaks: Membership Inference Attacks against Vision-Language-Action Models" establishes that the memorization behavior of such models creates a concrete privacy and intellectual-property risk: with white-box access to the trained policy, an adversary can infer membership from attention alone, without querying logits or confidence scores (Luan et al., 13 Jun 2026).

1. VLA models and the privacy problem they create

Vision-Language-Action models are a newly emerging class of end-to-end policies that couple visual perception, natural-language conditioning, and low-level control. In the formulation considered by VLALeaks, a model receives three modalities: a camera image or video sequence, a language instruction such as “pick up the red block,” and, during training, the expert action trajectory represented as joint velocities or discretized action tokens. By fine-tuning large vision-language backbones such as ViT+LLM and discretizing continuous joint controls into classification bins, these systems connect high-level semantic goals to executable motor behavior (Luan et al., 13 Jun 2026).

The privacy concern arises from the economics and scarcity of robotic data collection. Demonstration data are often collected via teleoperation or motion capture, making them costly and potentially proprietary. VLALeaks treats membership leakage as the ability of an adversary to decide whether a particular image/text/action triple belonged to the training set. In this setting, membership disclosure is not a minor audit artifact: it can reveal the use of expensive demonstrations, expose sensitive operational data, or support intellectual-property theft. The paper characterizes this as the first systematic study of MIAs on VLA models and frames the issue as a prerequisite for secure and trustworthy VLA deployment (Luan et al., 13 Jun 2026).

A central implication is that VLA privacy is not reducible to output confidentiality alone. The attack surface includes internal transformer behavior, specifically multi-modal attention patterns spanning vision, language, and action tokens. This shifts the threat model from conventional output-based inference toward representation-level auditing.

2. Attention discrepancies as the membership signal

VLALeaks is built on the observation that member and non-member samples produce subtly different attention patterns inside the VLA transformer. For a model with $L$ transformer layers and sequence length $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$, the method extracts the averaged multi-head attention matrix at each layer,

$A^{(l)} \in \mathbb{R}^{S \times S}.$

Each $A^{(l)}$ is partitioned into nine modality-pair sub-blocks, including within-modality blocks such as $A_{\mathrm{img}}^{(l)}$ and cross-modality blocks such as $A_{\mathrm{img2txt}}^{(l)}$, $A_{\mathrm{txt2img}}^{(l)}$, and action-linked terms such as $A_{\mathrm{act2img}}^{(l)}$ and $A_{\mathrm{act}}^{(l)}$ (Luan et al., 13 Jun 2026).

From these sub-matrices, VLALeaks computes per-layer statistical features. For any sub-matrix $\mathcal{A}$, the framework uses the mean $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$0, standard deviation $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$1, average entropy $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$2, and concentration $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$3, where concentration averages the row-wise maximum attention weight. These statistics summarize both dispersion and sharpness of the attention distribution. The use of entropy and concentration is especially important because membership is hypothesized to manifest not only through stronger weights but through more peaked and stable attention allocation.

The method also models cross-layer evolution. For a chosen interaction such as image-to-text attention, it tracks concentration across depth and derives a mean focus trend $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$4 and variance $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$5. It further measures inter-layer change using a Frobenius distance,

$$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$6

and a symmetric KL divergence,

$$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$7

with $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$8 and $$S = S_{\mathrm{img}} + S_{\mathrm{txt}} + S_{\mathrm{act}}$$9. An additional interaction-tendency term is defined as

$A^{(l)} \in \mathbb{R}^{S \times S}.$0

Action-modality features form a separate component of the signal. VLALeaks computes bidirectional statistics between action tokens and vision/text tokens, including $A^{(l)} \in \mathbb{R}^{S \times S}.$1, $A^{(l)} \in \mathbb{R}^{S \times S}.$2, $A^{(l)} \in \mathbb{R}^{S \times S}.$3, $A^{(l)} \in \mathbb{R}^{S \times S}.$4, $A^{(l)} \in \mathbb{R}^{S \times S}.$5, $A^{(l)} \in \mathbb{R}^{S \times S}.$6, $A^{(l)} \in \mathbb{R}^{S \times S}.$7, and $A^{(l)} \in \mathbb{R}^{S \times S}.$8. This emphasizes that action tokens are not incidental output placeholders; they are treated as a distinct memorization-bearing modality (Luan et al., 13 Jun 2026).

The resulting representation is a high-dimensional feature vector $A^{(l)} \in \mathbb{R}^{S \times S}.$9 obtained by concatenating per-layer, cross-layer, and action features. The paper reports an empirical distribution gap

$A^{(l)}$0

in these features, visualized via KDE or PCA/t-SNE. This suggests that membership information is encoded in the geometry and dynamics of multi-modal attention rather than in any single scalar statistic.

3. Two-stage attack construction

VLALeaks proceeds in two stages: membership feature extraction and attack model construction. The first stage is deterministic feature engineering over internal attention maps; the second stage turns those features into a binary membership predictor (Luan et al., 13 Jun 2026).

Component	Operation	Representative quantities
Per-layer features	Statistics on modality-pair attention blocks	$A^{(l)}$1, $A^{(l)}$2, $A^{(l)}$3, $A^{(l)}$4
Cross-layer features	Depth-wise evolution of interactions	$A^{(l)}$5, $A^{(l)}$6, $A^{(l)}$7, $A^{(l)}$8, $A^{(l)}$9
Action features	Attention between action and image/text tokens	$A_{\mathrm{img}}^{(l)}$0, $A_{\mathrm{img}}^{(l)}$1, $A_{\mathrm{img}}^{(l)}$2, $A_{\mathrm{img}}^{(l)}$3

The attack model is trained on a labeled dataset

$A_{\mathrm{img}}^{(l)}$4

where $A_{\mathrm{img}}^{(l)}$5 indicates membership. The framework uses a lightweight binary classifier $A_{\mathrm{img}}^{(l)}$6 instantiated either as a Random Forest (RF) or a Multilayer Perceptron (MLP). For the MLP, the loss is binary cross-entropy:

$A_{\mathrm{img}}^{(l)}$7

At inference time, the attacker predicts “member” when $A_{\mathrm{img}}^{(l)}$8, typically with $A_{\mathrm{img}}^{(l)}$9 (Luan et al., 13 Jun 2026).

Two technical aspects distinguish the framework from more conventional MIA formulations. First, it does not require logits, confidence scores, or output loss values. Second, it is explicitly multi-modal: the attack does not reduce the VLA model to a generic transformer but exploits the fact that image, language, and action tokens define separable but interacting attention subspaces. A plausible implication is that VLA-specific structure is not merely an implementation detail; it is the main source of attack leverage.

4. Experimental results on simulated and real VLA systems

The evaluation covers two simulated VLA benchmarks—OpenVLA and OpenVLA-oft on four LIBERO splits—and one real-world bimanual VLA model, $A_{\mathrm{img2txt}}^{(l)}$0. The reported metrics are attack AUC and TPR@1\%FPR, with the results intended to show whether membership can be recovered under deployment-relevant conditions (Luan et al., 13 Jun 2026).

On OpenVLA across LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-100, the best baselines, identified as RIM, achieve AUC up to 0.98 but collapse on LIBERO-Goal, where AUC is approximately 0.87 and TPR@1\%FPR is approximately 0.27. VLALeaks-RF improves these results, reaching AUC $A_{\mathrm{img2txt}}^{(l)}$1 on Spatial/Object, 0.95 on Goal, and 0.98 on 100; the corresponding TPR@1\%FPR values are up to 0.89, 0.88, 0.44, and 0.76. VLALeaks-MLP yields AUC approximately 0.999/0.999/0.93/0.996 and TPR@1\%FPR approximately 0.98/0.98/0.34/0.93 across the same four splits (Luan et al., 13 Jun 2026).

Setting	Reported VLALeaks performance	Comparator
OpenVLA on LIBERO-Spatial/Object/Goal/100	RF: AUC $A_{\mathrm{img2txt}}^{(l)}$2, 0.95, 0.98; MLP: AUC $A_{\mathrm{img2txt}}^{(l)}$3	Best baselines (RIM) up to 0.98 AUC, but $A_{\mathrm{img2txt}}^{(l)}$4 on Goal
OpenVLA on LIBERO-Spatial/Object/Goal/100	RF TPR@1\%FPR up to 0.89, 0.88, 0.44, 0.76; MLP $A_{\mathrm{img2txt}}^{(l)}$5	RIM on Goal: TPR@1\%FPR $A_{\mathrm{img2txt}}^{(l)}$6
Real $A_{\mathrm{img2txt}}^{(l)}$7 tasks	AUC ranges 0.95–0.99 across six tray-pull/pick-and-place tasks	—
Mobile manipulation tasks aa–dd	RF/MLP AUC $A_{\mathrm{img2txt}}^{(l)}$8–0.90 and 0.81–0.94	Random-guess baselines at $A_{\mathrm{img2txt}}^{(l)}$9

The fine-tuned OpenVLA-oft setting is reported to show similar gains. On the real-world $A_{\mathrm{txt2img}}^{(l)}$0 tasks, AUC ranges from 0.95 to 0.99 for RF and MLP across six tray-pull/pick-and-place tasks. On four mobile manipulation tasks labeled aa–dd, VLALeaks-RF and VLALeaks-MLP yield AUC approximately 0.83–0.90 and 0.81–0.94, respectively, versus random-guess baselines at approximately 0.50 (Luan et al., 13 Jun 2026).

These results support two conclusions stated by the paper. First, current VLA models leak membership information through subtle multi-modal attention patterns. Second, the leakage persists across both simulated and real robotic settings, indicating that it is not an artifact of a single benchmark family.

5. Ablations, defenses, and privacy implications

Ablation studies reported for VLALeaks show that each feature group—per-layer, cross-layer, and action—contributes meaningfully, and that attack strength grows with the number of features. This is important because it argues against a narrow failure mode tied to one particular statistic. Instead, the attack appears to aggregate weak but consistent signals from multiple parts of the transformer’s internal attention structure (Luan et al., 13 Jun 2026).

The defensive findings are notably unfavorable to simple mitigation strategies. Partial duplicate removal barely affects performance, with $A_{\mathrm{txt2img}}^{(l)}$1. Naive DP-SGD drastically degrades utility without blocking VLALeaks. The paper therefore treats standard deduplication and straightforward differentially private training as insufficient in this setting. This suggests that privacy protection for VLA models cannot be retrofitted by minor preprocessing or by utility-damaging generic DP procedures.

The paper outlines several mitigation directions. One is differential-privacy training at the attention-projection level, described as DP-attention. Another is attention regularization or pruning to reduce over-concentration on training samples. A third is fine-tuning with adversarial membership-regularizers that minimize $A_{\mathrm{txt2img}}^{(l)}$2 between members and non-members in the feature space. A fourth is model-architecture modification, including cross-expert attention dropout, to blur modality correlations. These proposals remain mitigation directions rather than validated defenses within the reported study (Luan et al., 13 Jun 2026).

The broader privacy implication is that expensive robotic demonstrations may be recoverable at the membership level from model internals even when conventional output channels are not exposed. In operational terms, white-box access to attention alone is sufficient. For proprietary robotics datasets, this converts model release or internal access into a direct privacy and IP risk.

6. Nomenclature, related leakage studies, and common misconceptions

A source of confusion is that the label “VLALeaks” is overloaded across recent literature. In (Luan et al., 13 Jun 2026), it denotes white-box membership inference against Vision-Language-Action models via attention discrepancies. By contrast, "VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents" uses the term for action-boundary propagation, where a target string visible in an image appears verbatim in downstream tool-call arguments (Wang et al., 29 May 2026). "Beyond Attack Success Rate: Examining Trigger Leakage in Vision-Language Agentic Systems" uses it for trigger leakage in backdoored vision-language agentic systems, formalized through Neighbor Leakage Rate (NLR) (Chang et al., 10 Jun 2026). "Exposing and Defending Membership Leakage in Vulnerability Prediction Models" uses it as shorthand for vulnerability-prediction model membership leakage rather than robotics (Liao et al., 9 Dec 2025).

Related, differently named work helps situate the VLA threat model. "VidLeaks: Membership Inference Attacks Against Text-to-Video Models" studies black-box MIAs against text-to-video generation using Sparse Reconstruction Fidelity and Temporal Generative Stability (Wang et al., 16 Jan 2026). "The Phantom Menace: Unmasking Privacy Leakages in Vision-LLMs" examines identity leakage in VLMs, including the finding that blurring and context changes do not fully eliminate leakage (Caldarella et al., 2024). These works collectively indicate that multi-modal systems leak information through several distinct channels—tool arguments, trigger neighborhoods, generated outputs, or latent memorization—but they should not be conflated with the specific VLA membership-inference mechanism introduced in (Luan et al., 13 Jun 2026).

A common misconception is that leakage in robotic multi-modal models must arise through logits, confidence scores, or overt regurgitation. VLALeaks refutes that assumption for VLA policies: the attack surface can be the internal attention structure itself. Another misconception is that action tokens are secondary to vision-language alignment. The reported feature design and ablations indicate the opposite: action-modality interactions are part of the leakage signal and materially improve attack performance. In that sense, VLALeaks identifies a leakage channel that is specific to the control-oriented, multi-modal structure of VLA systems rather than a direct transplant of MIAs from image or language classification.