VLALeaks: Membership Inference Attacks against Vision-Language-Action Models

Abstract: Vision-Language-Action (VLA) models enable end-to-end robot control and have garnered widespread attention. However, the memorization of training data inherent to VLA, coupled with the high cost of robotic data acquisition, raises serious concerns regarding data privacy leakage and intellectual property infringement. Membership inference attacks (MIAs) aim to determine whether a given sample belongs to the training set. While representing a significant privacy threat, this attack remains underexplored in the context of VLA models. To bridge this gap, we propose VLALeaks, which is based on attention discrepancies in VLA models. We reveal, for the first time, the privacy vulnerabilities of VLA models. Specifically, it comprises a two-stage process: (1) membership feature extraction, and (2) attack model construction. Experimental results across multiple VLA benchmarks demonstrate that VLALeaks readily reveals membership information and achieves optimal attack AUC and TPR@1\%FPR, highlighting the privacy vulnerabilities in current VLA model deployments. Our work is the first systematic study of MIAs on VLA models, aiming to provide insights for secure and trustworthy VLA models.

• The paper introduces VLALeaks, a two-stage white-box attack that leverages multi-layer attention features to effectively infer membership in VLA models.
• The methodology employs statistical extraction of per-layer and cross-modal features, achieving AUC scores up to 0.999 in robotic simulation and real-world tasks.
• The results highlight critical privacy vulnerabilities in VLA architectures, suggesting that traditional defenses are insufficient against targeted MIAs.

Membership Inference Attacks on Vision-Language-Action Models: An Analysis of VLALeaks

Introduction

The increasing integration of Vision-Language-Action (VLA) models within robotics has enabled highly capable embodied agents to process diverse multimodal observations and instructions, generating adaptive and context-sensitive control signals. However, the unique data acquisition costs and privacy implications of VLA training samples—especially compared to more readily available modalities such as textual web data—motivate a thorough scrutiny of privacy risks, notably membership inference attacks (MIAs). Despite substantial inquiry into MIAs on unimodal and even bi-modal models, research addressing membership privacy leakage in VLA frameworks remains notably sparse. "VLALeaks: Membership Inference Attacks against Vision-Language-Action Models" (2606.15165) initiates a systematic evaluation of VLA model vulnerabilities, introducing a tailored MIA methodology that leverages intrinsic attention matrix signals for high-fidelity member inference.

Problem Setting and Challenges in VLA MIAs

VLA models map image observations, language instructions, and corresponding actions into complex, entangled latent manifolds, frequently modulated through Transformer-style architectures with explicit attention computation. Privacy leakage in this context is nontrivial due to three primary characteristics: (1) multimodal entanglement breaking naive alignment-based analysis, (2) semantically opaque, discretized action representations inhibiting traditional output-based attacks, and (3) minimal inter-sample variation in collected robotics trajectories. These intricacies collectively render classical logit-based, loss-based, or confidence-based MIAs ineffective, highlighting the necessity for cross-modal, internal-representation-centric adversarial techniques.

The work delineates the specificity of VLA membership leakage through a taxonomy of sample configurations, for instance, observations with subtle visual differences sharing identical instructions, or entirely identical visual/textual input mapped to diverse action trajectories (Figure 1). The lack of one-to-one mapping between modalities fundamentally undermines unimodal or naive multimodal MIA feature extraction.

Figure 1: Subtle differences in images, but identical text; identical images, varying text.

VLALeaks: Methodology

Two-Stage Pipeline Structure

VLALeaks constitutes a two-stage white-box attack, assuming full access to the model's architectural parameters and internal attention activations:

1. Membership Feature Extraction: For each sample, multi-layer, multi-head self-attention matrices are extracted and partitioned by modality. Per-layer and cross-layer statistics are computed, including mean, variance, entropy, concentration, and dynamic evolution metrics, constructing a rich vector of statistical and distributional features. Moreover, bidirectional cross-modal (e.g., action-to-image, image-to-action) interaction statistics are included to capture subtle privacy gradients missed in prior work.
2. Attack Model Construction: With supervised knowledge of membership, a lightweight binary classifier (RF or MLP) is trained to distinguish member from non-member points via the fused attention features. The effectiveness of this approach is predicated on the hypothesis that VLA models differentially overfit or generalize attentional energy across modalities for samples present during training. The overall pipeline is visualized in Figure 2.

   Figure 2: Overview of the VLALeaks pipeline with membership feature extraction from self-attention and binary classifier-based inference.

Statistical evidence for the efficacy of the extracted features is provided via kernel density estimation, displaying that member and non-member samples induce distinguishable attention statistics (Figure 3).

Figure 3: VLALeaks-RF (AUC=0.99) and VLALeaks-MLP (AUC=0.98), demonstrating clear separability between member and non-member sample clusters.

Experimental Evaluation

Simulated and Real-World Platforms

VLALeaks is evaluated on both simulated robotic platforms (LIBERO/ OpenVLA) and complex real-world manipulation settings (bimanual AIRBOT Play arms), leveraging diverse task protocols (e.g., object placement, articulated body interactions). Novel datasets such as LIBERO-100 (long-horizon, knowledge-entangled tasks) and mobile manipulation scenarios are included to gauge robustness.

The platforms are visually depicted, highlighting their multimodal observation and action spaces (Figures 3, 4, 6, 10, 11).

Figure 4: LIBERO-100 task suite for simulated evaluation.

Figure 5: Real-world bimanual manipulation setup with dual-arm configuration.

Figure 6: LIBERO-Spatial tasks focusing on spatial reasoning in simulation.

Figure 7: Real-world mobile manipulation platform integrating arm and mobile base.

Figure 8: Four diverse tasks for whole-body robot evaluation.

Quantitative Benchmarks

VLALeaks outperforms all SOTA MIAs from the vision, language, and VLM literature, producing AUCs routinely >0.99 on several tasks. For instance, on OpenVLA (LIBERO-Spatial, LIBERO-Object, and LIBERO-100), VLALeaks-MLP achieves $AUC \approx 0.999$ and TPR@1%FPR approaching unity. Baseline attacks remain close to chance performance—reinforcing the nontriviality of privacy extraction absent tailored feature design.

Performance is resilient across both “in-the-lab” simulated environments and challenging real-world physical robot deployments, as evidenced by robust results (AUC > 0.81) across all mobile manipulation tasks. Figure 9 and Figure 10 further illustrate comparative attack AUCs.

Figure 9: Attack AUC for multiple manipulation tasks, showing the clear superiority of VLALeaks variants.

Figure 10: Additional AUC visualizations comparing VLALeaks to baseline MIAs.

An ablation study systematically disables each of per-layer, cross-layer, and action-specific features, revealing non-redundancy and task-dependent importance: while per-layer attention features are indispensable for "pull tray" tasks, action-centric signals dominate object manipulation membership leakage (see Figure 11).

Figure 11: Visual depiction of the "pull the tray right" task used in ablation analysis.

Discussion and Theoretical Implications

VLALeaks exposes the inherent privacy vulnerabilities in attention-based multimodal models, specifically VLA architectures deployed in data-constrained robotic settings. The attack’s success stems from modality-agnostic, high-dimensional, and layer-integrated feature representations, unearthing privacy signals at a granularity unattainable using logit- or confidence-based paradigms prevalent in MIAs for unimodal models.

Critically, standard defenses, including duplicate data deduplication and differential privacy mechanisms, do not fully mitigate leakage: VLALeaks continues to extract member information under these conditions, signifying that the root causes are embedded in the model's learning and representational biases rather than surface-level memorization effects.

Practical Implications and Directions for Future Work

The immediate implication is the inadequacy of current privacy protections for VLA models, especially as such models become foundational components in industrial, service, and assistive robotics. Effective auditing and possible regulatory oversight for robotic data curation and model deployment are now pertinent.

Looking forward, several research axes are suggested:

• Robust defense strategies: Developing regularization protocols that specifically desensitize inter- and intra-modal attention to member-specific idiosyncrasies, beyond classical DP.
• Membership auditing tools: Integration of feature-extraction-based auditing methodologies for ongoing privacy monitoring throughout the VLA model lifecycle.
• Generalization to black-box settings: Adapting the feature extraction process, or leveraging model inversion, to facilitate MIAs under restricted-access or proprietary VLA deployment.
• Understanding leakage in transfer and continual learning settings, given VLA models are continually fine-tuned in robotics applications.
• Interplay with adversarial robustness and interpretability: Investigating whether privacy-leaking attention patterns co-occur with adversarial vulnerabilities or serve as explanations for policy overfitting.

Conclusion

VLALeaks establishes a concrete and formal foundation for quantifying membership privacy leakage in VLA models, breaking new ground in understanding model-specific vulnerabilities rooted in attention-based multimodal integration. The systematic extraction and utilization of attention-driven, modality-aware features enable high-fidelity member inference, resilient to common defensive mitigations. These findings necessitate a reevaluation of current privacy assumptions in robotics AI and motivate the development of VLA-specific defense, monitoring, and regulatory frameworks.