LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories

Abstract: Scientific laboratories increasingly rely on AI systems to reason about experiments, but the physical act of doing science remains largely outside their reach. AI can help read literature, generate hypotheses, and plan protocols, yet the execution of those protocols at the bench still requires a human operator. Vision-Language-Action (VLA) models provide one possible interface between written protocols and robot execution, but existing policies are trained mostly on household and tabletop demonstrations and rarely encounter the instruments, transparent liquids, or fixed protocol workflows found in scientific laboratories. Closing this gap requires both laboratory-specific supervision and a unified learning framework that can accommodate the diverse robot embodiments used to execute experimental protocols. We therefore identify data and embodiment as central bottlenecks alongside model design. To address the data side, we build RoboGenesis, a simulation-based workflow and data engine that composes configured laboratory workflows from atomic skills, validates and filters rollouts, and exports structured demonstrations across supported robot profiles. On the policy side, we present LabVLA, trained with a two-stage recipe: FAST action token pretraining first makes the Qwen3-VL-4B-Instruct backbone action aware before any continuous control is learned, and flow matching posttraining then attaches a DiT action expert under knowledge insulation. On the LabUtopia benchmark, LabVLA achieves the highest average success rate among all evaluated baselines under both in-distribution and out-of-distribution settings.

• The paper introduces LabVLA, which bridges the embodiment gap by synthesizing validated laboratory data through the RoboGenesis pipeline.
• The methodology employs a dual-stage training process using FAST pretraining and flow matching with knowledge insulation for stable policy specialization.
• Experimental results on LabUtopia show robust performance with ID/OOD success rates around 70%, outperforming previous VLA baselines in lab environments.

LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories

Problem Formulation and Motivation

LabVLA addresses the embodiment gap in AI-enabled scientific research: while AI systems can design, plan, and analyze experiments, the physical act of laboratory protocol execution remains challenging due to domain-specific requirements such as fine spatial precision, physical state tracking, and instrument diversity. Existing VLA models, despite progress in household and tabletop domains, lack adequate exposure to laboratory-specific skills and embodiments, leading to poor generalization in real scientific environments. The core insight is that laboratory automation bottlenecks arise from data and embodiment, not solely from model design.

RoboGenesis: Laboratory Data Synthesis at Scale

To address the lack of laboratory-specific data, LabVLA introduces RoboGenesis, a simulation-centric workflow engine targeting three critical requirements: (1) executable scenes with physical and semantic validity, (2) long-horizon, composable protocol workflows, and (3) success-filtered, richly annotated demonstrations supporting multi-embodiment policies. RoboGenesis leverages a generative asset pipeline—using text-conditioned image generation, 3D reconstruction via TRELLIS 2.0, and physics-aware asset processing—to construct a large library of annotated experimental objects and environments.

RoboGenesis decomposes protocol descriptions into ordered atomic skill sequences instantiated across a pool of single-arm, bimanual, and mobile manipulator robot profiles. Extensive domain randomization over scene layout, object appearance, lighting, spatial perturbations, and camera perspectives is layered atop validated scenes to enforce visual and spatial invariance across episodes.

Figure 1: RoboGenesis data synthesis pipeline encompasses automated asset and scene generation, agentic workflow specification, and structured rollout export with comprehensive annotations.

Exported LabEmbodied-Data contains rich per-frame and per-step annotations, including robot state, camera geometry, object physical state, scene relationships, task alignment, success metrics, and contact/contact safety monitoring. The resulting dataset (10,000+ scenes, thousands of workflows) couples protocol-level diversity with cross-embodiment generality, serving as a robust substrate for downstream policy learning.

Figure 2: Diverse laboratory scenes generated by RoboGenesis, showcasing variation in geometry, material, lighting, and background to minimize overfitting and improve generalization.

Training Pipeline: FAST Pretraining and Flow Matching with Knowledge Insulation

LabVLA adopts a two-stage training paradigm:

1. FAST Token Pretraining: The Qwen3-VL-4B-Instruct backbone is jointly pretrained on large grounded corpora (Robointer-VQA, AgiBot World Beta, OXE-AugE, Droid) using next-token supervision for both language and action (FAST-token) streams. This aligns the vision-language prefix with action semantics, producing a representation sensitive to both perceptual context and low-level manipulation cues before exposure to continuous actions.
2. Flow Matching Posttraining with Knowledge Insulation: The DiT action expert is attached and trained to predict continuous action chunks via a flow matching objective, with a stop-gradient applied to the VLM prefix (knowledge insulation) to shield perceptual and reasoning representations from direct action-space gradients. This decouples VLM pretraining objectives from fine-grained action regression, stabilizing language following and visual grounding during policy specialization.

   Figure 3: The LabVLA training pipeline architecture, illustrating pretraining on grounded data via FAST tokenization and posttraining with flow matching under knowledge insulation.

Embodiment-agnostic batching (fixed tensor shape, masked loss reduction) and large-scale compute optimizations (gradient checkpointing, fused kernels, background streaming) enable efficient joint training on multi-source, high-dimensional data.

Quantitative and Qualitative Results

LabVLA is evaluated on the LabUtopia benchmark, which systematically varies laboratory tasks across six protocol archetypes (e.g., Pick Up, Open Door, Pour Liquid, Press Button, Heat Beaker, Transport Beaker) under both in-distribution (ID) and out-of-distribution (OOD) scene and object perturbations.

LabVLA achieves an average ID/OOD success rate of 71.1%/70.0%, outperforming all prior VLA baselines. Notably, it maintains a narrow performance gap between ID and OOD (1.1 percentage points), indicating strong generalization induced by domain randomization and data diversity. Task-wise, LabVLA leads or is tied on Pick Up, Open Door, and Press Button; Pour Liquid remains the primary failure mode across all methods due to demanding contact and state tracking.

Figure 4: Rollout snapshots for six representative LabUtopia tasks, demonstrating successful multi-stage laboratory manipulation across protocol categories.

Fine-tuning other VLA architectures (e.g., X-VLA) with LabEmbodied-Data yields significant gains (+15 to +19 percentage points), demonstrating the supervision transferability of RoboGenesis data. In real-world experiments on a Franka platform, LabVLA (simulation-trained) achieves competitive success rates (70–87%) on multi-step protocols involving shaking, pouring, stirring, and stopper manipulation; performance degrades gracefully under clutter and out-of-domain conditions.

Analysis and Discussion

Implications

• Supervision Transfer: Synthetic, success-filtered laboratory demonstrations generated via RoboGenesis substantially boost cross-embodiment and OOD generalization, with observed gains holding across architectures, confirming the central role of protocol-aware data.
• Model Design: Knowledge insulation prevents representation collapse in the VLM during flow-based action regression, supporting modular and stable policy specialization without sacrificing perceptual grounding or instruction adherence.
• Practical Deployment: While simulation-to-reality transfer is viable for basic laboratory operations, challenges such as hardware noise, reagent state, and protocol deviations remain.

Theoretical and Practical Limitations

• Open Challenges: Pour Liquid tasks highlight the limits of current closed-loop perception and fine-grained state estimation without direct fluid-level sensing.
• Scope of Autonomy: Current policy is fixed-protocol; levels requiring measurement awareness, real-time adaptation, or protocol modification (e.g., spontaneous error recovery or scientific inference) are outside demonstration.
• Sim2Real Gap: Real laboratory environments entail unmodeled phenomena (contamination, regulatory constraints), human-robot interaction, and scientific reasoning that are not covered.

Future Directions

Future work should extend the RoboGenesis schema to encompass more intricate physics, explicit measurement and observation handling, user-in-the-loop correction, and open-ended protocol modification. The progression toward instrument-level autonomy (Level 3: Specialist, Level 4: Scientist) will require integration of real laboratory feedback, richer agentic planning, and robust scientific context awareness.

Conclusion

LabVLA establishes a formal methodology for grounding VLA models in scientific laboratory settings by attacking data and embodiment bottlenecks with compositional simulation (RoboGenesis) and cross-embodiment action observation schemas. The dual-stage policy pipeline—FAST pretraining and knowledge-insulated flow-matching—demonstrates broad and stable protocol execution under the LabUtopia framework, setting a new standard for lab-oriented VLA research. The released assets (engine, data, training recipe) serve as a foundation for future embodied scientific AI, with deployment and scientific competence remaining as subsequent frontiers (2606.13578).