LabVLA: Vision-Language-Action for Labs

Updated 5 July 2026

LabVLA is a vision-language-action system designed for executing complex laboratory protocols by integrating simulated environments and embodied datasets.
It uses RoboGenesis to generate realistic 3D lab simulations and diverse realistic laboratory scenes, enabling robust protocol execution despite data scarcity.
The two-stage training approach, combining FAST pretraining and flow matching, adapts a Qwen3-VL backbone into an embodiment-agnostic lab control policy.

LabVLA is a vision-language-action system for scientific laboratories that addresses a specific gap in AI for science: contemporary models can already assist with literature reading, hypothesis generation, code drafting, and protocol planning, yet the physical execution of laboratory procedures still largely depends on human operators. It combines a simulation-based workflow and data engine called RoboGenesis, a laboratory-specific embodied dataset called LabEmbodied-Data, and a two-stage training recipe that turns a Qwen3-VL-4B-Instruct backbone into a continuous laboratory control policy for scientific protocols (Ren et al., 11 Jun 2026).

1. Problem setting and scope

LabVLA is framed around the claim that laboratory robotics is bottlenecked jointly by data, embodiment, and model design, rather than by policy architecture alone (Ren et al., 11 Jun 2026). The target environment differs materially from the household and tabletop settings that dominate datasets such as Open X-Embodiment, DROID, and BridgeData V2. Laboratory procedures involve domain-specific objects and instruments, including pipettes, centrifuges, heaters, thermal cyclers, reagent containers, and transparent liquids, as well as fixed multistep workflows in which one failed step propagates into later failure.

The system is therefore designed for a laboratory regime characterized by severe data scarcity, high embodiment diversity, transparent-liquid perception difficulty, and protocol-centric task structure. In this formulation, embodiment is not an incidental deployment detail but part of the central research problem: different laboratories may use single-arm systems, dual-arm systems, and mobile manipulators, and a useful policy must absorb supervision from all of them without collapsing into embodiment-specific heuristics (Ren et al., 11 Jun 2026).

LabVLA consequently treats scientific protocol following as a VLA problem in which written instructions, visual observations, robot state, and embodiment-specific action spaces must be aligned within a single training and deployment framework. The paper emphasizes level-2 “technician” capabilities, such as following fixed multistep procedures involving pouring, heating, stirring, shaking, and transport, rather than adaptive scientific reasoning or experimental design (Ren et al., 11 Jun 2026).

2. RoboGenesis and LabEmbodied-Data

RoboGenesis is the data engine underlying LabVLA. It is built on Isaac Sim and generates laboratory demonstrations from executable workflows rather than from isolated task scripts (Ren et al., 11 Jun 2026). The pipeline has three stages: environment building, agentic workflow generation, and structured export.

In the environment-building stage, RoboGenesis converts a plain-text object description into a 3D USD asset through a four-step process: generating a product-style reference image from text, reconstructing a textured 3D model with TRELLIS 2.0, postprocessing the mesh into a physics-ready USD/URDF asset with mass, friction, collision geometry, and metadata, and batching this process across categories. The resulting LabAssetLibrary contains 2,947 annotated assets. For liquid-bearing containers, RoboGenesis stores liquid color and fill fraction, renders colored liquid proxies, and transfers liquid meshes when pour actions succeed. It then assembles executable laboratory scenes with a seeded greedy placement solver, validates overlap and accessibility constraints, assigns textures from a LabTextureLibrary, and reports generating 10,000 laboratory scenes (Ren et al., 11 Jun 2026).

The workflow-generation stage represents a protocol as an ordered composition of atomic skills. The atomic skill library includes object-centric skills such as pick, place, pour, stir, shake, and move; instrument-level skills such as press, pressZ, open, and close; and navigation skills for mobile embodiments. Workflows can be authored manually as YAML templates or generated agentically from natural-language instructions. The agent-assisted authoring process queries available robots, assets, and task templates, proposes scene layouts and workflow YAML, and validates candidates for reachability, place-target conflicts, and repick-yaw risks. The paper notes that even composite workflows with more than 20 skill steps still achieve collection success rates above 75% (Ren et al., 11 Jun 2026).

The final stage exports successful rollouts into LabEmbodied-Data. Each atomic skill has a dedicated success checker, and a parallel contact safety monitor rejects unsafe collisions. Episodes store multicamera RGB images, robot joint states, executed actions, and language instructions, together with 15 categories of structured annotations, including robot state, camera intrinsics and extrinsics, step timing, per-frame subinstructions, object states, scene relations, object semantics, step-level success explanations, collision events, temporal segments, subgoals, quality scores, intervention flags, and episode metadata (Ren et al., 11 Jun 2026).

Stage	Function	Concrete details
Environment building	Create assets and scenes	2,947 assets; 10,000 scenes
Workflow generation	Compose protocols from skills	YAML or agentic generation
Structured export	Save validated demonstrations	15 annotation categories

A notable property of RoboGenesis is semantics-preserving randomization. The paper specifies six randomization axes: scene layout, visual clutter, camera perturbations, object swaps that preserve semantics, lighting changes, and spatial perturbations of object poses. Instructions are also paraphrased. This preserves task identity while inducing distribution shift, which is central to the benchmark’s in-distribution and out-of-distribution split (Ren et al., 11 Jun 2026).

3. Policy architecture and embodiment abstraction

LabVLA pairs a Qwen3-VL-4B-Instruct backbone with a diffusion-transformer-style action module trained by flow matching (Ren et al., 11 Jun 2026). The backbone encodes visual observations and language into hidden states, while a separate DiT action expert maps those hidden states together with robot state into continuous action chunks. The paper explicitly notes that the model is not a single unified self-attention stack over vision, language, state, and action tokens; instead, it deliberately decouples the pretrained VLM from a specialized action expert.

At time step $t$ , for embodiment $r$ , the policy observes up to $V$ RGB views $I_t^{1:V}$ , a language instruction $\ell$ , and robot state $q_t^r$ . It predicts a $K$ -step continuous action chunk $A_t^r = [a_t^r,\ldots,a_{t+K-1}^r] \in \mathbb{R}^{K \times d_r}$ . The Qwen3-VL backbone $f_\phi$ produces hidden states $H_\phi$ , which are linearly projected into the width expected by the action expert. The action expert $r$ 0 is an 18-layer DiT with width 1024, 8 attention heads, and head dimension 128. The current state and a noisy action chunk are each projected into tokens, concatenated as the DiT query sequence, and processed with cross-attention to the projected VLM states (Ren et al., 11 Jun 2026).

A central architectural feature is embodiment-agnostic batching. The paper defines $r$ 1 as the active action dimension of robot or dataset $r$ 2, pads all action and state vectors to a shared maximum $r$ 3, and uses a binary action-valid mask $r$ 4 to distinguish real from padded entries. Missing cameras are represented by dummy images and attention masks. Dataset schemas specify camera mappings, state keys, action keys, action dimensions, gripper dimensions, and delta-action masks. This allows single-arm, bimanual, and mobile-manipulator data to be mixed in one training pipeline (Ren et al., 11 Jun 2026).

The embodiment scope is unusually broad for a laboratory VLA. RoboGenesis supports 16 robot platforms, including single-arm systems, dual-arm systems, and mobile manipulators such as Franka Panda, FR3, UR-series arms, Piper, Rizon4, Festo, ARX X5, ARX R5, Split ALOHA, Lift2, FR3 Duo, and Ridgebase-mounted variants (Ren et al., 11 Jun 2026). The paper does not introduce a learned explicit embodiment token; conditioning instead arises from robot state, dataset schema, active action masks, and robot-specific execution interfaces.

4. Two-stage training recipe: FAST, flow matching, and knowledge insulation

LabVLA is trained in two stages. The first stage is FAST action token pretraining, intended to make the Qwen3-VL backbone “action aware” before any continuous control objective is introduced (Ren et al., 11 Jun 2026). Continuous action chunks are normalized using per-dataset statistics, encoded into discrete FAST tokens, padded only over active dimensions, and appended to the training sequence. Robot states are normalized, clipped to $r$ 5, discretized into 256 bins, serialized into text, and prepended to the instruction. The backbone is trained on grounded data from Robointer-VQA, AgiBot World Beta, OXE-AugE, and Droid; OXE-AugE contributes only its LeRobot-format subset, around 572k trajectories (Ren et al., 11 Jun 2026).

The second stage is flow-matching posttraining. After the VLM has been made action-aware, LabVLA attaches the DiT action expert and trains continuous control on OXE-AugE together with LabEmbodied-Data (Ren et al., 11 Jun 2026). The target action chunk is padded to the shared embodiment width, Gaussian noise is sampled, and a linear interpolation path is defined between noise and data. The DiT predicts a continuous velocity field, and training minimizes a masked mean-squared error over active action dimensions only. Inference samples Gaussian noise and integrates the learned vector field with $r$ 6 Euler steps.

A distinctive feature of this stage is knowledge insulation. The paper argues that if the continuous-control loss is backpropagated directly into the VLM, the visual-linguistic grounding learned during pretraining drifts, particularly for rare instruments and language-following behavior. To prevent this, the prefix hidden states used by the DiT are stop-gradient detached, so the DiT and projector learn from the continuous-control objective while the VLM continues to receive token-level supervision. The full posttraining objective is written as a weighted combination of flow-matching loss, FAST loss, and auxiliary cross-entropy terms, with $r$ 7 on the flow-matching term (Ren et al., 11 Jun 2026).

This training recipe is explicitly tied to embodiment mixing. The paper reports an appendix lesson that naïve averaging over padded action dimensions can silently rescale gradients: averaging loss over all 32 padded dimensions, when only about 8 are active for a single-arm robot, effectively weakens action gradients by $r$ 8. LabVLA therefore normalizes losses only over active dimensions and treats masking changes as optimizer-level changes (Ren et al., 11 Jun 2026).

5. Empirical evaluation

Evaluation is conducted primarily on LabUtopia, a high-fidelity simulation benchmark for scientific embodied agents, with six laboratory operations: Pick Up labware, Press Button, Open Door, Pour Liquid, Heat Beaker, and Transport Beaker (Ren et al., 11 Jun 2026). Each task is tested under both in-distribution and out-of-distribution settings, with 120 episodes per task per setting. The main metric is success rate.

Task	ID success	OOD success
Pick Up	49.2	48.3
Press Button	100.0	98.3
Open Door	65.0	65.8
Pour Liquid	43.3	34.2
Heat Beaker	83.3	87.5
Transport Beaker	85.8	85.8

LabVLA achieves the highest average success rate among all evaluated baselines in both regimes, with 71.1% average in-distribution and 70.0% out-of-distribution. The next-best average is $r$ 9 at 63.3% in-distribution and 63.2% out-of-distribution, yielding gaps of 7.8 and 6.8 percentage points, respectively (Ren et al., 11 Jun 2026). A prominent reported property is distribution-shift stability: LabVLA drops only 1.1 points from in-distribution to out-of-distribution evaluation, which the authors attribute to RoboGenesis randomization.

The benchmark also makes clear that laboratory VLA remains incomplete. Pour Liquid is difficult for every evaluated method; no baseline reaches 50% in-distribution, and LabVLA’s 43.3% in-distribution and 34.2% out-of-distribution are substantially lower than its performance on other tasks. The paper interprets this as evidence that transparent-liquid perception and precise tilt control remain open problems (Ren et al., 11 Jun 2026).

The paper also tests whether LabEmbodied-Data is useful beyond the LabVLA architecture itself. Fine-tuning X-VLA on LabEmbodied-Data raises its average over five non-saturated tasks from 49.3% to 64.3% in-distribution and from 43.7% to 63.0% out-of-distribution. Gains are especially large on Heat Beaker, from 25.8 to 68.3 in-distribution, and on Pour Liquid out-of-distribution, from 25.0 to 65.0 (Ren et al., 11 Jun 2026). This directly supports the paper’s claim that laboratory-specific supervision, not only the policy architecture, is a first-order factor.

Real-world evaluation is performed on a Franka robot across four benchtop tasks composed of 2–4 atomic skills: Shake Liquid, Pour Liquid, Magnetic Stir, and Stopper Plug/Unplug. Averaged over tasks, LabVLA attains 86.5% in clean in-domain settings, 80.0% in cluttered in-domain settings, 80.0% in clean out-of-domain settings, and 74.0% in cluttered out-of-domain settings. DreamZero is slightly ahead in cluttered settings, with 81.0 and 75.5, but LabVLA is competitive and leads DreamZero in clean out-of-domain evaluation, 80.0 versus 78.0. Both outperform $V$ 0 (Ren et al., 11 Jun 2026).

6. Position within VLA research, limitations, and future directions

Within the broader VLA literature, LabVLA occupies a domain-specific position centered on scientific laboratories rather than household manipulation. Its closest conceptual contrast is with systems that optimize a different axis of the VLA design space. EdgeVLA emphasizes deployment efficiency by removing autoregressive end-effector prediction and substituting a small LLM, with reported gains in inference speed and memory efficiency (Budzianowski et al., 18 Jul 2025). HiVLA instead decouples high-level semantic planning and visual grounding from low-level motor control through a hierarchical planner-plus-DiT structure, targeting long-horizon cluttered manipulation (Yang et al., 15 Apr 2026). Sci-VLA addresses scientific long-horizon workflows as an inference-time plugin that inserts transitional robot code between atomic skills without retraining the base lab-oriented VLA (Pang et al., 10 Feb 2026). LADEV is not a policy architecture at all, but a testing and evaluation platform for VLA manipulation models under controlled visual and linguistic perturbations (Wang et al., 2024). VLAgents addresses yet another layer of the stack: policy serving and transport for efficient VLA inference across simulation and remote hardware (Jülg et al., 16 Jan 2026). Taken together, these works suggest that the current VLA field is decomposing into separable problems of data generation, reasoning structure, action generation, evaluation, and deployment infrastructure.

LabVLA’s own limitations are explicit. Most validation remains in simulation, and the paper states that real laboratories introduce hardware drift, reagent variability, contamination risks, and safety constraints that sanitized simulation does not capture (Ren et al., 11 Jun 2026). The demonstrated autonomy is protocol following rather than adaptive scientific decision-making: LabVLA does not choose experimental conditions, revise protocols from measurements, substitute reagents, or determine whether a scientific objective has been met. Safety is largely implicit rather than formulated as a complete operational framework. Transparent-liquid perception remains weak, long-horizon reliability is still limited, and benchmark scope is narrow relative to full wet-lab workflows.

The authors propose scaling RoboGenesis to broader wet-chemistry and biology workflows, adding more instruments and robots, moving toward deployment in working laboratories, and incorporating explicit safety constraints (Ren et al., 11 Jun 2026). A plausible implication is that LabVLA should be read less as a final laboratory foundation model than as a domain-grounded systems template: workflow-synthesized supervision, embodiment-aware batching, and a training pipeline that first makes a VLM action-aware and then attaches a continuous action expert under knowledge insulation.