GR00T-H: Open VLA Model for Surgical Robotics

• GR00T-H is an open foundation vision-language-action model for surgical robotics that leverages extensive multi-embodiment data to enable task-specific control.
• It inherits a GR00T-N1.6-3B architecture with a Cosmos-2B visual-language backbone, DiT-based action expert, and robot-specific MLP action heads using a unified relative end-effector space.
• In empirical evaluations, GR00T-H uniquely achieved full end-to-end suturing completion and higher subtask success rates compared to other models, demonstrating robust performance in challenging surgical scenarios.

Searching arXiv for the specified paper and closely related references to ground the article. arxiv_search(query="(Consortium et al., 22 Apr 2026) Open-H-Embodiment GR00T-H", max_results=5, sort_by="relevance") arXiv search results for ([2604.21017](/papers/2604.21017)) Open-H-Embodiment GR00T-H:

• Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics (Consortium et al., 22 Apr 2026) Authors: ... Published: 2026-04-22 Summary: Introduces Open-H-Embodiment, GR00T-H, and Cosmos-H-Surgical-Simulator for medical robotics. Searching arXiv for the 6D rotation parameterization reference mentioned in the source material. arxiv_search(query="zhou continuity rotation representations neural networks 6D arXiv", max_results=5, sort_by="relevance") arXiv search results for zhou continuity rotation representations neural networks 6D arXiv:
• On the Continuity of Rotation Representations in Neural Networks (Zhou et al., 2018) Authors: Zhou et al. Summary: Introduces continuous rotation representations, including the 6D parameterization used to avoid discontinuities. Searching arXiv for GR00T-N1.6 to anchor the inherited architecture. arxiv_search(query="GR00T-N1.6 arXiv medical robotics foundation model", max_results=5, sort_by="relevance") arXiv search results for GR00T-N1.6 arXiv medical robotics foundation model:
• Relevant GR00T-family policy references were not clearly identified from this query.
• The architecture details used here are therefore attributed to the Open-H-Embodiment paper itself (Consortium et al., 22 Apr 2026). GR00T-H is an open foundation vision-language-action model specialized for surgical and healthcare robotics, introduced in conjunction with the Open-H-Embodiment corpus as the first open foundation VLA model for medical robotics (Consortium et al., 22 Apr 2026). It is obtained by post-training NVIDIA’s general-purpose GR00T-N1.6-3B policy on a large multi-embodiment surgical dataset and then fine-tuning on specific robot-task datasets such as suturing. In the reported evaluations, it is the only evaluated model to achieve full end-to-end task completion on a structured suturing benchmark, with 5 of 20 trials completed, and it attains 64% average success across a 29-subtask ex vivo suturing sequence. GR00T-H is therefore best understood as a surgical-domain policy model that attempts to transfer broad visual and manipulation priors across embodiments, rather than as a simulator, planner, or clinically deployable autonomous system.

1. Position within Open-H-Embodiment

GR00T-H is inseparable from the Open-H-Embodiment dataset that underlies it. Open-H-Embodiment is described as the largest open dataset of medical robotic video with synchronized kinematics to date, comprising 770 hours, 124,019 episodes, more than 49 institutions, 20 robotic platforms, and 33 task families across 5 environment types. The platforms include CMR Versius, da Vinci Si/Xi, dVRK, Rob Surgical BiTrack, Virtual Incision MIRA, Moon Surgical Maestro, KUKA Med14, Franka Panda, UR5e, Torin, and multiple flexible endoscope robots; the task families span full procedures, manipulation primitives, robotic ultrasound, tracked ultrasound, and flexible endoscopy (Consortium et al., 22 Apr 2026).

The stated motivation for GR00T-H has two parts. First, prior work on SutureBot showed that strong generalist VLAs such as OpenVLA, GR00T-N1, and $\pi_0$ fail badly on autonomous suturing, while a comparatively small surgical-specific transformer, ACT, trained only on SutureBot data, performs better. Second, the pre-existing surgical datasets emphasized in the paper—JIGSAWS, SutureBot, Expanded CRCD, and ImitateCholec—are characterized as small, single-embodiment, and typically single-institution. The work frames this as a domain-gap problem: surgical robotics requires long-horizon multi-step procedures with narrow error margins and irreversible mistakes, dexterous bimanual manipulation of needle, thread, and deformable tissue, and endoscopic visual conditions rather than fixed RGB views of rigid objects.

For GR00T-H specifically, the authors select a 601-hour real-world surgical subset of Open-H-Embodiment, excluding ultrasound, flexible endoscopy navigation, and simulation. The raw-hour mixture is dominated by clinical Versius data, but sampling of that source is capped at 20% of training steps to avoid overfitting to one platform or procedure; the remaining datasets are sampled proportionally to dataset size. This design is intended to supply broad surgical priors, including tissue behavior, endoscopic optics, and bimanual strategies, while also exposing the policy to heterogeneous kinematics and camera configurations.

2. Model architecture and action representation

GR00T-H inherits the GR00T-N1.6-3B architecture. Its backbone is a Cosmos-2B VLM for visual and language encoding, followed by a DiT-based “action expert” that serves as the action sequence model, and embodiment-specific action heads implemented as per-robot MLP projections from a shared latent action representation into a robot’s normalized action space (Consortium et al., 22 Apr 2026). The model is explicitly a VLA policy rather than a world model: it maps current observation and prompt, together with limited history, directly to actions instead of predicting future video.

The input modality is heterogeneous by design. Visual observations may consist of endoscopic images, wrist-camera images, or multi-view inputs; during post-training, images are resized to $224 \times 392$. The augmentation stack includes random crop at 95% of original size, random rotation of $5^\circ$, coarse pixel dropout of 0.5, color jitter with brightness 0.12, contrast 0.15, saturation 0.15, hue 0.02, and multi-view dropout of 0.25. Language enters as natural-language task or subtask prompts, embedded by the same backbone. In ex vivo suturing, the prompts include short imperative forms such as “pickup needle,” “handover needle,” “wrap suture,” “tie knot,” and “cut suture.”

A central design decision is the use of a unified relative end-effector action space. For each embodiment, raw kinematics are transformed into relative end-effector control, where translation is the delta tool-tip position between time $t$ and $t+1$, orientation uses a 6D rotation matrix parameterization, and gripper states are appended. The 6D representation follows the continuity-oriented formulation introduced by Zhou et al. (Zhou et al., 2018). In the paper’s interpretation, relative actions simplify learning because the model need not infer each platform’s forward kinematics; it only needs to predict how the tool should move relative to its current pose.

The output layer is deliberately not fully shared across embodiments. Each robot configuration, not merely each robot family, receives its own MLP action head. The reason given is that many surgical systems are cable-driven and exhibit non-linear, instance-specific kinematics involving cable stretch, hysteresis, and instrument variation. A single shared head per platform would therefore conflate hardware differences that are operationally significant. The architecture instead combines a single core policy with thin embodiment-specific adapters at the output layer.

3. Data processing, normalization, and optimization

All source datasets are converted into LeRobot v2.1 format, with kinematics stored in Parquet and video in MP4, while synchronized video-kinematic alignment is preserved by the contributing groups (Consortium et al., 22 Apr 2026). Per-dataset README files document robot type, whether the data are teleoperated or autonomous, operator skill, synchronization details, active arms, and clutching conditions under which kinematics may be temporarily invalid.

For GR00T-H, kinematics are transformed into relative end-effector deltas, 6D orientation, and gripper states, and then normalized. The normalization is computed per dataset and merged into per-configuration statistics through weighted moment matching, where the weights reflect the sampling ratios used in the training mixture. The normalization is per action dimension and per temporal step within the action chunk. Actions are z-score normalized and clipped to the interval $[-5, 5]$ before entering the model. This choice preserves short-horizon resolution while bringing disparate robot outputs onto a comparable scale.

The post-training hyperparameters are specified explicitly. Initialization is nvidia/GR00T-N1.6-3B; the backbone is not frozen during post-training; the action horizon is 50; action normalization is temporal z-score; state dropout is 100%, so the training regime does not rely on non-visual state. Optimization uses AdamW with learning rate $3 \times 10^{-5}$, weight decay $1 \times 10^{-5}$, cosine decay with 5% warmup, gradient clipping at 1.0, global batch size 1024, and 65,000 training steps. The reported compute budget is 32 A100 80 GB GPUs for approximately 1.5 days.

The downstream specialization recipe differs from post-training. For fine-tuning on a target task, the recommended procedure is to freeze the VLM backbone and optimize only the action expert and embodiment-specific action heads. The stated rationale is that this preserves the surgical-domain visual representation acquired during post-training while adapting the control layers to a specific task and robot. The fine-tuning datasets in the paper range from approximately 6 hours for full SutureBot training to 5.2 hours for Versius peg transfer and only 22 minutes for MIRA needle pickup.

A practical issue emphasized in the training pipeline is checkpoint selection. The authors argue that imitation-learning validation loss is a poor predictor of closed-loop task success. Their solution uses Cosmos-Surg-dVRK, a dVRK-specific action-conditioned video world model: candidate GR00T-H checkpoints are rolled out in simulation, the resulting generated videos are scored by a fine-tuned V-JEPA-2 model for per-subtask success, and checkpoints with high simulated success are selected before costly physical trials.

4. Policy interface and task specification

GR00T-H operates as a closed-loop, step-by-step policy (Consortium et al., 22 Apr 2026). At each control step, it receives current camera frames, a task or subtask prompt in natural language, and past context encoded through the transformer and associated context windowing. It then outputs an action chunk: a sequence of relative end-effector motions and gripper commands up to the action horizon. In deployment, the system can take only the first action from the predicted horizon or execute part of the chunk while replanning at a fixed rate.

This interface has two important implications. First, the model is purely vision-conditioned in the reported experiments because state dropout is 100%; it does not depend on explicit low-dimensional robot state at inference. Second, structured task decomposition is implemented through language prompts rather than separate hard-coded policies. In SutureBot and ex vivo suturing, a single VLA policy responds to prompts such as “pickup needle,” “handover needle,” “throw needle,” “wrap suture,” “tie knot,” and “cut suture.” This suggests a prompt-conditioned multi-task policy rather than a bank of isolated controllers.

A common misunderstanding is to treat GR00T-H as a planner or simulator because it is trained on long-horizon surgical data. The paper is explicit that it is neither. It is not a world model, does not minimize image reconstruction objectives, and does not generate future observations as its primary output. Its learned function is direct action prediction under visual and linguistic conditioning.

5. Empirical performance across surgical benchmarks

The headline benchmark is end-to-end SutureBot suturing on dVRK-Si. The task is decomposed into five subtasks—needle pickup, needle handover, needle throw through tissue, needle extraction, and knot tying—and success is tracked as survival through the sequence (Consortium et al., 22 Apr 2026). On needle pickup, GR00T-H, GR00T-N1.6, and ACT each retain 20 of 20 trials. At handover, GR00T-H remains at 20 of 20, while GR00T-N1.6 drops to 16 of 20 and LingBot-VA to 12 of 20. At the throw stage, GR00T-H retains 12 of 20 trials, compared with 4 of 20 for GR00T-N1.6, 4 of 20 for ACT, and 1 of 20 for LingBot-VA. The final end-to-end completion rate is 5 of 20 trials, or 25%, for GR00T-H, versus 0 of 20 for ACT, GR00T-N1.6, and LingBot-VA. Within the reported evaluation, GR00T-H is therefore the only model that completes the full pickup-to-knot-tying loop.

The paper also reports out-of-distribution SutureBot evaluation under unseen wound geometry and altered lighting, where standard endoscope illumination is replaced by external lighting. Here the metric is per-subtask performance over pickup-plus-handover, throw-plus-extract, and knot tying. ACT achieves 15% on pickup-plus-handover and 0% on the other tasks. GR00T-N1.6 obtains a 3-task average of 30%, with 40% on pickup-plus-handover, 5% on throw-plus-extract, and 50% on knot tying. GR00T-H reaches a 3-task average of 54%, with 70% on pickup-plus-handover, 42.5% on throw-plus-extract, and 45% on knot tying. The largest gain appears in throw-plus-extract, which the paper treats as a particularly difficult surgical subtask.

A separate data-efficiency study compares fine-tuning with 33% and 100% of the SutureBot dataset. At 33% of the data, approximately 2 hours, GR00T-H and ACT both reach roughly 47% average success across the three evaluated subtasks, whereas GR00T-N1.6 reaches roughly 20%. At 100% of the data, approximately 6 hours, GR00T-H reaches roughly 73%, ACT roughly 50%, and GR00T-N1.6 roughly 37%. The authors interpret this as evidence that surgical post-training improves scaling with downstream data volume.

Multi-embodiment evaluation extends beyond dVRK-Si to CMR Versius peg transfer and Virtual Incision MIRA needle pickup. Fine-tuning uses 5.2 hours for Versius peg transfer and only 22 minutes of demonstrations for MIRA needle pickup. Across all subtasks and robots, GR00T-H shows significant gains in success rate relative to GR00T-N1.6, and the aggregated average improvement is reported as statistically significant with $p < 0.001$ under Fisher’s exact test with Holm-Bonferroni correction.

The most detailed tissue-based evaluation is ex vivo suturing on skin-on pork belly using dVRK-Si. The benchmark spans a 29-subtask sequence covering needle pickup, re-grasp, handovers, set-down, wound opening, throw and extraction, suture manipulation, multiple wrapping and pulling steps, knot tying, and two cutting steps. The overall average success is 64% across all 29 subtasks. Several subtasks are near-perfect or perfect, including needle pickup at 10/10, multiple handover stages at 9/10, 10/10, and 9/10, set-down needle at 10/10, and three distinct knot-tying steps each at 10/10. The difficult subtasks include readjust needle/grasp at 4/10, open wound at 4/10, wrapping steps at 8/10, 4/10, and 8/10, grabbing the suture tail at 5/10, 6/10, and 3/10, and the two cutting subtasks at 2/10 and 3/10. Cutting is identified as a consistent failure mode.

6. Generalization, robustness, and relation to world modeling

The paper attributes GR00T-H’s generalization properties to several interacting design choices: relative end-effector actions instead of joint-space control, 6D rotation, embodiment-specific action heads, per-configuration normalization, and training on a mixture that spans dVRK, Versius, Torin, BiTrack, MIRA, KUKA, Maestro, and other systems, together with benchtop, ex vivo, and clinical environments (Consortium et al., 22 Apr 2026). The central architectural claim is that the shared transformer learns high-level surgical manipulation patterns, while thin adapters absorb embodiment-specific kinematic nuances.

The reported robustness evidence includes not only out-of-distribution visual and geometric perturbations but also hardware drift. During the evaluation period, the dVRK instruments reportedly wore out and wrist cameras were replaced, changing kinematics and optics. ACT, which had been trained on narrower data, degraded below its originally reported performance, whereas GR00T-H maintained 25% end-to-end completion. The authors interpret this as a consequence of broader prior experience rather than narrow task memorization. This suggests, though it does not prove, that multi-embodiment post-training may improve tolerance to ordinary hardware variation.

GR00T-H is also situated alongside Cosmos-H-Surgical-Simulator, but the two systems solve different problems. Cosmos-H is an action-conditioned world model based on a 2B-parameter latent video diffusion transformer fine-tuned from Cosmos-Predict 2.5; given a current frame and a sequence of actions, it generates future frames and can be rolled out autoregressively. GR00T-H, by contrast, is the controller that outputs actions. The relationship is complementary: Cosmos-Surg-dVRK, a precursor world model, is already used to evaluate candidate GR00T-H checkpoints in silico, and Cosmos-H extends the same general idea to nine embodiments for policy evaluation and synthetic data generation.

7. Limitations, safety, and prospective development

The paper is explicit that GR00T-H is not sufficiently reliable for clinical use (Consortium et al., 22 Apr 2026). Its 25% end-to-end success on SutureBot and 64% average per-subtask success on ex vivo tissue remain far below the threshold required for human surgery. The failure profile is strongly concentrated in long-horizon compounding and fine-contact manipulation. Early stages such as pickup and handover are comparatively reliable, whereas later stages involving wrapping, suture-tail grasping, and especially cutting terminate many episodes.

The cutting failure mode is discussed in mechanistic terms. According to the paper, cutting requires very precise, quick motions and nuanced contact forces, but GR00T-H uses no force sensing and has limited examples of such fine contact dynamics in training. More generally, the evaluation domain remains limited to phantoms and ex vivo tissue in controlled laboratory settings. The authors emphasize a substantial domain gap to in vivo surgery, where live tissue stiffness, bleeding, motion, physiological changes, instrument faults, and patient movement introduce contingencies that the present policy cannot detect or handle.

The dataset itself is also presented as incomplete. Open-H predominantly contains successful demonstrations; the authors note that failure cases would likely be valuable for both world-model training and robust policy learning. Existing annotations are also limited. Although some Versius phase and gesture labels exist, fine-contact labels such as specific tool-tissue contact states and failure-mode annotations are missing. The paper further notes that GR00T-H is an end-to-end black-box policy without explicit safety monitor, error detector, or decision-explanation mechanism.

The future directions listed in the source follow directly from these limitations. They include expanding Open-H to cover more procedures, tissue types, failure cases, and multimodal sensing such as force, ultrasound, and gaze; improving multi-embodiment generalization through more structured embodiment encodings; extending automated closed-loop evaluation from Cosmos-Surg-dVRK to multi-embodiment simulation with Cosmos-H; and incorporating human-in-the-loop or supervisory safety layers so that models like GR00T-H can function as assistive modules for suggestion, shared control, or subtask automation rather than as unsupervised autonomous surgeons. The paper therefore positions GR00T-H as a proof-of-concept for surgical-domain VLA scaling, but not as a complete solution to surgical autonomy.