Cosmos-H Surgical Simulator (C-H-S-S)
- Cosmos-H Surgical Simulator is a multi-embodiment world model that predicts future surgical video frames conditioned on kinematic actions.
- It implements a unified action space with normalized kinematic vectors to simulate multiple robotic platforms from a single checkpoint.
- The model enables in silico policy evaluation and realistic synthetic data generation for training and benchmarking surgical robotics.
Searching arXiv for the specified paper and closely related surgical simulation references. arXiv search query: (Consortium et al., 22 Apr 2026) Cosmos-H-Surgical-Simulator Open-H-Embodiment Cosmos-H-Surgical-Simulator (C-H-S-S) is introduced as the first multi-embodiment, kinematic action-conditioned world foundation model for surgical robotics simulation. It is built to enable surgical scene synthesis, in silico policy evaluation, and synthetic data generation, and its defining property is that it supports multiple distinct surgical robot embodiments from a single jointly trained model checkpoint (Consortium et al., 22 Apr 2026). Within the same research program, C-H-S-S is positioned as one of the foundation-model outcomes enabled by Open-H-Embodiment, which is described as the largest open dataset of medical robotic video with synchronized kinematics to date and as infrastructure for advances in robot learning, world modeling, and related areas (Consortium et al., 22 Apr 2026).
1. Research setting and problem formulation
The motivating problem is a data and evaluation bottleneck in autonomous medical robotics. Open-H-Embodiment states that existing medical robotic datasets are small, single-embodiment, and rarely shared openly, which restricts the development of the foundation models needed by the field. In response, the dataset spans more than 49 institutions and multiple robotic platforms including the CMR Versius, Intuitive Surgical’s da Vinci, da Vinci Research Kit (dVRK), Rob Surgical BiTrack, Virtual Incision’s MIRA, Moon Surgical Maestro, and a variety of custom systems, covering surgical manipulation, robotic ultrasound, and endoscopy procedures (Consortium et al., 22 Apr 2026).
C-H-S-S is the simulation-oriented model derived from that program. Its stated purpose is not merely video generation in the generic sense, but action-conditioned prediction of the visual consequences of surgical robot motion. The descriptions emphasize three uses: generating realistic future video frames given observed data and intended actions, evaluating policies in silico rather than on real hardware, and producing synthetic data for training or benchmarking robotic policies (Consortium et al., 22 Apr 2026).
The model also sits within a specific line of surgical world-model research. The supplied comparison states that prior surgical world models were single-embodiment, trained and evaluated for one robot platform, typically dVRK, whereas C-H-S-S is the first to support multi-embodiment world modeling and simulation from a single checkpoint (Consortium et al., 22 Apr 2026). A directly relevant predecessor is Cosmos-Surg-dVRK, a surgical fine-tune of the Cosmos World Foundation Model for automated online evaluation on the dVRK platform (Zbinden et al., 17 Oct 2025).
2. Core model architecture
C-H-S-S is created by fine-tuning Cosmos-Predict~2.5, described in the supplied material as a 2B-parameter latent video diffusion transformer. The core model is a diffusion transformer that generates video in the latent space of a pre-trained variational autoencoder. Its input consists of one context video frame and a sequence of robot-specific kinematic action vectors, and its output is an autoregressive prediction of future frames conditioned on those actions (Consortium et al., 22 Apr 2026).
Action conditioning is implemented through an MLP that conditions the denoiser on each per-timestep kinematic action. This design allows the model to predict the effect of a particular control trajectory on future visual observations. Through iterative rollout, it produces sequences matching complete surgical trajectories (Consortium et al., 22 Apr 2026).
The training configuration given in the source is unusually explicit. Video resolution is with a aspect ratio. Each sample contains 13 frames, consisting of 1 context frame and 12 predictions, with actions resampled to align with video. Training uses 64 A100 80GB GPUs, the AdamW optimizer, 42,000 training steps, and batch size 1,024 (Consortium et al., 22 Apr 2026).
This architecture places C-H-S-S squarely in the class of action-conditioned world models rather than explicit rigid-body or finite-element simulators. The descriptions do not present it as relying on hand-authored mechanics or manually specified tissue parameters; instead, its operative object is the learned mapping from surgical video plus synchronized kinematics to future visual state (Consortium et al., 22 Apr 2026).
3. Unified action space and multi-embodiment simulation
The principal technical mechanism enabling multi-embodiment use is a unified action space. The supplied description states that C-H-S-S handles all embodiments via a unified action space, using zero-padding to reconcile dimensional differences between robots. All action modalities are converted to a relative translation plus 6D rotation representation and normalized per embodiment, with the action vector zero-padded to 44D (Consortium et al., 22 Apr 2026).
The consequence is that a single checkpoint can generate video for any robot that appears in its training distribution. The summary explicitly states that the model can simulate surgical video across multiple robot embodiments without retraining per robot, enabling seamless simulation of policy rollouts, consistent benchmarking across embodiments, and streamlined synthetic data generation (Consortium et al., 22 Apr 2026).
The training data used specifically for C-H-S-S is described as 32 datasets from 9 distinct robot embodiments and 10+ institutions. The broader data mixture spans benchtop settings, ex vivo tissue, clinical or cadaver tissue, and in vivo settings, and includes variable lighting, tissue deformation, occlusion, and dynamic camera viewpoints (Consortium et al., 22 Apr 2026). Clinical data is said to be highly represented; one example given is that CMR Surgical Versius-500 provides 50% of training compute across several clinical procedures (Consortium et al., 22 Apr 2026).
The supported embodiments listed in the supplied materials include commercial clinical platforms such as CMR Versius, Moon Surgical Maestro, Virtual Incision MIRA, da Vinci Research Kit variants, and BiTrack, as well as research or laboratory systems such as dVRK, KUKA LBR iiwa, Franka Panda, UR5e, and USTC Torin (Consortium et al., 22 Apr 2026). This suggests that the model’s notion of embodiment is operational rather than brand-specific: the checkpoint is trained to absorb heterogeneous kinematic interfaces into one normalized conditioning scheme.
4. Evaluation protocol and empirical behavior
The paper summary notes that open-loop evaluation of action-conditioned surgical video models is difficult because standard video-generation metrics such as FID and FVD lack action fidelity. For C-H-S-S, the reported quantitative metrics are L1, defined as mean absolute error, and SSIM, the Structural Similarity Index (Consortium et al., 22 Apr 2026).
Evaluation is reported across 25 held-out test datasets and more than 150 total rollouts. The reported pattern is differentiated by scene class. Benchtop scenes maintain low L1 and high SSIM over 72 generated frames, indicating faithful and coherent long-horizon video prediction, whereas tissue-based scenes, including clinical, cadaver, and ex vivo data, show higher L1 and lower SSIM, reflecting greater visual complexity arising from lighting, occlusion, and tissue deformation (Consortium et al., 22 Apr 2026).
The summary also highlights distributional structure in the errors. Narrow error bands in benchtop datasets are reported as indicating stable generation, while wider bands in tissue-based datasets suggest more stochasticity and greater modeling difficulty. An autoregressive “sawtooth” effect appears at chunk boundaries where the model reconditions, but the overall state is reported to be maintained well over long rollouts (Consortium et al., 22 Apr 2026).
These observations matter because the stated use cases of C-H-S-S depend on long-horizon consistency rather than only short-horizon perceptual plausibility. The available description therefore treats rollout stability across approximately 70 or more frames as central to its value for in silico policy evaluation and synthetic data generation (Consortium et al., 22 Apr 2026).
5. Relationship to Cosmos-Surg-dVRK
Cosmos-Surg-dVRK is the most direct single-platform antecedent in the supplied literature. It is described as a surgical fine-tune of the Cosmos WFM, specifically the Cosmos-Predict2-2B-Video2World variant, and is designed for fully automated online evaluation and benchmarking of surgical policies on dVRK-like tasks (Zbinden et al., 17 Oct 2025).
Its rollout protocol is explicit: at each iteration the model takes the current visual state and a sequence of kinematic actions with , then autoregressively predicts the next video frames . Conditioning is performed directly on standard relative Cartesian robot actions including translation, rotation, and jaw angle. A frozen V-JEPA 2 backbone with a 4-layer attentive classifier probe is then used to classify generated videos into success, failure, and anomaly, enabling automated evaluation rather than only manual inspection (Zbinden et al., 17 Oct 2025).
The reported empirical results provide a concrete benchmark for what a surgical world model can support. On tabletop suture pad tasks, automated evaluation in Cosmos-Surg-dVRK achieved Pearson correlation with manual labeling and with automated labeling, with both , and the agreement between human labelers and the V-JEPA 2-derived classifier reached 0. In ex-vivo porcine cholecystectomy, real versus simulated evaluations were reported as 8/9 versus 7/9 successful tasks across 9 trials. A failure-episode ablation increased MBE from 0.140 to 0.325 and reduced correlation, which the paper uses to argue that both successes and failures should be included in world-model fine-tuning (Zbinden et al., 17 Oct 2025).
In that lineage, C-H-S-S can be understood as generalizing the action-conditioned surgical world-model concept from a single embodiment to a multi-embodiment checkpoint. This is an interpretation rather than a separate claim of the paper, but it follows directly from the supplied descriptions of the two systems.
6. Relation to visuo-haptic and physics-based surgical simulation
The surrounding literature in the supplied corpus shows that surgical simulation is not a single methodological tradition. One major branch is visuo-haptic simulation for training. The survey literature describes force feedback, torque feedback, stereoscopic visualization, and decoupled visual and haptic rendering loops, and reviews frameworks such as H3D, CHAI 3D, SOFA, GiPSi, and ReachIn API, along with devices such as PHANTOM Omni/Desktop, Novint Falcon, Omega.x/Delta.x, and Maglev 200TM (Hamza-Lup et al., 2019). The low-cost “haptic” liver simulator is representative of this branch: it uses Phantom Omni devices, laparoscopic instrument handles, stereoscopic display, and a recommended diagnostic palpation force range of 2.1–2.5 Newtons, with force-map visualization for assessment (Hamza-Lup et al., 2018).
A second branch centers on explicit mechanics. The 2D surgical simulation framework for tool-tissue interaction uses mesh generation from tissue images, Position-Based Dynamics for tissue deformation, tool-tissue collision detection via python-fcl, and deformation energy computation via an implicit Euler method (Han et al., 2020). SOniCS represents a higher-order finite-element variant of this tradition by coupling SOFA and FEniCS, allowing constitutive laws to be changed by one line of code only and supporting Saint Venant-Kirchhoff, Neo-Hookean, Mooney-Rivlin, and Holzapfel-Ogden anisotropic models, with real-time haptic simulation involving a surgical tool in contact with a hyperelastic liver (Mazier et al., 2022).
Against that background, the available descriptions present C-H-S-S as a different simulation object. It is an action-conditioned world model that predicts future video frames from a context frame and kinematic action sequence, and its advertised uses are policy evaluation and synthetic data generation rather than tactile skill training or explicit force-feedback rendering (Consortium et al., 22 Apr 2026). A common misconception would therefore be to treat it as interchangeable with a visuo-haptic trainer or an explicit FEM/PBD tissue simulator. The supplied literature instead suggests complementarity: haptic and mechanics-based systems focus on force realism, collision response, constitutive modeling, and skill assessment, whereas C-H-S-S focuses on learned visual dynamics across multiple embodiments (Hamza-Lup et al., 2018, Han et al., 2020, Mazier et al., 2022, Hamza-Lup et al., 2019).
7. Limitations and prospective directions
The limitations stated in the supplied material are specific. Pixel-level metrics such as L1 and SSIM do not capture all aspects of surgical relevance, including accurate tool localization or physically plausible tool-tissue interactions. Model performance degrades in the highest-complexity scenes, particularly in vivo or highly dynamic tissue settings. Open-loop evaluation also does not capture closed-loop policy interactions (Consortium et al., 22 Apr 2026).
The same source outlines several directions for extension: development of domain-specific metrics such as spatial consistency of the tool tip or semantic segmentation, better clinical realism, expanded failure cases, pre-clinical animal validation, and closing the loop with real-time policy interaction (Consortium et al., 22 Apr 2026). The single-embodiment Cosmos-Surg-dVRK results further suggest that including both successful and failed trajectories in training is important for reducing hallucinated successes and simulation bias (Zbinden et al., 17 Oct 2025).
More broadly, the relation to Open-H-Embodiment suggests an infrastructural interpretation. The dataset paper argues that open, large-scale medical robot data collection can enable advances in robot learning and world modeling, and C-H-S-S is presented as one of the principal mechanisms by which those data become usable for multi-platform simulation, benchmarking, and synthetic-data production (Consortium et al., 22 Apr 2026). A plausible implication is that future work will connect world-model simulators, visuo-haptic environments, and task-level policy learners more tightly, but the supplied descriptions stop short of claiming such integration has already been achieved.