Cloak-VLA: Zero-Shot Cross-Embodiment Transfer

• Cloak-VLA is a method that masks the end-effector from wrist camera input to remove embodiment-specific visual cues, enabling zero-shot transfer from single-embodiment training data.
• It employs geometry-based rendering, silhouette calibration, and innovative mask augmentation to canonicalize wrist images, ensuring consistent perception across varied robots.
• Empirical results demonstrate that Cloak-VLA improves task progression and success rates when transferring policies to different grippers, arms, and even a five-fingered hand without extra target training data.

Searching arXiv for the cited paper and a few related works mentioned in the provided data. Cloak-VLA is a vision–language–action manipulation policy designed for zero-shot cross-embodiment transfer from single-embodiment training data. It is introduced in "Cloak: Zero-Shot Cross-Embodiment Manipulation by Masking the End-Effector from the VLA" (Piseno et al., 22 Jun 2026), where the central training recipe, "Cloak," masks the robot’s own end-effector from its wrist camera during both training and deployment. The stated objective is to remove embodiment-specific visual features from the wrist view so that a policy trained entirely on a Franka Panda with a Robotiq 2F-85 parallel-jaw gripper can be deployed zero-shot on unseen embodiments, including a different gripper, a different arm-and-gripper platform, and a five-fingered anthropomorphic hand, without collecting any target-embodiment training data (Piseno et al., 22 Jun 2026).

1. Problem setting and conceptual motivation

Cloak-VLA targets the setting of zero-shot cross-embodiment transfer from data collected on a single source embodiment. The source embodiment is a Franka Panda arm with a Robotiq 2F-85 parallel-jaw gripper, trained on DROID using approximately 70k successful episodes after filtering. Deployment is performed on embodiments never seen during training, with no new data collection and no retraining for the target robots (Piseno et al., 22 Jun 2026).

The motivating claim is that, in wrist-camera setups such as DROID, the dominant obstacle is the visual embodiment gap rather than the action space alone. In the wrist image, the end-effector occupies a large and consistent region; a ViT-based encoder therefore learns strong embodiment-specific features from its shape, color, and local motion. When the end-effector is replaced by a geometrically and visually distinct device, the wrist image becomes out-of-distribution even if the task and surrounding objects remain similar. The paper argues that standard forward-kinematics and inverse-kinematics retargeting can address kinematic differences, but that the wrist-view appearance mismatch is what prevents reliable zero-shot transfer (Piseno et al., 22 Jun 2026).

Cloak addresses this by rendering an accurate mask of the end-effector directly from robot geometry, joint state, and calibrated camera extrinsics, then removing that region from the model’s effective visual input. During training, the mask is further augmented so that the model does not merely learn a fixed silhouette corresponding to the source gripper. This yields an embodiment-agnostic wrist representation: the policy consistently observes the scene and objects with an occluded region where "some robot" is present, rather than a specific gripper or hand (Piseno et al., 22 Jun 2026).

The paper characterizes this as enabling “data outliving the hardware it was collected on.” In that framing, wrist-camera data are canonicalized into a representation that is decoupled from the concrete morphology that generated them. A plausible implication is that previously collected datasets can remain useful across hardware changes provided that geometric masking and retargeting are available.

2. Base VLA architecture and training recipe

Cloak-VLA is obtained by fine-tuning $\pi_{0.5}$ on DROID with Cloak applied (Piseno et al., 22 Jun 2026). The model uses a ViT-style vision encoder operating on 224×224 images with patch size $P=14$, producing $16 \times 16 = 256$ tokens per view. Inputs consist of a wrist image and an external camera image. These visual tokens are fused with instruction tokens and proprioceptive state in a VLA backbone, and the action head outputs joint-space action chunks in the source robot’s 8-DoF action space: seven arm joints plus one gripper degree of freedom (Piseno et al., 22 Jun 2026).

The source robot state is denoted $q_{\text{src}} \in \mathbb{R}^8$. Training uses standard behavior cloning on a demonstration dataset $\mathcal{D}$, with the paper writing the objective as

$$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$

Cloak does not alter this loss; it changes only the wrist-image inputs and the associated attention mask (Piseno et al., 22 Jun 2026).

The training pipeline combines standard and Cloak-specific elements. Standard components include the use of DROID, behavior cloning on action chunks, a pre-trained $\pi_{0.5}$ backbone and vision encoders, and a fine-tuning schedule of 100k gradient steps, approximately one epoch, on 8×H100 GPUs using an open-source "openpi" recipe. Cloak-specific additions are per-episode Silhouette Calibration for wrist camera extrinsics, geometry-based end-effector mask generation per frame, three-stage mask augmentation with rolling-based fill-in, attention-level masking aligned to the ViT patch grid, and tip-pose retargeting for deployment on new embodiments (Piseno et al., 22 Jun 2026).

A key design decision is that only the wrist view is cloaked. The external camera is not masked, because the paper states that external views have a much smaller embodiment-specific footprint (Piseno et al., 22 Jun 2026).

3. Geometry-based masking and Silhouette Calibration

The geometric mask is defined over the wrist image $I \in \mathbb{R}^{H\times W\times 3}$, with $H=W=224$. Let $q$ be the joint state, $P=14$0 the set of robot link meshes from the URDF or MJCF, $P=14$1 the wrist camera pose in the end-effector frame, $P=14$2 the end-effector world pose, and $P=14$3 the camera world pose. For each link $P=14$4, with mesh vertices $P=14$5 and world pose $P=14$6, the binary mask $P=14$7 is produced by rasterizing the union of the projected link meshes:

$P=14$8

The mask is then slightly dilated to absorb small calibration errors (Piseno et al., 22 Jun 2026).

This procedure is performed both during training and at test time, frame by frame, using forward kinematics, projection, and fast rasterization in MuJoCo. The paper explicitly states that no learned segmentation and no generative model are used (Piseno et al., 22 Jun 2026).

Because DROID wrist-camera extrinsics are too noisy for pixel-level alignment, the method introduces per-episode Silhouette Calibration. A pseudo ground-truth mask $P=14$9 is computed using image statistics from frames in which the gripper is open. The image is converted to grayscale and cropped to the bottom-right region where the gripper consistently appears. Temporal median intensity $16 \times 16 = 256$0 and temporal standard deviation $16 \times 16 = 256$1 are computed per pixel. Pixels with intensity below the median form $16 \times 16 = 256$2, pixels with standard deviation below the median form $16 \times 16 = 256$3, and the pseudo ground-truth mask is their intersection:

$16 \times 16 = 256$4

The extrinsics are then refined by optimizing the rendered mask against this pseudo ground-truth. For a candidate camera pose $16 \times 16 = 256$5, the calibration objective is

$16 \times 16 = 256$6

This 6-DoF optimization is solved with Nelder–Mead, initialized from the dataset mean pose $16 \times 16 = 256$7. The optimized $16 \times 16 = 256$8 is subsequently used to precompute masks for the training set and to construct the runtime rendering stack (Piseno et al., 22 Jun 2026).

The paper reports that mismatch sources such as model error relative to the real robot and pinhole-camera approximations are negligible in practice. This suggests that the mask-generation pipeline is robust so long as geometric and calibration errors remain within the tolerance absorbed by dilation and patch-level masking.

4. Mask augmentation and masked attention

The raw geometric mask exactly covers the source gripper. The paper argues that training directly on this fixed mask would still expose the model to a characteristic silhouette, making the silhouette itself an embodiment-specific cue. Cloak therefore randomizes the mask shape at training time through a three-stage augmentation process (Piseno et al., 22 Jun 2026).

The first stage is image rolling. Before modifying the mask shape, the method fills the currently masked region with content rolled from elsewhere in the image:

$16 \times 16 = 256$9

with $q_{\text{src}} \in \mathbb{R}^8$0. The paper gives several reasons for this choice: a ViT drops a patch only if most pixels are masked, so a few visible gripper pixels can leak through; later augmentation steps may remove parts of the mask and expose the underlying gripper; and solid-color fills, noise, or generative inpainting are either out-of-distribution, heavy, or unreliable. Rolling uses in-distribution image content already present in the frame (Piseno et al., 22 Jun 2026).

The second stage attaches outward-pointing capsules to the mask boundary. The number of capsules is sampled as $q_{\text{src}} \in \mathbb{R}^8$1. Each capsule anchor point $q_{\text{src}} \in \mathbb{R}^8$2 is sampled uniformly from the boundary $q_{\text{src}} \in \mathbb{R}^8$3, with radius $q_{\text{src}} \in \mathbb{R}^8$4 px, length $q_{\text{src}} \in \mathbb{R}^8$5 px, and orientation $q_{\text{src}} \in \mathbb{R}^8$6. These capsules are unioned with the original mask (Piseno et al., 22 Jun 2026).

The third stage removes random disks from the enlarged mask. The number of removed disks is sampled as $q_{\text{src}} \in \mathbb{R}^8$7, with centers $q_{\text{src}} \in \mathbb{R}^8$8 drawn uniformly inside the current mask and radii $q_{\text{src}} \in \mathbb{R}^8$9 px. The overall augmented mask is

$\mathcal{D}$0

The resulting family of mask shapes varies in size, jaggedness, elongation, and partial occlusion, preventing the policy from observing the exact Robotiq silhouette during training (Piseno et al., 22 Jun 2026).

At test time, this augmentation is disabled. The deployed system uses only geometric mask construction, with no capsules, disks, or rolling; the paper describes this as “just mask + attention gating” (Piseno et al., 22 Jun 2026).

The augmented mask is applied at the transformer level. The wrist image is patchified at resolution $\mathcal{D}$1, and the mask is downsampled to patch resolution by majority vote: a patch is marked as masked if more than 50% of its pixels lie within $\mathcal{D}$2. The flattened patch mask $\mathcal{D}$3 has 256 entries for the 16×16 patch grid (Piseno et al., 22 Jun 2026).

Rather than removing masked tokens from the sequence, the method preserves all tokens but prohibits attention to masked keys. For self-attention with $\mathcal{D}$4, the paper defines

$\mathcal{D}$5

where

$\mathcal{D}$6

Thus no token can attend to a masked patch as a key. The same patch-level mask is reused in all self-attention layers and is also passed to the cross-modal VLA backbone to gate attention from text and state onto visual tokens (Piseno et al., 22 Jun 2026).

5. Cross-embodiment transfer via tip-pose retargeting

Cloak-VLA retains the source embodiment’s action space, so deployment on new robots requires a retargeting mechanism. The paper states that this is done through tip-pose retargeting, which connects the source robot’s joint-space outputs to target embodiments with different kinematics and morphology (Piseno et al., 22 Jun 2026).

The output mapping, denoted src2trg in the paper’s description, begins from a predicted source joint action $\mathcal{D}$7. Forward kinematics are used to compute two fingertip poses:

$\mathcal{D}$8

Inverse kinematics on the target robot then solves for

$\mathcal{D}$9

The target gripper or hand degrees of freedom are controlled by the scalar gripper command $$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$0 contained in $$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$1. For a multi-finger hand, the paper states that the hand joints are linearly interpolated between predefined open and closed joint configurations (Piseno et al., 22 Jun 2026).

The input mapping, denoted trg2src, converts target state into the source-state representation expected by the policy. Given target state $$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$2, target fingertip poses are computed as

$$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$3

and inverse kinematics on the source embodiment solves

$$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$4

The source gripper opening is computed analytically from the distance between target fingertip positions (Piseno et al., 22 Jun 2026).

Inverse kinematics is implemented with Mink and uses different task structures for the two mappings. For src2trg, the paper lists FrameTask on left and right fingertips with position cost 1.0 and orientation cost 0.05, a wrist orientation regularizer, a DampingTask that is heavier on the arm and lighter on the hand, and a PostureTask with a small bias on arm joints. For trg2src, it lists a position-only FrameTask on tips with orientation cost 0, a wrist orientation regularizer, very heavy gripper damping, and a PostureTask on all degrees of freedom (Piseno et al., 22 Jun 2026).

This design separates arm positioning from gripper closure and is intended to suppress unstable null-space motion. The paper also identifies a limitation: because Cloak-VLA represents manipulation through two fingertip poses plus a scalar grasp variable, it is suited to skills expressible in that interface and does not exploit the full dexterity of a multi-finger hand for in-hand reorientation or complex grasping (Piseno et al., 22 Jun 2026).

6. Dataset, embodiments, evaluation protocol, and empirical results

The training dataset is DROID collected on the source embodiment: a Franka Panda arm with a Robotiq 2F-85 parallel-jaw gripper and a wrist camera mounted near the gripper. The paper reports approximately 95k episodes in total, from which approximately 22% labeled failures are removed following OpenPi filtering, approximately 1.7% camera-extrinsic outliers are removed using Silhouette Calibration, and 5% are held out for validation. The final training set contains approximately 70k episodes, corresponding to approximately 350 hours (Piseno et al., 22 Jun 2026).

Each episode contains wrist and external images, joint trajectories $$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$5, gripper state $$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$6, a language instruction, and expert joint-command actions for behavior cloning. No training data from any robot other than the source embodiment are used (Piseno et al., 22 Jun 2026).

Evaluation is performed on four physical setups:

Embodiment	Configuration	Role
Original gripper	Franka arm + Robotiq 2F-85	Source embodiment baseline
UMI gripper	Franka arm + different parallel-jaw gripper	Unseen gripper, same arm
YAM arm and gripper	Different arm + different gripper platform	Unseen arm and gripper
Sharpa hand	Franka arm + five-fingered anthropomorphic hand	Unseen dexterous hand

The UMI embodiment differs in color, geometry, opening width, and fingertip shape; the YAM platform differs in kinematics, link geometry, and appearance and requires a separate IK pipeline; the Sharpa hand has a drastically different silhouette and uses a custom wrist-camera mount to roughly align the finger region with the source gripper’s image region (Piseno et al., 22 Jun 2026).

The benchmark comprises four tabletop tasks, each with 12 randomized trials per embodiment and a maximum duration of 60 seconds per trial: Pick and Place, Remove, Move, and Fold/Unfold towel. Progress is measured by progression rate, defined as the fraction of progression stages completed, and by binary success rate corresponding to completion of the final stage (Piseno et al., 22 Jun 2026).

The principal baselines are: v0, a $$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$7 model fine-tuned on DROID with no masking and deployed on unseen embodiments with tip-pose retargeting; v3.0, which uses the bare source mask with no augmentation; v3.2, a target-mask overlay variant inspired by Shadow that overlays target silhouettes known in advance during training; and LAP-VLA, a language-action-pretraining style VLA that acts in a 6-DoF wrist-pose space and is deployed with IK (Piseno et al., 22 Jun 2026).

Task-average progression rate is reported as follows (Piseno et al., 22 Jun 2026):

Embodiment	v0 / v0 + retargeting	LAP-VLA	v3.0	v3.2	Cloak-VLA
Source gripper	89.3%	87.9%	85.8%	81.6%	88.0%
UMI gripper	69.0%	60.9%	83.2%	82.1%	85.1%
YAM arm + gripper	69.5%	68.5%	77.8%	71.9%	86.3%
Sharpa hand	54.4%	60.2%	65.1%	72.1%	81.8%

These results are used in the paper to support three claims. First, Cloak-VLA largely preserves source-embodiment performance: on the original gripper it is within error margin of v0 and close to LAP-VLA. Second, it substantially improves zero-shot transfer to unseen embodiments, especially Sharpa and YAM. Third, the improvement is attributed primarily to closing the visual embodiment gap rather than only changing the action interface (Piseno et al., 22 Jun 2026).

Success-rate results intensify the same pattern. For the Sharpa hand, task-average success is reported as 16.7% for v0 + retargeting, 31.2% for LAP-VLA, 39.6% for v3.0, 50.0% for v3.2, and 62.5% for Cloak-VLA (Piseno et al., 22 Jun 2026). The paper further notes that Cloak-VLA is consistently best or second-best on UMI and YAM as well.

7. Ablations, interpretation, related work, and limitations

A central ablation concerns mask augmentation. On Sharpa, the progression rates are 65.1% for v3.0, 72.1% for v3.2, and 81.8% for Cloak-VLA. On UMI the corresponding values are 83.2%, 82.1%, and 85.1%; on YAM they are 77.8%, 71.9%, and 86.3% (Piseno et al., 22 Jun 2026). The paper interprets these results as showing that masking alone already helps, but a fixed silhouette remains brittle because it is still tied to the source embodiment. Target-mask overlay helps in some cases but requires target silhouettes to be known during training, which is incompatible with open-ended zero-shot deployment. Cloak’s capsule-and-disk augmentation does not assume any specific target and is presented as more robust (Piseno et al., 22 Jun 2026).

The paper also addresses the concern that masking may remove useful visual information even on the source robot. It reports that source-embodiment performance remains essentially unchanged for Cloak-VLA relative to v0, while an overly aggressive masking variant, v3.0, does degrade performance. The stated explanation is that the masked region contains mostly robot body, while the model can still rely on surrounding object and table geometry, the external camera, and proprioception (Piseno et al., 22 Jun 2026).

Qualitative behavior aligns with the quantitative findings. Cloak-VLA is described as achieving clean pick-and-place on Sharpa, successful towel folding on UMI, and clean removal of a marker from a bowl on YAM. By contrast, v0 + retargeting is reported to misgrasp objects or grasp both object and container, and LAP-VLA is reported to misalign grasps because of wrist-view visual mismatch despite having an alternative action interface (Piseno et al., 22 Jun 2026). This is used to reinforce the paper’s thesis that closing the vision gap is more important than refining the action space alone.

In its discussion of related work, the paper contrasts Cloak with several classes of prior methods. Multi-embodiment VLAs such as OpenVLA, Octo, $$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$8, $$\mathcal{L}(\theta) \;=\; \mathbb{E}_{(I, I_{\text{ext}}, q, \ell, a)\sim\mathcal{D}} \left[\; \ell_{\text{BC}}\Big(f_\theta(I, I_{\text{ext}}, q, \ell),\, a\Big) \right].$$9, X-VLA, GR00T, and RDT-1B are described as typically aggregating data from many robots and using robot IDs, robot-specific tokenizers, or padded action spaces; they support transfer among seen robots but generally require new data or adaptation for new embodiments. RDT2 and LAP-VLA are described as closer antecedents, but the paper argues that they do not confront a severe visual embodiment gap because their transferred wrist views remain relatively similar across embodiments (Piseno et al., 22 Jun 2026).

The paper also contrasts Cloak with image-editing and generative approaches such as Mirage, RoboEngine, ROVIAug, and human-to-robot video pipelines, which use segmentation, inpainting, or robot replacement. It identifies segmentation errors, inpainting mismatches, and generative artifacts as limitations of those approaches, whereas Cloak uses only geometry-based masking and therefore avoids learned segmentation and online synthesis (Piseno et al., 22 Jun 2026). Relative to Shadow, ARRO, and RoboEngine, Cloak is distinguished by its reliance on known geometry rather than learned segmentation and by its mask augmentation, which avoids requiring the target silhouette to be specified in advance (Piseno et al., 22 Jun 2026).

The paper identifies several limitations. Cloak-VLA is constrained by a two-tip-plus-scalar-grasp action abstraction; it depends on reasonably accurate URDF or MJCF models, calibrations, and camera intrinsics and extrinsics; it is tailored to wrist-camera settings where the end-effector dominates a known image region; and it retains a residual transfer gap on challenging tasks and embodiments. The inverse-kinematics regularization used to stabilize retargeting can also suppress useful wrist-tilt behaviors (Piseno et al., 22 Jun 2026). Suggested future directions include combining Cloak with richer dexterous skill datasets, per-embodiment IK tuning, few-shot adaptation from the embodiment-agnostic checkpoint, and extension of masking to additional views or modalities when robot-specific signals dominate them (Piseno et al., 22 Jun 2026).

Overall, Cloak-VLA is presented as a method for isolating cross-embodiment transfer from the dominant wrist-view appearance shift. Its technical contribution is not a new action-learning objective but a geometry-consistent visual canonicalization procedure integrated into an existing VLA pipeline. Within the evidence reported in (Piseno et al., 22 Jun 2026), that choice is sufficient to preserve source-embodiment performance while markedly improving zero-shot transfer to unseen grippers, unseen arm-and-gripper platforms, and an unseen five-fingered hand.