EmbodieDreamer: Real2Sim2Real Transfer Framework

• EmbodieDreamer is a framework for advancing Real2Sim2Real transfer by aligning simulator dynamics and photorealistic observations.
• It couples a differentiable physics calibration module (PhysAligner) with a conditional video diffusion module (VisAligner) to bridge physical and appearance gaps.
• Experiments demonstrate a 3.74% reduction in parameter estimation error and a 29.17% boost in real-world task success during policy training.

EmbodieDreamer is a framework for advancing Real2Sim2Real transfer for policy training via embodied world modeling. It targets the two dominant sources of transfer failure in embodied manipulation: the Real2Sim physical gap and the Sim2Real appearance gap. Its architecture couples a differentiable physics calibration module, PhysAligner, with a conditional video diffusion module, VisAligner, so that policy learning can proceed in a simulator whose dynamics are aligned to real trajectories and whose observations are translated into photorealistic videos conditioned on simulation states. In reported experiments, PhysAligner reduced physical parameter estimation error by 3.74% compared to simulated annealing methods while improving optimization speed by 89.91%, and policies trained in the generated photorealistic environment improved average real-world task success by 29.17% after reinforcement learning (Wang et al., 7 Jul 2025).

1. Real2Sim2Real setting and problem scope

EmbodieDreamer is formulated within the Real2Sim2Real workflow. In this workflow, real robot data are first used to configure or calibrate a simulator, policies are then trained primarily in simulation, and the learned policies are finally deployed or fine-tuned on the real robot. The central claim is that simulator quality must be improved along two orthogonal axes for transfer to be reliable: physical dynamics and visual appearance. The physical gap arises because simulated dynamics depend on approximate rigid-body models, simplified contacts, and uncertain robot-specific parameters such as joint stiffness, damping, friction, and control gains. The appearance gap arises because simulator renderings do not reproduce real textures, lighting, reflections, clutter, and camera artifacts with sufficient fidelity. These mismatches degrade downstream policy transfer, particularly for vision-heavy manipulation (Wang et al., 7 Jul 2025).

The framework is motivated by the cost structure of real-world data collection. Real robot operation requires hardware maintenance, scene resets, human supervision, and safety procedures, and it proceeds in real time. Simulation, by contrast, offers cheap resets, scalable rollout generation, and safe exploration, but only if the simulator is sufficiently faithful. EmbodieDreamer therefore treats simulator improvement itself as the primary learning target: state-space alignment is handled by physics calibration, while pixel-space alignment is handled by conditional video generation. The paper describes this combination as embodied world modeling because it models how the world evolves for the robot both in dynamics and in observation space (Wang et al., 7 Jul 2025).

2. Architecture and embodied world modeling

The pipeline begins with a modest collection of real robot episodes on Cobot Mobile ALOHA. Each episode includes joint positions $q$, actions $a$, and real RGB video $V_{\mathrm{gt}}$. These data drive two alignment stages. First, PhysAligner calibrates simulator parameters from real state-transition data $(S_{t-1}, a_{t-1}, S_t)$. Second, VisAligner uses the aligned simulator to render low-fidelity robot observations and then translates those renderings into photorealistic video conditioned on simulation states. The resulting environment is then used for downstream policy training by reinforcement learning and imitation learning (Wang et al., 7 Jul 2025).

The framework separates Real2Sim from Sim2Real, but its internal logic is explicitly coupled. Real2Sim consists of physics alignment: the simulator is tuned so that open-loop simulated trajectories better match real trajectories. Sim-world modeling consists of two learned components: a differentiable surrogate model for dynamics and a conditional video diffusion model for appearance. Sim2Real then uses this improved simulator as the training substrate for ACT-based policies. The design therefore does not replace the simulator with a monolithic latent world model; rather, it augments the simulator with learnable modules that address its dominant failure modes (Wang et al., 7 Jul 2025).

Component	Function	Key details
PhysAligner	Reduce the Real2Sim physical gap	Jointly optimizes robot-specific parameters such as control gains and friction coefficients
VisAligner	Bridge the Sim2Real appearance gap	Conditional video diffusion translates low-fidelity simulated renderings into photorealistic videos conditioned on simulation states
Policy training stack	Use the aligned environment for learning	ACT-based SFT, preference-based RL, and IL augmentation

3. PhysAligner and differentiable system identification

PhysAligner is a differentiable physics module for simulator calibration. In the simulator, the next pose under parameters $(f,p,d)$ is written as

$P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$

where $f$, $p$, and $d$ denote friction, stiffness, and damping. To avoid direct optimization through a non-differentiable or non-smooth simulator, the framework builds a simulation dataset by randomly sampling parameter settings, replaying recorded actions, and collecting the resulting simulated next states. A surrogate model is then trained:

$P_t^{*} = M_{f,p,d}(a_{t-1}, S_{t-1}),$

with $a$0 implemented as a 3-layer MLP (Wang et al., 7 Jul 2025).

The surrogate is learned with a mean-squared objective against simulator outputs,

$a$1

and then frozen. Physical parameters are subsequently refined by minimizing mismatch to real trajectories,

$a$2

The appendix further defines a trajectory loss with separate translation and rotation terms, with translation computed as

$a$3

This formulation turns system identification into supervised surrogate learning followed by gradient-based parameter fitting, instead of black-box search (Wang et al., 7 Jul 2025).

The reported comparison is against a SimplerEnv baseline using simulated annealing with 400 optimization steps on 20 random episodes from RT-1. PhysAligner samples 50 parameter sets to build the initial training set and uses gradient descent with learning rate $a$4. Quantitatively, trajectory error decreases from $a$5 to $a$6, rotation error from $a$7 to $a$8, and time cost from $a$9 s to $V_{\mathrm{gt}}$0 s, while translation error changes from $V_{\mathrm{gt}}$1 to $V_{\mathrm{gt}}$2. The aggregate interpretation in the paper is that physical parameter estimation error is reduced by 3.74% and optimization speed is improved by 89.91% relative to simulated annealing (Wang et al., 7 Jul 2025).

4. VisAligner and appearance transfer by conditional video diffusion

VisAligner addresses the Sim2Real appearance gap by learning a conditional latent video diffusion model. For a manually collected episode of length $V_{\mathrm{gt}}$3, the inputs are joint positions and actions $V_{\mathrm{gt}}$4 together with ground-truth camera video $V_{\mathrm{gt}}$5. Conditioning is explicitly disentangled into three factors. The robot condition is obtained by replaying trajectories in a simulator with the robot URDF $V_{\mathrm{gt}}$6,

$V_{\mathrm{gt}}$7

The foreground object condition is obtained through Grounded-SAM / SAM2 segmentation, producing a binary mask that is added pixel-wise to the robot observation to form $V_{\mathrm{gt}}$8. The background condition is a single reference image $V_{\mathrm{gt}}$9, justified by the static-camera assumption (Wang et al., 7 Jul 2025).

For a video of length $(S_{t-1}, a_{t-1}, S_t)$0, the condition sequence is

$(S_{t-1}, a_{t-1}, S_t)$1

and both condition and target video are encoded with a VAE encoder into latent tensors. Forward diffusion is defined as

$(S_{t-1}, a_{t-1}, S_t)$2

and the denoising network $(S_{t-1}, a_{t-1}, S_t)$3 is trained with

$(S_{t-1}, a_{t-1}, S_t)$4

The implementation uses an SVD-style latent video diffusion backbone with spatial-temporal attention, latent channel dimension $(S_{t-1}, a_{t-1}, S_t)$5, input resolution $(S_{t-1}, a_{t-1}, S_t)$6, and sequence length $(S_{t-1}, a_{t-1}, S_t)$7 frames (Wang et al., 7 Jul 2025).

The ablation on object segmentation isolates the value of explicit foreground conditioning. On the RT-1 test set, removing object segmentation yields FVD $(S_{t-1}, a_{t-1}, S_t)$8, PSNR $(S_{t-1}, a_{t-1}, S_t)$9, SSIM $(f,p,d)$0, and LPIPS $(f,p,d)$1. EmbodieDreamer with segmentation reduces FVD to $(f,p,d)$2, increases PSNR to $(f,p,d)$3, raises SSIM to $(f,p,d)$4, and lowers LPIPS to $(f,p,d)$5. The qualitative interpretation given is that segmentation preserves object structure and pose during grasp, whereas omitting it leads to deformation or disappearance of manipulated objects (Wang et al., 7 Jul 2025).

5. Policy learning, evaluation tasks, and transfer results

The downstream policy is ACT, an Action Chunking Transformer that consumes visual observations and joint positions and outputs an action chunk of fixed length $(f,p,d)$6. Supervised fine-tuning uses 50 real demonstrations per task for 1000 epochs. Reinforcement learning starts from a single-view SFT checkpoint and uses preference-based optimization inspired by GRAPE and TPO. The trajectory-level objective is

$(f,p,d)$7

with TPO-style training on preferred and dispreferred trajectories. The trajectory log-likelihood for ACT is defined as

$(f,p,d)$8

The action space is represented as 14-dimensional action vectors, and trajectory ranking is based on terminal Euclidean distance between end-effector and target (Wang et al., 7 Jul 2025).

Real-world evaluation is performed on Cobot Mobile ALOHA over four manipulation tasks: Grab Paper Cup, Put Pen in Cup, Put Remote in Box, and Put Cup on Mat. The paper reports both grasp success and full task success where applicable. Task-level results show that RL with EmbodieDreamer improves Grab Paper Cup to $(f,p,d)$9 task success, Put Pen in Cup to $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$0 grasp/task success, Put Remote in Box to $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$1, and Put Cup on Mat to $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$2. These exceed both the three-view SFT baseline and the one-view SFT baseline across the reported tasks (Wang et al., 7 Jul 2025).

Method	Setting	Avg task success
SFT	3 views	0.60
SFT	1 view	0.49
RL with EmbodieDreamer	starts from one-view SFT	0.77

The average task success of $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$3 constitutes the paper’s headline downstream result: a 29.17% improvement over the best SFT baseline, which achieved $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$4. The same environment is also used for imitation-learning augmentation. By modifying foreground and background conditions with Gemini-2.0-Flash-Preview-Image-Generation, replaying original actions in simulation, and synthesizing new photorealistic videos with VisAligner, the framework produces additional demonstrations for robustness training. In the background-change experiment with added A4 paper, the generated-data-plus-resume setting reaches $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$5 success on A4 horizontal and $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$6 on A4 vertical, indicating that the appearance model can support robustness-oriented augmentation rather than only reinforcement learning (Wang et al., 7 Jul 2025).

6. Relation to other “Dreamer” systems and reported limitations

EmbodieDreamer belongs to a broader but heterogeneous cluster of embodied systems that use the language of “dreaming,” “imagination,” or narrative internal models. It is distinct from the DREAMT model for embodied motivational conversational storytelling, which organizes an embodied storytelling system around the mnemonic layers Description/Dialogue/Definition/Denotation, Realization/Representation/Role, Explanation/Education/Entertainment, Actualization/Activation, Motivation/Modelling, and Topicalization/Transformation (Powers, 2019). It is also distinct from Layout-aware Dreamer, an embodied referring expression grounding agent whose Layout Learner predicts room-type distributions and whose Goal Dreamer imagines the destination beforehand; on REVERIE test-unseen, that system improves navigation success by $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$7 and remote grounding success by $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$8 relative to the previous state of the art (Li et al., 2022). A further neighboring use of the label is Grounded-Dreamer, a text-to-3D pipeline that inserts a 4-view bottleneck, applies attention refocusing during multi-view diffusion, and performs hybrid SDS plus sparse RGB optimization; its full model requires $P_t = \mathrm{Sim}_{(f,p,d)}(a_{t-1}, S_{t-1}),$9 GPU-hours per asset and reaches CLIP R-Precision $f$0 on the reported benchmark (Li et al., 2024).

Within that landscape, EmbodieDreamer should not be read as a single canonical “Dreamer” algorithm. The 2025 system is neither a conversational storytelling architecture, nor a navigation-by-goal-imagination policy, nor a text-to-3D asset synthesizer. It is also not a latent world-model in the Dreamer RL sense: its learned dynamics component is a surrogate for system identification, its visual component is a conditional diffusion model for photorealistic observation synthesis, and reinforcement learning is performed on top of the improved simulator rather than inside a purely latent imagined environment. This suggests that, across embodied AI, “dreaming” functions as a family resemblance term for structured internal simulation, generative priors, or imagined intermediate representations rather than a single method class (Wang et al., 7 Jul 2025).

The reported limitations are correspondingly specific. VisAligner is diffusion-based and slow, requiring about 2 minutes per sequence, which makes it unsuitable for real-time control without acceleration. PhysAligner depends on the fidelity of the underlying simulator and therefore inherits the simulator’s modeling limitations in complex or unstructured environments. The appearance model assumes a static camera and relies on segmentation quality for foreground fidelity. The evaluation is concentrated on single-arm manipulation tasks with a static camera, so extensions to multi-camera or dynamic-camera settings, richer contact phenomena, and more complex embodied tasks remain open directions identified by the paper (Wang et al., 7 Jul 2025).