--- title: 1X World Model Challenge url: https://www.emergentmind.com/topics/1x-world-model-challenge type: topic --- # 1X World Model Challenge The 1X World Model Challenge is a benchmark for learning generative world models from real humanoid-robot interaction data, centered on predicting what a humanoid robot will observe in the future conditioned on its own state and action history. It is organized around two complementary tracks: **sampling**, focused on forecasting future image frames, and **compression**, focused on predicting future discrete latent codes. In the challenge’s winning technical report, Team Revontuli adapts **Wan-2.2 TI2V-5B** to video-state-conditioned future frame prediction and trains a **Spatio-Temporal Transformer** from scratch for discrete-token forecasting, achieving **23.0 dB PSNR** in the sampling task and a **Top-500 CE of 6.6386** in the compression task [2510.07092]. ## 1. Conceptual scope and formal setting The challenge sits within a broader research program in which world models are treated as internal simulators of future states. In contemporary formulations, a world model may be written as a next-state simulator \(s' \sim p(s' \mid s, a)\), or as a parametric approximation of an action-conditioned transition kernel such as \(\hat{\mathcal{P}}_\phi(o_{t+1}\mid o_t, a_t)\) in a partially observable MDP [2603.25887; 2605.17912]. Within that perspective, the 1X benchmark specializes the problem to **real-world humanoid interaction**, with visual observation prediction conditioned on proprioceptive robot state. A plausible implication is that the benchmark targets a particularly demanding regime of embodied prediction: the model must account simultaneously for egocentric scene evolution, humanoid self-motion, and the induced visual consequences of robot-state trajectories. Unlike simulator-only or purely video-quality-oriented evaluations, the challenge is explicitly grounded in **real-world humanoid robot interaction data** and frames world modeling as a predictive interface for embodied control [2510.07092]. ## 2. Benchmark tasks, data, and representations The benchmark uses clips containing **77 RGB frames at \(512\times512\)** together with a state sequence \(\mathbf{s}_{0:76} \in \mathbb{R}^{77\times 25}\), where each state is **25-dimensional** [2510.07092]. The two tracks differ primarily in the representation of future prediction. | Track | Input | Output | Evaluation | |---|---|---|---| | Sampling | First 17 frames \(\mathbf{x}_{0:16}\) and \(\mathbf{s}_{0:76}\) | The 77th frame, 2 seconds into the future | PSNR | | Compression | 3 past token grids and \(\mathbf{s}_{0:63}\) | 3 future token grids of \(32\times32\) | Top-500 CE | In the **sampling track**, the task is defined as: input the first 17 frames \(\mathbf{x}_{0:16}\) and the whole state sequence \(\mathbf{s}_{0:76}\), then predict the last frame in the 77-frame clip [2510.07092]. In the **compression track**, video frames are first encoded using the **Cosmos \(8\times8\times8\) tokenizer**. A block of 17 RGB frames at \(256\times256\) becomes **3 token grids** of shape \(3 \times 32 \times 32\), and the dataset contains **306k samples**, each with **6 consecutive token grids** and **robot states \(\mathbf{s} \in \mathbb{R}^{64 \times 25}\)** [2510.07092]. The compression task is formalized as predicting future token grids from past token grids and robot states: \[ \hat{\mathbf{z}}_{H:H+M-1} \sim f_{\theta}(\mathbf{z}_{0:H-1}, \mathbf{s}_{0:63}), \] with training by teacher forcing to minimize \[ \min_\theta \, \mathbb{E}_{(\mathbf{z}_t, \mathbf{s}_t)_{t=0:K+M-1} \sim \mathcal{D}} \left[ \sum_{t=K}^{K+M-1} \text{CE}\big(\hat{\mathbf{z}}_t, \mathbf{z}_t\big) \right]. \] These two tracks therefore separate pixel-space forecasting from discrete latent-space dynamics modeling [2510.07092]. ## 3. Sampling track: adapting a foundation video model For the sampling track, Revontuli adapt **Wan-2.2 TI2V-5B**, described as a large **flow-matching generative video model** with a **30-layer DiT** backbone, to video-state-conditioned future prediction [2510.07092]. The starting point is a **Text-Image-to-Video** model, but the adaptation changes the conditioning regime from a single fixed image to a short past video segment together with robot-state trajectories. The video is temporally downsampled by a factor of 4, turning the 77-frame clip into an effective **21-frame** sequence. With the **Wan2.2-VAE**, the temporal latent length becomes \[ T_{\text{latent}} = 1 + \frac{L-1}{4} = 1 + \frac{20}{4} = 6, \] and the model operates on latent tensors of shape \(T_{\text{latent}} \times 16 \times 16\) [2510.07092]. The conditioning frames are the past frames preserved in latent form, while future latent slices are sampled. Robot-state conditioning is implemented with **AdaLN-Zero** in every DiT block. The state pipeline first downsamples the state sequence to match the video schedule, augments continuous state values with sinusoidal features, projects them with an MLP to hidden size \[ r_\text{dim} = 256, \] and then compresses them temporally with a **2-layer 1D convolutional network** to produce features of shape \((6,256)\) [2510.07092]. These features are mapped to modulation vectors that are added to the standard timestep modulation, so each latent time slice is conditioned on the aligned robot state. To adapt the 5B-parameter model efficiently, the team uses **LoRA** with **rank 32** inside the DiT backbone: \[ W' = W + \Delta W = W + BA. \] Only the low-rank update is trained while the base model remains frozen [2510.07092]. Training uses **AdamW** with constant learning rate \(4 \times 10^{-4}\), **23k training steps**, and effective batch size **1024** on **4 nodes × 8 × NVIDIA B200 GPUs** [2510.07092]. At inference time, the report emphasizes **ensemble averaging**. Multiple stochastic generations \(\hat{\mathbf{x}}^{(1)}, \dots, \hat{\mathbf{x}}^{(N)}\) are averaged, \[ \bar{\mathbf{x}} = \frac{1}{N} \sum_{i=1}^N \hat{\mathbf{x}}^{(i)}, \] which improves PSNR by smoothing regions of predictive uncertainty such as moving arms, though it worsens some perceptual metrics through over-smoothing [2510.07092]. The report explicitly compares this strategy with **Gaussian blur + histogram matching** and finds ensemble averaging more effective for the challenge metric. ## 4. Compression track: spatio-temporal token dynamics The compression-track system is a **Spatio-Temporal Transformer** trained from scratch, described as being inspired by **Genie** and **ViT** [2510.07092]. Its input consists of past token grids together with robot-state embeddings, and its output is a categorical distribution over future token grids. The design alternates **spatial attention** and **temporal attention**. Spatial attention is applied independently within each \(32\times32\) token grid, while temporal attention is applied causally across time at each fixed spatial coordinate. This factorization reduces memory cost relative to full joint space-time attention while preserving autoregressive temporal structure [2510.07092]. The model configuration is explicit: **24 ST-Transformer layers**, **8 attention heads**, embedding dimension **512**, **Pre-LayerNorm**, **QKNorm**, and **dropout 0.1** for attention, MLP, and residual pathways [2510.07092]. Robot states are incorporated as additive embeddings obtained by projecting \(\mathbf{s}_t \in \mathbb{R}^{25}\) with an MLP, processing them with a **1D convolution with kernel 3 and padding 1**, and adding absolute positional embeddings before fusion with token embeddings [2510.07092]. The probabilistic factorization is written as \[ p_\theta(\mathbf{z}_{H:H+M-1} \mid \mathbf{z}_{0:H-1}, \mathbf{s}_{0:63}) = \prod_{t=H}^{H+M-1} p_\theta(\mathbf{z}_t \mid \mathbf{z}_{