Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories

Abstract: We present Xiaomi-Robotics-1, a foundational vision-language-action (VLA) model capable of (1) following diverse language instructions to perform a wide range of mobile manipulation tasks in unseen environments out-of-the-box, and (2) efficiently adapting to novel downstream tasks with minimal fine-tuning data. We propose a two-stage training recipe consisting of pre-training and post-training. During pre-training, we imbue the model with broad and generalizable action-generation capabilities by training on over 100k hours of real-world manipulation trajectories collected via UMI devices. Crucially, we develop a scalable auto-labeling pipeline that annotates trajectory clips with natural languages describing scene state transitions, providing rich and precise conditioning for action learning. During post-training, we aim to align these capabilities with robot embodiments and imperative instructions that humans naturally use to prompt robots. Extensive experiments demonstrate strong scaling behavior. Xiaomi-Robotics-1 consistently improves with increased data scales and model sizes during pre-training. This scaling behavior directly transfers to post-training, where a stronger pre-training model yields better out-of-the-box real-robot performance in unseen environments. Furthermore, Xiaomi-Robotics-1 serves as a strong robot foundation policy that can be efficiently fine-tuned on complex, dexterous tasks with high data efficiency. Across multiple simulation benchmarks, Xiaomi-Robotics-1 outperforms state-of-the-art methods. Notably, it establishes a new state-of-the-art with a 57.6% success rate on RoboCasa365, surpassing the previous best of 46.6%. Furthermore, it achieves an average score of 20.07 on RoboDojo, significantly outperforming the prior state-of-the-art (13.07). Code and model checkpoints will be released. Project page: https://robotics.xiaomi.com/xiaomi-robotics-1.html

• The paper demonstrates that scaling real-world pre-training data consistently improves action prediction and transfers to robot performance, raising out-of-the-box success from 26% to 75%.
• Its two-stage recipe combines Qwen3-VL, diffusion-based action generation, automated language labeling, and cross-embodiment post-training to support diverse manipulation tasks with limited adaptation data.
• The model achieves leading results on four simulation benchmarks, including 57.4% on RoboCasa365 and 32.1% on its Composite-Unseen split, while the experiments suggest data volume currently matters more than model size.

Xiaomi-Robotics-1 is a vision-language-action (VLA) foundation model trained on more than 100,000 hours of real-world manipulation trajectories, with the explicit goal of studying whether scaling behavior—well established for large language and vision-LLMs—transfers to robot policies evaluated on real hardware. The paper makes two central claims: first, that pre-training on massive UMI-collected data with automated language annotation yields consistent gains in action prediction as data and model size grow; second, that these pre-training gains transfer monotonically to out-of-the-box real-robot performance after post-training. The model also achieves state-of-the-art results on four simulation benchmarks, including a 57.6% success rate on RoboCasa365 versus a prior best of 46.6%.

Two-stage training recipe

The system follows an LLM-inspired recipe of pre-training followed by post-training. Pre-training uses over 100k hours of real-world trajectories captured with Universal Manipulation Interface (UMI) handheld grippers across households, commercial premises, industrial sites, offices, and outdoor spaces. Because manual segmentation and language annotation are prohibitive at this scale, the authors build an auto-labeling pipeline that segments trajectories into fixed-length clips and uses Qwen3.5-27B to caption scene state transitions (of grippers and interacting objects). A producer–consumer pipeline with hundreds of concurrent captioning requests labels the entire corpus in roughly two weeks—a key enabler for the scale claimed.

Post-training uses roughly 10k hours of cross-embodiment data: over 7,200 hours of in-house mobile-manipulator and dual-arm robot data, over 1,000 hours of instruction-labeled UMI data, and open-source datasets (Bridge V2, RT-1, DROID), with idle segments filtered out. Post-training serves two alignment functions: transferring action generation from UMI grippers to robot embodiments, and shifting language conditioning from descriptive state-transition prompts to imperative instructions humans naturally issue.

Architecture

The model is a Mixture-of-Transformers coupling a pre-trained Qwen3-VL backbone with a diffusion transformer (DiT) of matching depth but smaller hidden size, generating action chunks via flow matching conditioned on proprioceptive state and the VLM's KV cache. Flow-matching timesteps are sampled from a Beta(1.5, 1) distribution weighted toward noisier timesteps, following $\pi_0$, and inference uses 5-step Euler integration ($\Delta\tau = 0.2$). Three variants are trained: 2B, 5B, and 10B total parameters.

Two design details are notable. First, an auxiliary Choice Policies head lets the VLM itself predict $K$ candidate action chunks with scores under a winner-takes-all loss, accelerating convergence by steering VLM representations toward action-relevant features. Second, the authors observe empirically that letting DiT tokens attend to the KV cache of these action-related tokens degrades performance—hypothesized to be a shortcut where the DiT copies VLM outputs rather than grounding generation in visual-textual context—so those tokens are excluded from DiT attention. Training jointly minimizes flow-matching loss, regression loss, and next-token-prediction loss on vision-language data (weight $\lambda = 0.1$; VL-to-trajectory sampling ratio 1:9), with token packing and four flow-matching timesteps per sample to amortize VLM compute.

Scaling behavior

Data-scaling experiments with the 5B variant on 12.5%–100% of ~20k hours show monotonic validation MSE reduction; at 12.5% and 25%, validation loss rises after initial descent (overfitting), while 50% and 100% yield monotonic improvement. Model scaling from 2B to 10B also improves validation error, but the gaps between model sizes are smaller than those across data scales. The authors draw a pointed conclusion: at billions-parameter scale, capacity may already suffice for the current data distribution, making data volume the primary bottleneck—a claim that prioritizes further corpus collection over larger models.

Critically, this scaling transfers downstream. Post-training checkpoints initialized from progressively larger pre-training corpora show overall out-of-the-box success rates rising monotonically from 26% (no action pre-training, Qwen3-VL initialization only) to 75% (full pre-training data); even 12.5% of pre-training data more than doubles the baseline (53%). Gains are largest on contact-rich tasks—the no-pre-training baseline fails completely on shoe tidying, whereas the full-scale model reaches 75%. Model size similarly transfers: 61% (2B), 75% (5B), 79% (10B), with shoe tidying climbing from 58% to 92%. Notably, doubling pre-training data from 50% to 100% still adds 6 percentage points, so the authors report no saturation—though they concede marginal gains moderate as the corpus grows.

Downstream adaptation and benchmark results

Fine-tuned on four held-out dexterous tasks (phone packing, laundry loading, printer refilling, box packing) with under 10 hours of data per task on average, Xiaomi-Robotics-1 reaches 75% average success and 90% progress, versus 40%/66% for $\pi_{0.5}$ fine-tuned via the official OpenPi protocol. The largest margins appear on deformable-object manipulation (printer refilling: 70% vs. 20% for the best baseline in the low-data setting) and long-horizon mobile manipulation (laundry loading: 80% success, 96% progress).

On simulation benchmarks, the model reports state-of-the-art results across all four evaluations:

Benchmark	Result	Prior best
RoboCasa	74.5%	72.6%
RoboCasa365	57.4%	46.6%
RoboDojo (avg. score)	20.07	13.07
VLABench (avg. SR)	59.1%	53.2%

The most consequential result is RoboCasa365's Composite-Unseen split (32.1% vs. 7.9–14.9% for baselines), which tests zero-shot recomposition of atomic skills into novel long-horizon tasks—direct evidence that the pre-training recipe supports compositional generalization rather than task-specific overfitting. On RoboDojo, the model ranks first on four of five capability dimensions; it concedes the Memory dimension to Hy-Embodied-0.5-VLA, which explicitly models memory, noting that Xiaomi-Robotics-1 was evaluated without history observations. The paper also demonstrates a long-horizon room-level suitcase-packing task exceeding 10 minutes autonomously.

Limitations and open questions

Several caveats bear directly on the headline claims. The scaling experiments use only ~20k hours of the full 100k-hour corpus due to compute budget, so the reported scaling trends are extrapolated from a subset rather than demonstrated at full corpus scale. Validation metrics during pre-training are action MSE on held-out UMI data, not task success; the link between lower validation error and real-robot competence is established only indirectly through the post-training transfer experiments. The auto-labeling pipeline depends on Qwen3.5-27B caption quality, and the paper does not quantify annotation error rates or their effect on conditioning precision. The shortcut-mitigation finding (excluding action-related tokens from DiT attention) is supported empirically but its mechanism remains a hypothesis. Finally, whether the observed absence of saturation persists beyond 100k hours—and whether model scaling becomes valuable again once data scales further—are questions the paper leaves open.

Conclusion

Xiaomi-Robotics-1 provides systematic evidence that LLM-style scaling—massive diverse pre-training data plus automated semantic annotation, followed by cross-embodiment post-training—yields measurable, monotonic improvements in real-robot manipulation, both out-of-the-box in unseen environments and under low-data fine-tuning. Its strongest quantitative results (RoboCasa365 composite-unseen generalization, sub-10-hour adaptation to dexterous tasks) support the paper's thesis that data volume, not model capacity, is currently the binding constraint for generalist robot policies.