ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning

Abstract: Building general-purpose embodied agents across diverse hardware remains a central challenge in robotics, often framed as the ''one-brain, many-forms'' paradigm. Progress is hindered by fragmented data, inconsistent representations, and misaligned training objectives. We present ABot-M0, a framework that builds a systematic data curation pipeline while jointly optimizing model architecture and training strategies, enabling end-to-end transformation of heterogeneous raw data into unified, efficient representations. From six public datasets, we clean, standardize, and balance samples to construct UniACT-dataset, a large-scale dataset with over 6 million trajectories and 9,500 hours of data, covering diverse robot morphologies and task scenarios. Unified pre-training improves knowledge transfer and generalization across platforms and tasks, supporting general-purpose embodied intelligence. To improve action prediction efficiency and stability, we propose the Action Manifold Hypothesis: effective robot actions lie not in the full high-dimensional space but on a low-dimensional, smooth manifold governed by physical laws and task constraints. Based on this, we introduce Action Manifold Learning (AML), which uses a DiT backbone to predict clean, continuous action sequences directly. This shifts learning from denoising to projection onto feasible manifolds, improving decoding speed and policy stability. ABot-M0 supports modular perception via a dual-stream mechanism that integrates VLM semantics with geometric priors and multi-view inputs from plug-and-play 3D modules such as VGGT and Qwen-Image-Edit, enhancing spatial understanding without modifying the backbone and mitigating standard VLM limitations in 3D reasoning. Experiments show components operate independently with additive benefits. We will release all code and pipelines for reproducibility and future research.

• The paper introduces ABot-M0, a VLA foundation model trained on more than six million standardized trajectories from 20+ robot embodiments using task-uniform sampling and unified delta-action representations.
• The paper proposes Action Manifold Learning, which directly predicts clean action chunks through flow matching and improves robustness as action dimensionality and chunk size increase.
• The paper reports state-of-the-art simulated results, including 98.6% on LIBERO, 80.5% zero-shot on LIBERO-Plus, 58.3% on RoboCasa, and 80.4%/81.2% on RoboTwin 2.0, while real-world transfer remains untested.

ABot-M0 is a vision-language-action (VLA) foundation model for robotic manipulation that couples a large-scale data curation effort with architectural and training-objective changes. The work is organized around three claims: that heterogeneous open-source datasets can be unified into a single pre-training corpus; that action prediction should target clean actions rather than noise or velocity; and that VLM semantic features, augmented by plug-and-play 3D modules, suffice to condition precise manipulation. The model achieves average success rates of 98.6% on LIBERO, 80.5% on LIBERO-Plus (zero-shot), 58.3% on RoboCasa GR1 Tabletop, and 80.4%/81.2% (clean/randomized) on RoboTwin 2.0, outperforming baselines including $\pi_{0.5}$, UniVLA, OpenVLA-OFT, GR00T-N1.6, and X-VLA (2602.11236).

The UniACT-dataset: curation and standardization

The authors aggregate six public datasets—OXE, OXE-AugE, AgiBot-Beta, RoboCOIN, RoboMind, and Galaxea—totaling over seven million raw trajectories, into the UniACT-dataset of more than six million trajectories spanning 9,500+ hours and 20+ embodiments, which they state is the largest collection in the non-private domain. Curation is dataset-specific: OXE provides broad single-arm coverage, OXE-AugE adds morphological diversity, AgiBot-Beta and Galaxea contribute high-quality dual-arm trajectories (with AgiBot-Beta's sampling ratio deliberately reduced to counter embodiment bias), and RoboCOIN/RoboMind supply fine-grained long-horizon decompositions across morphologies.

A multi-stage cleaning pipeline addresses concrete defects documented in the source data: non-English or garbled instructions, frame-instruction misalignment, frame rates as low as 5 FPS in some OXE subsets, visually degraded frames, abnormal action deltas, and ambiguous action annotations (e.g., unspecified rotation representations). Approximately 16% of trajectories are discarded. Notably, the authors argue that neglecting text alignment would reduce the trained model to a vision-action model without language grounding—an explicit warning about silent degradation in naively merged corpora.

Standardization rests on three principles. All actions are converted to delta actions in the end-effector frame with rotations encoded as rotation vectors ($\bm{r} = \theta \bm{k}$), yielding 7-dimensional per-arm vectors $$[\Delta x, \Delta y, \Delta z, \bm{r}, \text{gripper}]$$. A pad-to-dual strategy zero-pads single-arm data so one network handles both configurations, always emitting 14-dimensional dual-arm outputs while activating only relevant channels at execution. Finally, multi-granularity uniform sampling counteracts distributional imbalance (OXE-AugE alone constitutes 67% of volume; the four dual-arm datasets together only ~17.2%).

Action Manifold Learning

The central methodological contribution is the Action Manifold Hypothesis: effective action sequences lie on a low-dimensional, smooth manifold shaped by physics and task constraints, whereas noise- or velocity-prediction targets ($\epsilon$-pred/$v$-pred) are high-dimensional and off-manifold. Extending the manifold hypothesis from perception data to action space, the authors propose Action Manifold Learning (AML): a DiT-based action expert ($V_\theta$, 0.16B parameters) conditioned on VLM features $\phi_t$, robot state $q_t$, and noisy input $A^\tau_t$, directly predicts the clean chunk $\hat{A}_t = V_\theta(\phi_t, A^\tau_t, q_t)$ under flow matching. Although the network predicts actions, the loss is computed on velocity, which equals a reweighted action MSE with weight $\bm{r} = \theta \bm{k}$0—a Jacobian-derived weighting that emphasizes fine refinement at low noise levels. Inference follows an ODE solver trajectory with few denoising steps (4 by default).

The ablation evidence for AML is the paper's most striking result. Under matched conditions (same Qwen3-VL initialization, same 0.16B expert size, no 3D module), ABot-M0 outperforms the noise-predicting GR00T expert by 1.7% at default settings, but the gap widens dramatically as task difficulty grows: with an oversized action chunk of 30, GR00T's LIBERO-Plus success rate collapses by 23.6 points (to 45.7%) while ABot-M0 drops only 8.2 points (to 62.8%). This supports the claim that regressing high-dimensional noise targets consumes model capacity inefficiently, and that the advantage grows with action dimensionality—a property directly relevant to dexterous and whole-body control.

Perception-to-action design

Two empirical findings shape the feature interface. First, after robotics pre-training, final-layer VLM features alone outperform intermediate layers, multi-layer concatenation, and any use of learned action queries; concatenating queries with hidden states performs worst (63.8% vs. 71.0% on LIBERO-Plus). The interpretation is that VLA pre-training already aligns the VLM's representational space with the action domain, making auxiliary adaptation modules redundant. Second, because VLM features remain qualitatively spatial ("the cup is left of the box") rather than metrically grounded, a plug-and-play 3D injection module fuses geometry-aware features via single-layer cross-attention—the best of three tested fusion strategies (concatenation, cross-attention, Q-Former). Two sources are used: VGGT feedforward single-image 3D features, and implicit multi-view features synthesized by lightly fine-tuned Qwen-Image-Edit (only 50 paired samples per dataset). Two synthesized views outperform one, with up to +14 points on LIBERO-Plus's camera perturbation subset. The module is toggleable without backbone modification.

Pre-training sampling strategy

A controlled study compares trajectory-uniform, task-uniform, and embodiment-uniform sampling within the bimanual regime (single-arm fixed at 50%). Trajectory-uniform preserves scale dominance (training dominated by AgiBot-G1); embodiment-uniform improves embodiment balance but concentrates probability mass on high-frequency skills, degrading skill coverage growth. Task-uniform sampling yields a lower Gini coefficient over skills, faster Coverage@T growth, the lowest validation MAE across datasets and most embodiments, and the best downstream transfer to Libero Plus (72.4 vs. 71.3/71.6). The conclusion—that task-level organization provides more transferable supervision than strict embodiment uniformity—is validated across cross-embodiment, cross-dataset, and downstream dimensions.

Benchmark results

On LIBERO, training one model jointly across all suites yields 98.6% average, exceeding X-VLA (98.1%), OpenVLA-OFT (97.1%), and $\bm{r} = \theta \bm{k}$1 (96.9%). On LIBERO-Plus zero-shot evaluation (trained only on standard LIBERO), ABot-M0 reaches 80.5%, 10.9 points above the next best (RIPT-VLA at 68.4%) and 64.9 points above OpenVLA, with particularly strong robustness to lighting (96.2%) and language perturbations (86.4%). On RoboCasa GR1 Tabletop—24 tasks with a 29-dimensional action space covering dual arms, hands, and waist (464 predicted values per chunk)—ABot-M0 attains 58.3% versus 47.6–48.8% for GR00T-N1.6, Qwen3GR00T, and Qwen3OFT variants. On RoboTwin 2.0 multi-task evaluation (50+ tasks, 27,500 demonstrations mixing clean and heavily randomized scenes), it averages 80.42% clean / 81.16% randomized versus 72.80/72.84 for X-VLA and ~43 for $\bm{r} = \theta \bm{k}$2. The randomized-setting result slightly exceeds the clean setting, which the authors do not explicitly discuss but which suggests the heavy randomization in training data transfers effectively.

Limitations and open questions

Several caveats bear directly on the results. All headline evaluations are in simulation; no real-robot deployment results are reported, so sim-to-real transfer of both the AML paradigm and the 3D injection modules remains unverified. The Action Manifold Hypothesis itself is asserted rather than formally characterized—the paper does not measure intrinsic dimensionality of action manifolds or verify smoothness empirically. The AML ablation is confined to LIBERO-Plus-style short-horizon tasks; the claimed advantages at high action dimensionality rest on a single stress test (chunk size 30 on RoboCasa) rather than systematic scaling analysis. The 3D module depends on external models (VGGT, Qwen-Image-Edit) whose own failure modes under domain shift are not analyzed, and the multi-view synthesis was fine-tuned on only 100 paired samples total, raising questions about robustness beyond Bridge/LIBERO-like distributions. Data-wise, the corpus remains dominated by single-arm data (~67% from OXE-AugE), and the authors concede that finer control over task balance and embodiment coverage "requires deeper study." Finally, the framework has not been extended to legged systems, drones, or force/tactile modalities, which the authors identify as prerequisites for genuine cross-morphology generality.

Conclusion

ABot-M0 demonstrates that systematic engineering of public resources—unified delta-action representations, pad-to-dual modeling, task-uniform sampling, direct action prediction via AML, and modular 3D feature injection—yields state-of-the-art simulation performance across four benchmarks without proprietary data. The additive, independently ablated contributions and the sharply widening AML advantage under high-dimensional action prediction constitute the paper's strongest evidence. Whether these gains survive contact with real hardware, and whether the manifold hypothesis holds quantitatively at humanoid-scale action spaces, remain the principal open questions.