RoboGen: Scalable Robotics Data Generation

• RoboGen is a suite of frameworks for scalable robot learning, integrating modular hardware design with generative simulation and closed-loop task curation.
• It employs self-guided propose–generate–learn cycles and LLM/VLM-driven reward optimization to enhance diversity and real-world performance.
• Empirical results demonstrate significant gains in task success rates and adaptability, reducing human supervision in robotics dataset generation.

RoboGen denotes a suite of frameworks, agents, and automated pipelines for scalable data generation, skill acquisition, and reward optimization in robotics. Spanning evolutionary hardware design, generative policy learning, and task generation for data-hungry vision-language agents (VLAs), RoboGen systems operationalize compositional robot construction, prompt-driven simulation scene synthesis, and closed-loop agentic curation, with explicit focus on diversity, physical plausibility, and real-world performance. Distinct implementations across the literature share the meta-goal of reducing human supervision and increasing the reliability and coverage of robot learning datasets.

1. Evolutionary and Modular Hardware Design in RoboGen

Early RoboGen frameworks focus on modular hardware architecture and evolutionary optimization. Robot morphologies are represented as connected graphs of modules—Core (C), Brick (B), Horizontal Hinge (A₁), and Vertical Hinge (A₂)—each with specified attachment geometry. Genotypes are encoded as Compositional Pattern Producing Networks (CPPNs) that map discrete grid coordinates to module types and parameters. Morphogenesis proceeds via NEAT, evolving both morphology and central pattern generator (CPG) controllers. Fitness for tasks such as directed locomotion is rigorously measured: $$f = \frac{|d_\parallel|}{L + \epsilon} \cdot \left(\frac{d_\parallel}{\delta+1} - \text{penalty}\right)$$ where $d_\parallel$ is projected displacement, $L$ path-length, and $\delta$ angular deviation (Oud et al., 2022).

Notably, the integration of a prismatic Linear Actuator (LA) module—modeled as a mass–spring–damper with neural activation—demonstrated that, for plain environments, morphological complexity and task performance remain unaffected (p = 0.85). In contrast, introducing rough terrain drives significant increases in branching, limb count, and behavioral sophistication (Wilcoxon p ≪ 0.01 on all morphological metrics), suggesting that environmental diversity, rather than actuator variety, is the main driver of robot morphospace richness in RoboGen-style evolution (Oud et al., 2022).

2. Generative Simulations and Self-Guided Policy Learning

Recent RoboGen frameworks employ self-guided propose–generate–learn cycles to automate the generation of tasks, simulation environments, and supervisory signals (Wang et al., 2023). The pipeline is as follows:

1. Propose: Sample a robot and object, and invoke a LLM to recommend a set of composite tasks $T = \{T_j\}$.
2. Generate: Compose scene configurations by retrieving and verifying assets (Objaverse, vision-LLMs), assign physical parameters, and configure objects’ initial joint states and spatial layouts.
3. Learn: Decompose tasks into subtasks; select learning approaches (RL, Motion Planning, Trajectory Optimization); synthesize reward functions via code generation; and train policies (e.g., SAC, BIT* planners).

In this setup, each subtask may receive the most appropriate learning method—policy gradient, sampling-based planning, or differentiable optimization—with supervision and rewards synthesized by the LLM using API-driven templates. Empirical assessments show this framework produces the most diverse task set by self-BLEU and embedding similarity, improves skill-learning returns versus RL-only baselines, and maintains high scene validity when verified with VLMs. Removal of motion planning or trajectory optimization degrades task success rates significantly (Wang et al., 2023).

3. Closed-Loop Diversity-Driven Task Generation

RoboGene frameworks for large-scale real-world dataset curation implement closed-loop agentic pipelines with the following components (Zhang et al., 18 Feb 2026):

• Diversity-Driven Sampling: Tasks are proposed using least-frequently-used (LFU) sampling across scenario, object, and skill libraries to achieve broad coverage.
• Self-Reflection: Multi-faceted evaluators (physical/kinematic feasibility, novelty, constraint adherence) operate as scoring functions $s_i(T)\in[0,1]$; only tasks passing thresholds are retained. E.g., physical feasibility mandates $s_{\text{phy}}(T)\geq\tau_{\text{phy}}$.
• Human-in-the-Loop Memory: Feedback from real hardware deployments is abstracted into heuristics and stored; these are retrieved and prepended to future generator prompts, providing RAG-based (retrieval-augmented generation) refinement.

Extensive experiments validate RoboGene's impact. With 18,000 real-world robotic trajectories across 1,200 curated tasks, RoboGene achieves superior metrics in clarity (0.9910), type consistency (0.9876), logical validity (0.9899), object coverage (0.6323), skill coverage (0.9152), and physical feasibility (0.9899). In downstream VLA policy training, RoboGene data increases fine-tuning success rates on unseen dual-arm tasks (average 48%) over human (38%) and LLM-only (35–36%) baselines. Scenario diversity approaches uniformity over 8 categories, in stark contrast to conventional LLMs, which collapse to domestic/kitchen contexts (>80%) (Zhang et al., 18 Feb 2026).

4. Automated Reward Evolution and Structured Task Decomposition

In the context of reinforcement learning, RoboGen environments serve as benchmarks for frameworks such as Reward Evolution with Graph-of-Thoughts (RE-GoT) (Yao et al., 19 Sep 2025). The RE-GoT procedure alternates between:

• Task Decomposition: Each RoboGen task is encoded as a text-attributed directed graph $G = (V,E,T_v,T_e)$, with sub-goal nodes and robot action transitions, fully elicited by an LLM and validated by heuristics.
• Bi-Level Optimization: Lower-level RL training under current reward $R_\theta$ produces policy $\pi_{R_\theta}$. Upper-level reward refinement minimizes alignment loss between LLM-inferred sub-goals and VLM-generated rollout analysis: $d_\parallel$0 The iterative process leverages VLM feedback for diagnosing failures and suggesting functional or scalar adjustments, enabling rapid, human-free reward tuning.

Empirically, RE-GoT on RoboGen leads to a mean success-rate improvement of 32.25%, with task-specific gains exceeding +55 points on 2-substep tasks and robust enhancement up to 8 substeps; failure modes reflect irreducible environment limitations rather than algorithmic deficiencies. Ablations confirm the necessity of graph-based decomposition and VLM-informed iteration (Yao et al., 19 Sep 2025).

5. Vision-Language Pipeline Extensions for Synthetic Data Generation

"IGen" introduces a RoboGen-enabled vision pipeline that transforms open-world images into robot manipulation data (Gu et al., 1 Dec 2025). The process starts with monocular 2D-to-3D scene conversion using Metric3Dv2 for depth estimation and Segment-Anything/DINOv2 for object segmentation and feature extraction. Completed objects are generated in 3D by TRELLIS and pose-estimated in SE(3).

A VLM (e.g., Qwen-2.5-VL) is prompted with the scene and a command, decomposing it into sequential spatial subgoals. These are rendered as control functions that chain SE(3) end-effector trajectories, which are motion-planned in simulators such as IsaacSim. The dynamic execution yields paired RGB/depth videos precisely matched to ground-truth joint commands.

IGen achieves superior scene fidelity (PSNR 27.0 vs 17.3, SSIM 0.85 vs 0.68) and downstream policy success; for instance, 1000 synthetic demos yield >75% real-robot success, exceeding policies trained solely on 100 real demos (Gu et al., 1 Dec 2025).

6. Implications, Benchmarks, and Future Directions

RoboGen and its related frameworks now encompass diverse robotic paradigms:

Implementation Context	Main Features	Core Contributions
Evolutionary Hardware (2018–2022)	Modular morphogenesis, CPPN+NEAT, actuator variation	Morphology–task interaction analysis
Generative Simulation (2023–)	LLM-driven task/environment/reward synthesis; propose–generate–learn	Infinite skill/task data, compositionality
Closed-loop Curation (2026–)	LFU sampling, multi-faceted reflection, RAG memory	Balanced, high-coverage datasets; improved VLA generalization
Reward Evolution (RE-GoT)	Bi-level (LLM/VLM) reward optimization	Automated, scalable RL tuning
Vision-Language Synthesis (IGen)	2D→3D recon, open-world image → action mapping	Synthetic datasets with joint-semantics

A plausible implication is the convergence of these streams: fully closed-loop, self-improving RoboGen agents leveraging multi-modal sensory feedback, compositional simulation, and real-world memory. Open challenges include sim-to-real transfer (domain randomization, tactile simulation), the systematization of module parameter exploration, and extending data-generation pipelines to multi-agent and deformable-object scenarios. Current technology has demonstrated that RoboGen-derived datasets yield substantial increases in VLA task generalization and policy robustness under novel conditions (Oud et al., 2022, Wang et al., 2023, Yao et al., 19 Sep 2025, Gu et al., 1 Dec 2025, Zhang et al., 18 Feb 2026).

• (Oud et al., 2022) The Effects of the Environment and Linear Actuators on Robot Morphologies
• (Wang et al., 2023) RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation
• (Yao et al., 19 Sep 2025) Reward Evolution with Graph-of-Thoughts: A Bi-Level LLM Framework for Reinforcement Learning
• (Gu et al., 1 Dec 2025) IGen: Scalable Data Generation for Robot Learning from Open-World Images
• (Zhang et al., 18 Feb 2026) RoboGene: Boosting VLA Pre-training via Diversity-Driven Agentic Framework for Real-World Task Generation