Papers
Topics
Authors
Recent
Search
2000 character limit reached

Rho: A Foundation for Efficiently Adaptable VLA Models

Published 29 Sep 2026 in cs.RO | (2609.38164v1)

Abstract: General-purpose physical AI models must combine broad visual and linguistic capabilities with precise control across robot embodiments and efficient adaptation to downstream tasks. We introduce Rho, a family of open-weights VLA models for bimanual manipulation designed for data-light task adaptation on 3 embodiments representative of dual-arm robots across research labs and the industry -- YAM Box, UR AI Trainer, and FR3 Duo. We systematically ablate Rho's action-expert architecture and training recipe, and show in controlled simulation and physical-robot experiments that embodiment midtraining improves downstream adaptation. The resulting Rho variants for YAM Box, UR AI Trainer, and FR3 Duo match or outperform existing open-weights VLAs and achieve the strongest overall performance across the tasks, embodiments, and baselines evaluated in this report. We further demonstrate the Rho model family's built-in capacity for online adaptation: an internal latent policy learns from corrective feedback to select observation-conditioned noise inputs for the frozen flow-matching action expert. With as few as 15 corrected episodes, adapting this lightweight module enables Rho to handle task situations at the fringe of its offline finetuning distribution. Together, these results position Rho as both a strong general-purpose robotic manipulation model and a practical foundation for adaptation. We release the base Rho model and the embodiment-specific checkpoints to facilitate Rho's deployment in research experiments and practical industrial use cases.

Summary

  • The paper introduces Rho, a foundation model for vision-language-action (VLA) robotics that is designed for efficient adaptation to specific robot configurations, offering a reduction of the required task-specific data through two-stage adaptation.
  • The midtraining stage familiarizes Rho with the kinematics and workspace of a target robot, reducing the amount of needed task-specific data later.
  • Over flow-matching, Delta comparisons showed that transforming the policy output into a continuous rotational representation diminishes discontinuities and improves robot performance: 73.4% mean success in RoboEval

Rho addresses a central bottleneck in vision-language-action (VLA) robotics: a foundation model may possess broad visual and linguistic competence yet remain inefficient to adapt to a particular robot, sensor configuration, action space, and task distribution. The paper’s central design decision is to factor adaptation into two stages. First, embodiment midtraining familiarizes a general VLA with the kinematics, sensing, control conventions, and workspace of a target robot across many tasks. Second, vertical task adaptation specializes that embodiment checkpoint to a particular downstream task. The authors argue, and experimentally support, that this separation reduces the amount of task-specific data required for reliable manipulation.

Rho is released as an open-weights family of approximately 5.2 billion parameters, comprising a 4.68-billion-parameter physically grounded vision-language backbone and a 542-million-parameter flow-matching action expert. The released family includes a general multi-embodiment checkpoint and embodiment-specific variants for three dual-arm platforms: I2^2RT YAM Box, Universal Robots AI Trainer, and Franka FR3 Duo. The evaluation combines controlled simulation studies with physical-robot experiments, covering multi-stage packing, insertion, stacking, object organization, deformable-object manipulation, and articulated-object interaction. The paper also integrates a lightweight online adaptation mechanism, FlowDAgger, that learns from corrective interventions without updating the principal VLA parameters (2609.38164).

Model architecture and physical grounding

Rho initializes its visual-language component from a Phi-family vision-LLM. The backbone contains a causal language decoder with 32 layers and width 3,072, together with a SigLIP 2 NaFlex vision encoder containing approximately 428 million parameters. A two-layer cross-modal projector maps visual features into the decoder embedding space, where image tokens are interleaved with language tokens. The robot policy receives up to three RGB views, a natural-language instruction, and proprioceptive state.

The backbone is physically grounded before and during action learning. In addition to general visual-language data, the authors introduce robotics-relevant grounding and spatial-reasoning data, including RoboPoint, RoboRefer, RefSpatial, object-reference data, pointing data, counting, OCR, and synthetic multi-image difference-detection examples. Approximately 6 million of the final 100 million supervised fine-tuning records belong to the explicitly robotics-relevant categories, while a further 30 million records provide general captioning, perception, relational, and grounding supervision. This grounding stage is intended to improve the representation of affordances, spatial relationships, and object references before the model is exposed to the much smaller robot-demonstration distribution.

The action interface is intentionally embodiment-agnostic. State and action vectors are padded to a maximum width of 32 dimensions, with masks excluding inactive dimensions and invalid temporal positions from the loss. Although the architecture can support joint-space or end-effector actions, delta or absolute targets, and different arm counts, Rho’s principal pretraining representation is a chunk-relative Cartesian end-effector representation. Each arm contributes three translational deltas, a continuous six-dimensional rotation representation, and an absolute gripper command. The use of a continuous six-dimensional rotation representation avoids discontinuities associated with conventional angular parameterizations (2609.38164).

Flow-matching action expert

Rho generates action chunks with a flow-matching transformer rather than autoregressively predicting individual commands. Given a clean action chunk aa, Gaussian noise ϵ\epsilon, and interpolation time tt, the training state is formed by linearly interpolating between noise and the action. The action expert predicts the corresponding velocity field, and inference numerically integrates that field from noise toward an action chunk. The default solver uses 10 explicit Euler steps.

The action stream contains a projected robot-state token followed by noisy-action tokens. Each of the 12 transformer blocks applies action-stream self-attention, cross-attention to the VLM context, and a feed-forward update. The flow timestep modulates the blocks through shared adaLN-Zero maps with block-specific offsets. Grouped-query attention uses 16 query heads and four key/value heads, with head dimension 128. This produces a 2,048-dimensional action representation while limiting the memory cost of attention.

The action expert is conditioned on the hidden sequence from an intermediate VLM layer rather than the final decoder representation. Specifically, Rho extracts the representation after decoder block 14, projects it from width 3,072 to width 2,048, and reuses that projected sequence as the cross-attention context in every action-expert block.

Figure 1

Figure 1: Rho’s flow-matching action expert combines state and noisy-action tokens with repeated cross-attention to an intermediate VLM representation and timestep-dependent adaLN-Zero modulation.

The architectural ablations identify a nontrivial relationship between width, depth, and attention-head dimensionality. On RoboEval, a 1,024-wide expert with eight 128-dimensional heads reaches 0.640 success, compared with 0.617 for the same-width model using sixteen 64-dimensional heads. At width 2,048, sixteen 128-dimensional heads reach 0.673 success, whereas eight 256-dimensional heads reach only 0.623. Increasing the depth of the 1,024-wide model from 16 to 32 blocks produces almost no improvement, reaching 0.642. These results indicate that representational width and head-level geometric resolution matter more than depth in the evaluated regime.

The authors then compress the selected architecture. Sharing adaLN modulation reduces the action expert from 1.649 billion to 882 million parameters without reducing RoboEval success. Grouped-query attention reduces the count to 693 million while retaining 0.672 success, and reducing the depth from 16 to 12 blocks yields a 542-million-parameter expert with 0.671 success. Thus, the final expert removes approximately two thirds of the parameters of the uncompressed wide baseline while preserving its performance.

The intermediate-layer interface is also empirically consequential. Having successive action-expert blocks attend to different VLM hidden states reduces RoboEval success to 0.605, compared with 0.673 when all blocks attend to the same block-14 representation. This result supports the simpler shared-context interface, although it does not establish that intermediate-layer reuse is universally preferable across VLA architectures or data scales.

Figure 2

Figure 2

Figure 2

Figure 2: Rho’s released embodiment-specific checkpoints target three stationary dual-arm platforms spanning laboratory, industrial, and mixed research–industry use.

Multi-embodiment pretraining

Rho-Tomyum combines heterogeneous robot demonstrations with physical-grounding vision-language data. Approximately 90% of the mixture consists of robot or UMI-style demonstrations, and approximately 10% consists of vision-language data. The robot component spans more than a dozen embodiments, with a deliberate emphasis on dual-arm platforms. It includes public datasets, proprietary demonstrations, UMI-style data, and a comparatively small simulated dataset generated for the UR AI Trainer in Isaac Sim.

The authors standardize robot trajectories into approximately one-second action chunks. Per-source, per-dimension, and per-temporal-position percentile normalization accounts for the fact that displacement magnitudes typically increase with temporal offset. Demonstration segments containing labeled erroneous motions are removed from the curated target-embodiment datasets. This data-cleaning choice is important: the reported physical results therefore measure a pipeline using relatively high-quality target-embodiment data, rather than an entirely uncontrolled mixture of successful and failed demonstrations.

Robot updates are sampled with probability 0.9 and vision-language updates with probability 0.1. The vision-language cross-entropy loss is multiplied by 0.02, while the robot flow-matching loss is left unscaled. The backbone uses a peak learning rate of 10−610^{-6}, whereas newly initialized action modules use 10−410^{-4}. The full pretraining run uses 48 NVIDIA B200 GPUs, a global robot batch size of 3,072, and a 45,000-update cosine-decay schedule.

The ablations show that physical-grounding co-training improves downstream adaptation. After 20,000 task-adaptation steps, the robot-plus-vision-language condition reaches 0.963 mean LIBERO success, compared with 0.943 for robot-only pretraining and 0.924 without action pretraining. On RoboEval, the corresponding values at 20,000 steps are 0.623, 0.558, and 0.542. At 40,000 steps, robot-plus-vision-language pretraining reaches 0.673, compared with 0.652 for robot-only pretraining and 0.610 without pretraining.

The gains are uneven across LIBERO suites. Relative to robot-only pretraining, physical-grounding co-training increases success from 0.913 to 0.923 on LIBERO-Spatial, from 0.983 to 0.990 on LIBERO-Object, from 0.947 to 0.977 on LIBERO-Goal, and from 0.927 to 0.960 on LIBERO-Long. The largest improvements occur on Goal and Long, where instruction interpretation and multi-stage temporal structure are more important. The implication is that grounding supervision contributes most when task completion requires semantic-to-behavioral composition rather than simple object recognition.

Embodiment midtraining as a separate adaptation stage

Rho’s main methodological claim is that task adaptation should not simultaneously solve embodiment adaptation. The embodiment checkpoint is trained on a broad, multi-task distribution for one target robot, with the objective of learning its visual input layout, kinematics, action normalization, reachable workspace, and characteristic motion patterns. The authors describe this as horizontal adaptation, in contrast to vertical adaptation to a particular task.

The three physical midtraining mixtures contain approximately 300 hours for YAM Box, 299 hours for UR AI Trainer when physical and simulated data are combined, and 278 hours for FR3 Duo. The mixtures span between 19 and 40 task families, rather than concentrating on one downstream task. The UR AI Trainer checkpoint additionally receives approximately 36 hours of simulation data generated on the same embodiment and camera layout, covering stacking, placement, and insertion under domain randomization.

The controlled RoboTwin experiment isolates the effect of midtraining by reserving 45 tasks for embodiment training and five disjoint tasks for downstream adaptation. The midtrained checkpoint reaches approximately the same success as the pretrained-only checkpoint using half as much task-finetuning data across the tested data regimes. With only 12.5% of the downstream data, the midtrained model is approximately 30% better than the pretrained-only model. The benefit is observed under both Easy and Hard protocols, although it decreases as the downstream dataset approaches its full size.

The physical FR3 Duo experiment provides a complementary compute study. With the same 50,000-step task-adaptation budget for plug insertion, success increases from 60.0% without embodiment midtraining to 76.7% after one midtraining epoch and 86.7% after two epochs. The reported improvements are particularly pronounced for initial plug configurations near the edge of the task-training distribution. This supports the interpretation that midtraining supplies a broader model of target-platform reachability and motion geometry, rather than merely improving memorization of task-specific demonstrations.

These results establish the strongest empirical claim in the paper: embodiment midtraining approximately halves downstream data requirements under the tested protocols. The claim is not equivalent to showing that any amount of robot-specific pretraining yields the same benefit. The effect may depend on the diversity, quality, and embodiment match of the midtraining data; the study evaluates only three physical dual-arm platforms and one controlled simulated setting.

Simulation performance

On RoboEval, task-adapted Rho reaches 73.4% mean success, exceeding the next-best baseline by 6.3 percentage points. It achieves the highest task-level success on three of eight tasks and matches the strongest baseline on the remaining five. The largest advantage occurs on Stack Single Book on Shelf, a task requiring precise contact and alignment.

Rho also performs favorably on behavioral metrics. It obtains the highest task progression and the shortest Cartesian, joint, and orientation path lengths among the compared models. The aggregate results are:

Model Success Task progression Cartesian path length Joint path length
MolmoAct2 0.61 0.79 1.48 7.97
GR00T N1.7 0.61 0.80 1.66 9.64
π0.5\pi_{0.5} 0.67 0.82 1.45 7.98
Rho 0.73 0.85 1.41 7.70

Rho does not dominate every safety-related metric. π0.5\pi_{0.5} records fewer environment collisions, while MolmoAct2 obtains lower self-collision and slip rates. The result is therefore better characterized as a broad performance advantage rather than universal dominance across all behavioral criteria.

On LIBERO, Rho reaches 97.9% average success after offline adaptation followed by latent-policy adaptation. It obtains 95.9% on LIBERO-Long, exceeding the next-best reported value by 2.7 percentage points. On the shorter suites, its scores are within 0.5 points of the best reported values. However, the baseline results are drawn from model reports and official releases with potentially different training and evaluation protocols. The LIBERO comparison is consequently informative but less controlled than the RoboEval comparison.

Physical-robot evaluation

The physical experiments evaluate embodiment-specific Rho checkpoints against π0.5\pi_{0.5} and GR00T N1.7, with MolmoAct2 included in the YAM Box comparison. Evaluations use matched initial configurations and parallel A/B execution to reduce environmental drift and reset variability.

On YAM Box, Rho reaches a 90% mean success rate on the six-category BusyBox benchmark, matching π0.5\pi_{0.5} and exceeding GR00T N1.7 and MolmoAct2. This is a relatively high-data regime with approximately 2,000 demonstrations and short-horizon tasks. The result shows that the Rho checkpoint can reach competitive performance when abundant task-specific supervision is available, but it does not by itself demonstrate a data-efficiency advantage.

The UR AI Trainer experiments are more directly aligned with the paper’s adaptation claim. With approximately 150–160 demonstrations per task, Rho reaches 46.7% success on Toolbox packing, compared with 20.0% for aa0 and 16.7% for GR00T N1.7. On Small-electronics cleanup, Rho reaches 60.0%, compared with 40.0% and 13.3%, respectively. Absolute success remains modest because both tasks are long-horizon and precision-sensitive. On Small-electronics cleanup, Rho’s mean task progress is 82.5%, only slightly above aa1 at 80.8%, indicating that the binary-success advantage partly reflects better completion of the final stages rather than uniformly superior execution throughout the task.

The FR3 Duo results are stronger. Across five task variants, Rho obtains 80.0% mean success and 88.7% mean task progress, compared with 50.0% and 71.3% for GR00T N1.7 and 46.7% and 64.5% for aa2. Rho’s largest observed margins occur on alignment-sensitive forward tasks: it exceeds the strongest baseline by 33.3 percentage points on tumbler stacking and 30.0 points on test-tube racking. It also exceeds the strongest baseline by 20.0 points on plug insertion.

The asymmetry between forward and reverse tasks is informative. Stacking and racking require establishing precise alignment under contact, whereas unstacking and unracking are more tolerant once objects have been separated. Rho’s gains are largest on the former, consistent with the hypothesis that embodiment midtraining improves the model’s geometric and reachability prior. Nevertheless, the physical evaluations use 30 trials per task variant, so the point estimates remain subject to substantial binomial uncertainty; the paper does not provide confidence intervals or formal significance tests for every comparison.

Online adaptation in latent action space

Offline adaptation cannot cover every state encountered after deployment. Rho addresses this limitation with an internal latent policy that predicts the initial condition of the frozen flow-matching action expert. Standard inference samples Gaussian noise; online adaptation replaces this with an observation-conditioned latent policy. The VLM backbone and action expert remain frozen, and only the lightweight latent module is updated.

Corrective action chunks are inverted through the discretized flow to recover latent targets. The latent policy is then trained by regression to those targets. This design constrains online learning to the generator’s existing action manifold. It reduces the risk of catastrophic modification to the pretrained controller, but it also means that online adaptation cannot represent behaviors that the frozen generator cannot produce.

In Meta-World, 50 corrective episodes per task increase mean success from 57.5% to 73.3%. The largest improvement is on Assembly, from 45% to 78%, followed by Stick pull, from 58% to 78%. Pick out of hole remains difficult, improving only from 22% to 37%, while Basketball improves by six points.

The physical FR3 Duo results are more notable because only 15 corrected episodes are used per task. On hard initial configurations for test-tube assembly, success increases from 30.0% to 70.0%, while task progress increases from 56.7% to 81.7%. For plug insertion, success increases from 66.7% to 93.3%, and task progress from 83.3% to 96.7%. Thus, 15 corrected episodes, one tenth of the offline demonstration budget, more than double success on the weaker task and reduce failures from ten to two on plug insertion.

These experiments demonstrate targeted correction of known failure regions rather than unrestricted online generalization. The corrective episodes are collected on the hard configurations used for evaluation, and Meta-World uses a privileged scripted expert. The results therefore establish rapid adaptation under a favorable intervention protocol, while leaving open how well the method selects informative corrections when failure regions are not known in advance.

Limitations and open questions

The paper’s strongest claims are restricted to the evaluated baselines, training budgets, tasks, and robot platforms. The physical study covers three stationary dual-arm embodiments, and the midtraining benefit may depend on the quantity and diversity of target-embodiment data. The release is open-weight, but the main pretraining mixture includes proprietary data, so exact reproduction of the reported checkpoint is not possible from the public resources alone.

Language steerability is not systematically evaluated. The experiments use task instructions, but they do not measure robustness to paraphrase, compositional language variation, ambiguous references, or instruction distributions outside the adaptation datasets. Similarly, the model is primarily evaluated with RGB observations and proprioception. Tactile sensing, despite its likely relevance to contact-rich manipulation, is not incorporated into the reported system.

The online adaptation experiments also rely on corrective supervision that is unusually structured: scripted privileged experts in simulation and human SpaceMouse interventions on selected hard configurations in physical experiments. The latent-policy approach is constrained by the expressivity and support of the frozen action generator, and the paper does not quantify how often flow inversion fails, how sensitive adaptation is to correction timing, or whether latent-policy updates preserve performance on previously solved states.

Finally, comparisons across LIBERO partly rely on published baseline numbers obtained under differing protocols. The controlled RoboEval and matched physical experiments provide stronger evidence, but the claim that Rho is the strongest overall open-weights VLA should be interpreted within the paper’s evaluation envelope rather than as a protocol-independent ranking.

Conclusion

Rho presents a VLA training and adaptation pipeline organized around a practical distinction between embodiment competence and task competence. Its architecture combines a physically grounded 4.68-billion-parameter VLM with a 542-million-parameter flow-matching action expert, while its training recipe uses heterogeneous multi-embodiment data, physical-grounding co-training, and target-platform midtraining.

The principal empirical result is that embodiment midtraining substantially improves downstream data efficiency: in controlled simulation it approximately halves the task-specific data required for comparable success, and on FR3 Duo it raises plug-insertion success from 60.0% to 86.7% as midtraining exposure increases from zero to two epochs. The physical evaluations further show strong performance across three dual-arm platforms, while latent-space corrective adaptation yields large gains from as few as 15 additional episodes.

The paper therefore establishes Rho as an adaptable open-weights VLA family and identifies embodiment-specific midtraining as a consequential intermediate stage between general robot pretraining and downstream task specialization. The remaining technical questions concern the dependence of this benefit on data composition, the robustness of language conditioning, the limits of frozen-generator latent adaptation, and the transfer of the recipe to additional embodiments and sensing modalities.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Explain it Like I'm 14

1. ¿De qué trata el artículo?

El artículo presenta Rho, una familia de modelos de inteligencia artificial diseñados para controlar robots con dos brazos.

Rho combina tres habilidades:

  • Ver lo que ocurre mediante cámaras.
  • Entender instrucciones escritas o habladas.
  • Mover los brazos del robot para realizar tareas.

Por ejemplo, una persona podría decirle al robot: “Recoge la taza y colócala junto al plato”. El modelo intentaría mirar la escena, entender la orden y producir los movimientos necesarios.

El objetivo principal de Rho es que los robots puedan aprender tareas nuevas usando menos ejemplos y menos tiempo de entrenamiento que otros sistemas.

El texto proporcionado parece ser un fragmento del artículo y termina durante una tabla de resultados. Por eso, algunos detalles numéricos de los experimentos finales no aparecen completos.

2. ¿Qué preguntas intenta responder el estudio?

Los investigadores quieren saber principalmente:

  1. ¿Qué tipo de diseño hace que Rho controle mejor un robot?
  2. ¿Ayuda entrenar primero al modelo para un tipo específico de robot?
  3. ¿Puede Rho aprender una tarea nueva con pocos ejemplos?
  4. ¿Puede mejorar después de empezar a trabajar, usando correcciones de una persona?
  5. ¿Funciona Rho en distintos robots, simuladores y situaciones reales?

La idea central es separar el aprendizaje en etapas. Primero, el modelo aprende a controlar un tipo de robot en general. Después, aprende una tarea concreta, como ordenar objetos o insertar un cable.

Es parecido a enseñar a una persona a conducir un automóvil antes de pedirle que aprenda una ruta específica. Si ya sabe conducir, necesitará menos práctica para aprender la ruta.

3. ¿Cómo realizaron la investigación?

El modelo Rho

Rho tiene aproximadamente 5.000 millones de parámetros. Los parámetros son valores internos que el modelo ajusta durante el entrenamiento, como si fueran miles de millones de pequeños controles que le ayudan a reconocer patrones.

El modelo comienza con una parte llamada modelo visión-lenguaje. Esta parte aprende a relacionar imágenes y palabras. Después se le añade una parte especial para producir movimientos del robot, llamada experto de acciones.

Información que recibe el robot

En cada momento, Rho puede recibir:

  • Imágenes de una o varias cámaras.
  • Una instrucción, como “mete el bloque azul en la caja”.
  • Información sobre el estado del robot, como la posición de sus brazos.
  • Datos sobre el movimiento que debería realizar.

El modelo no predice solamente un movimiento. Predice un grupo de movimientos futuros, llamado action chunk. Esto es parecido a planear varios pasos de una coreografía en vez de decidir cada paso justo cuando llega el momento.

Generación de movimientos

Rho utiliza una técnica llamada flow matching. En términos sencillos, el modelo comienza con una especie de movimiento aleatorio y lo va corrigiendo poco a poco hasta convertirlo en una secuencia útil para el robot.

Una analogía sería empezar con un dibujo borroso y mejorarlo gradualmente hasta obtener una imagen clara. En Rho, el resultado final no es una imagen, sino una serie de movimientos para los brazos.

Entrenamiento con muchos robots

Durante el entrenamiento inicial, Rho observa datos de más de una docena de tipos de robots y muchas tareas diferentes. Estos datos incluyen:

  • Robots de uno o dos brazos.
  • Tareas en laboratorios y entornos cotidianos.
  • Ejemplos reales y simulados.
  • Información visual y preguntas sobre imágenes.

Los investigadores llaman a esta mezcla de datos Rho-Tomyum.

También usan datos de robots específicos para crear versiones adaptadas a tres plataformas:

  • YAM Box
  • UR AI Trainer
  • Franka FR3 Duo

Estos robots tienen dos brazos y cámaras, pero no son exactamente iguales. Cada uno tiene distintas posiciones, sensores y formas de moverse.

Adaptación en dos etapas

Rho sigue una estrategia llamada adaptación progresiva regularizada. Sus etapas principales son:

  1. Preentrenamiento general: aprende visión, lenguaje y movimientos de muchos robots.
  2. Entrenamiento del tipo de robot: aprende las cámaras, los controles y la forma de moverse de un robot específico.
  3. Adaptación a una tarea: aprende una tarea concreta con demostraciones adicionales.
  4. Adaptación en línea: después de comenzar a trabajar, puede aprender de correcciones humanas.

Correcciones durante el uso

Rho también tiene una pequeña parte interna llamada política latente. Si el robot se equivoca y una persona corrige su movimiento, esta parte aprende cómo elegir mejor sus movimientos iniciales la próxima vez.

Lo importante es que no se cambia todo el modelo. Se mantiene congelada la mayor parte de Rho y solo se ajusta este pequeño componente. Según el artículo, puede ser útil con tan solo 15 episodios corregidos, es decir, unas 15 demostraciones o intentos guiados.

4. ¿Cuáles son los principales resultados?

El entrenamiento específico para cada robot ayuda

El resultado más importante es que el entrenamiento de adaptación al tipo de robot mejora el aprendizaje de tareas nuevas.

Los modelos que recibieron este entrenamiento específico necesitaron aproximadamente la mitad de los datos para alcanzar un nivel de éxito similar al de los modelos que solo recibieron el entrenamiento general.

Esto significa que, antes de enseñar al robot una tarea concreta, conviene enseñarle cómo funciona su propio cuerpo:

  • Cómo ven sus cámaras.
  • Hasta dónde pueden llegar sus brazos.
  • Cómo funcionan sus pinzas.
  • Qué significan sus controles.

Rho obtuvo buenos resultados frente a otros modelos abiertos

En las pruebas mencionadas, las versiones de Rho alcanzaron o superaron a otros modelos de visión-lenguaje-acción cuyos parámetros están disponibles públicamente.

Los experimentos se realizaron en:

  • Simuladores de robots.
  • Pruebas con el robot YAM Box.
  • Pruebas con el UR AI Trainer.
  • Pruebas con el FR3 Duo.

El artículo afirma que Rho obtuvo el mejor rendimiento general entre los modelos comparados en los conjuntos de tareas, robots y pruebas evaluados.

La arquitectura de las acciones es importante

Los investigadores probaron diferentes tamaños y diseños para el experto de acciones. Descubrieron que no basta con tener un modelo grande: también importa cómo se organiza.

En una tabla de pruebas de RoboEval, algunas configuraciones obtuvieron tasas de éxito distintas. Por ejemplo:

Configuración Tasa media de éxito
1.024 de ancho, 16 cabezas, 16 bloques 61,7 %
1.024 de ancho, 8 cabezas, 16 bloques 64,0 %
1.024 de ancho, 8 cabezas, 32 bloques 64,2 %
2.048 de ancho, 8 cabezas, 16 bloques 62,3 %
2.048 de ancho, 16 cabezas, 16 bloques 67,3 %

En este contexto, la tasa de éxito indica cuántas veces el robot completó correctamente las tareas. La mejor configuración de esa tabla consiguió un éxito medio del 67,3 %.

El robot puede seguir mejorando con correcciones

Después del entrenamiento normal, Rho puede aprender de correcciones humanas. Si el robot encuentra una situación poco común que no aparecía en sus ejemplos iniciales, una persona puede ayudarlo y el modelo puede usar esa información para mejorar.

Este método es importante porque los robots reales suelen encontrarse con problemas inesperados, por ejemplo:

  • Un objeto está ligeramente fuera de lugar.
  • Una cámara ve la escena desde un ángulo diferente.
  • Un objeto se mueve o se dobla.
  • El robot necesita aplicar más o menos fuerza.

5. ¿Por qué son importantes estos resultados?

Entrenar robots suele ser difícil y caro. Se necesitan muchas demostraciones, equipos especiales y bastante tiempo.

Si Rho puede aprender una tarea nueva usando menos datos, entonces podría ser más práctico para:

  • Fábricas.
  • Laboratorios de investigación.
  • Almacenes.
  • Ayuda en tareas domésticas.
  • Manipulación de objetos delicados.
  • Trabajos que requieren dos brazos coordinados.

La adaptación por etapas también podría facilitar el uso del mismo modelo en robots diferentes. En vez de construir un modelo completamente nuevo para cada robot, se podría comenzar con Rho y adaptarlo.

Además, la posibilidad de aprender de correcciones humanas puede ser más sencilla que recopilar miles de demostraciones perfectas. Una persona solo tendría que intervenir cuando el robot comete un error.

6. Posibles limitaciones

Aunque los resultados son prometedores, el artículo no demuestra que Rho pueda realizar cualquier tarea.

Hay varias limitaciones importantes:

  • El robot todavía necesita entrenamiento específico para cada plataforma.
  • Las tareas largas y precisas siguen siendo difíciles.
  • Un buen resultado en un simulador no siempre se repite exactamente en el mundo real.
  • Las pruebas cubren ciertos robots y tareas, pero no todos los posibles escenarios.
  • El fragmento proporcionado no incluye todos los resultados numéricos ni la discusión final completa.
  • El modelo necesita cámaras, datos de entrenamiento y suficiente capacidad informática.

También es importante recordar que una tasa de éxito del 67,3 % significa que el robot todavía falla en una parte considerable de los intentos.

Conclusión

El artículo presenta Rho como un sistema general para enseñar a robots de dos brazos a ver, entender instrucciones y mover objetos.

La idea principal es que el aprendizaje debe dividirse en partes: primero el robot aprende a controlar su cuerpo, después aprende tareas concretas y finalmente puede mejorar con correcciones durante el uso.

Los experimentos indican que este método hace que la adaptación sea más rápida y necesite menos datos. Si estos resultados se confirman en más robots y tareas, Rho podría ayudar a crear robots más flexibles, capaces de aprender nuevas habilidades sin tener que ser programados desde cero cada vez.

Practical Applications

Immediate Applications

  • Data-efficient adaptation of dual-arm industrial robots — Industry / manufacturing Companies can use the released embodiment-specific Rho checkpoints for tasks such as cable routing, component insertion, packing, sorting, tool handling, garment manipulation, and surface preparation. A practical workflow is: collect a small set of demonstrations on the target robot, fine-tune the relevant checkpoint offline, validate it in simulation and controlled cells, and deploy with conservative replanning and safety limits. Potential products/tools: robot-cell programming assistants, reusable VLA controller libraries, and task-specific deployment packages for YAM Box, UR AI Trainer, and FR3 Duo systems. Dependencies: The target platform must have compatible sensors, camera placement, robot-state interfaces, action conventions, and sufficient demonstrations. The reported results do not establish unattended operation in safety-critical production environments.
  • Rapid robot commissioning and reprogramming — Industrial automation / robotics integrators Embodiment midtraining can be used as a reusable intermediate layer between a general robot model and a specific production task. Integrators could maintain a platform-specific checkpoint and adapt it repeatedly for new workstations instead of training each task from a generic model. The paper reports that embodiment midtraining can halve the task-finetuning data needed to reach a given success-rate threshold relative to finetuning the pretrained model alone. Potential workflow: platform calibration → embodiment-specific midtraining → task demonstration collection → offline task adaptation → physical validation. Dependencies: The reported data-efficiency advantage depends on having broad, representative multi-task data for the embodiment and on maintaining compatible normalization and action representations.
  • Human-in-the-loop recovery from robot failures — Manufacturing, logistics, laboratory automation Operators can take over when a robot encounters an unfamiliar configuration and provide corrective action chunks. Rho’s FlowDAgger-style latent policy can then learn from these corrections while keeping the main vision-language backbone and action generator frozen. This supports a “correct once, improve later” workflow for edge cases such as misaligned objects, occlusions, unusual object poses, or partial task failures. Potential products/tools: teleoperation consoles, intervention-recording systems, automatic correction datasets, and deployment dashboards that identify states requiring human takeover. Dependencies: Human corrections must be accurately captured and mapped into the model’s latent action space. Safety supervisors, intervention latency limits, and reliable flow inversion are required.
  • Small-batch and high-mix manufacturing — Manufacturing / electronics / consumer products The combination of reusable embodiment checkpoints and low-data offline adaptation is suitable for production environments in which tasks change frequently or product variants are manufactured in small quantities. Example applications include inserting storage devices, connecting cables, wrapping products, assembling small components, and packing variable objects. Dependencies: Task variation must remain within the model’s visual and kinematic coverage. Performance may degrade for novel materials, reflective objects, severe occlusion, contact dynamics, or tasks requiring force sensing not represented in the training data.
  • Research platforms for bimanual manipulation — Academic and corporate robotics research Researchers can use the open-weight foundation and the three embodiment-specific checkpoints as standardized starting points for studying robot learning, imitation learning, flow-matching control, multimodal perception, online adaptation, and cross-embodiment transfer. The common interface supports up to 32 state/action dimensions and multiple control parameterizations, facilitating controlled comparisons across robot platforms. Potential tools: benchmark implementations, adaptation baselines, reproducible robot-learning pipelines, and shared datasets of corrective interventions. Dependencies: The released code, models, datasets, licenses, hardware interfaces, and computational requirements must be sufficient for independent reproduction. The paper’s evaluation is limited to the reported embodiments, tasks, and baselines.
  • Simulation-based robot training and pre-deployment testing — Robotics software / digital twins Rho can be integrated with simulators such as NVIDIA Isaac Sim for synthetic demonstrations, domain randomization, policy testing, and failure analysis before physical deployment. The paper specifically uses simulated UR AI Trainer trajectories and randomizes object appearance, camera position, and focal length. Potential workflow: generate synthetic trajectories → adapt or validate the policy in simulation → test perturbations and recovery behavior → transfer to the physical robot. Dependencies: Sim-to-real transfer requires sufficiently accurate robot kinematics, camera models, object physics, grasp properties, and workspace layouts. Simulation success should not be treated as evidence of physical safety.
  • Robot-learning curriculum and dataset design — Academia / industrial R&D The paper provides an actionable training recipe: first ground the vision-language backbone in spatial and physical reasoning, then perform multi-embodiment action pretraining, followed by horizontal embodiment midtraining and vertical task adaptation. Organizations can use this structure to organize heterogeneous robot demonstrations rather than directly mixing all data or training each task independently. Dependencies: The effectiveness of the recipe depends on data quality, source balancing, action normalization, clean demonstrations, and the availability of sufficiently broad multi-task data.
  • Standardized evaluation of general-purpose manipulation models — Academia / benchmarking Rho’s architecture and released checkpoints can serve as baselines for experiments in RoboTwin, LIBERO, RoboEval, and physical dual-arm settings. Researchers can compare zero-shot performance, offline adaptation efficiency, correction efficiency, embodiment transfer, and robustness to visual or geometric perturbations. Dependencies: Comparisons should control for model size, training data, compute, task demonstrations, hardware configuration, and evaluation protocols. Reported “strongest overall” performance applies only to the baselines and settings evaluated in the paper.
  • Operator-assistance systems for laboratories and workshops — Laboratory automation / technical education A dual-arm robot equipped with Rho could assist with repetitive handling tasks such as sorting equipment, organizing objects, preparing materials, connecting cables, packing items, or manipulating tools. An operator could specify tasks in natural language and intervene when the robot encounters an unfamiliar state. Dependencies: Natural-language instructions must be unambiguous, workspaces must be constrained, and safeguards must prevent collisions, contamination, tool damage, or handling of hazardous materials.

Long-Term Applications

  • General-purpose warehouse and fulfillment manipulation — Logistics / warehousing A scaled version of Rho could support mixed-item picking, packing, unpacking, cable or strap handling, and coordinated two-arm manipulation of deformable goods. The action-chunking design and online correction mechanism are potentially useful for long-horizon operations in changing warehouse environments. Dependencies: Large-scale deployment requires much broader data coverage, reliable object identification, grasp-force and tactile feedback, robust handling of deformable or fragile items, high uptime, and formal safety validation. The current evidence does not demonstrate warehouse-scale reliability.
  • Adaptive home-assistance robots — Daily life / domestic robotics Future systems could use Rho-like models for folding clothes, organizing household objects, loading appliances, handling packaging, cleaning surfaces, or preparing simple workspaces. Online corrections could allow a household member to guide the robot when object layouts or preferences change. Dependencies: Homes are less structured than the evaluated workspaces. Safe interaction with people, children, pets, fragile objects, liquids, sharp tools, and unknown household layouts requires substantial research in perception, force control, uncertainty estimation, and human-robot interaction.
  • Assistive robotics for older adults and people with disabilities — Healthcare / rehabilitation / accessibility A bimanual robot could perform object retrieval, placement, packaging, dressing-related assistance, or manipulation of daily-use items. Natural-language instructions combined with a corrective-feedback interface could enable personalization to an individual’s environment and motor needs. Dependencies: Medical and assistive applications require clinically validated reliability, fail-safe behavior, privacy protection, explainable intervention policies, and compliance with applicable medical-device and accessibility regulations. The paper does not evaluate physical human assistance or close-contact operation.
  • Flexible industrial “robot-as-a-service” platforms — Manufacturing / robotics services Rho-like embodiment checkpoints could support subscription-based robot cells that are configured for new tasks using customer demonstrations rather than extensive custom engineering. A service provider could maintain a model registry for different robot embodiments and offer task adaptation, simulation validation, monitoring, and periodic online improvement. Dependencies: This requires standardized robot interfaces, fleet-wide calibration, monitoring for distribution shift, versioned safety policies, secure model updates, and reliable performance across factories with different lighting, tooling, layouts, and object distributions.
  • Cross-embodiment transfer to new robot platforms — Robotics research and industry The 32-dimensional padded interface and embodiment-midtraining concept could be extended to new dual-arm robots, mobile manipulators, humanoids, or robots with different joint configurations and sensor suites. A general embodiment adaptation stage could reduce the amount of task-specific data required on each new platform. Dependencies: New embodiments may have substantially different kinematics, reachability, dynamics, end-effectors, latency, or sensing modalities. Transfer will require new data adapters, calibration procedures, action representations, and likely additional architectural or training changes.
  • Closed-loop learning from continuous deployment — Industrial AI / autonomous systems Organizations could build systems that collect successful rollouts, operator corrections, intervention events, and failure cases, then periodically update a lightweight latent policy or task adapter. This could produce a continuously improving robot controller without repeatedly retraining the entire foundation model. Dependencies: Continuous learning introduces risks of catastrophic forgetting, feedback-loop amplification, unsafe exploration, and model drift. It requires rigorous data filtering, rollback mechanisms, validation gates, uncertainty estimation, and human approval of updates.
  • Multimodal robot policies with richer sensing — Robotics / healthcare / hazardous environments Rho’s vision-language-action structure could be extended with depth, tactile, force/torque, audio, or event-camera inputs. Such extensions could improve contact-rich manipulation, insertion, fragile-object handling, and operation under poor lighting or occlusion. Dependencies: The current paper primarily describes RGB observations and proprioceptive state. Additional modalities require new datasets, synchronization, sensor calibration, fusion architectures, and evaluation protocols.
  • Policy-driven quality inspection and rework — Electronics, automotive, and precision manufacturing A future system could combine visual inspection with bimanual corrective manipulation: identify a misplaced component, retrieve the appropriate tool, reposition or reconnect it, and verify the result. Rho’s spatial reasoning and continuous action generation provide a possible foundation for this perception-to-action loop. Dependencies: Inspection and rework require very low defect rates, traceability, force control, precise metrology, and integration with manufacturing execution systems. The current results do not establish the accuracy or certification needed for such workflows.
  • Public-policy and workforce training programs for physical AI — Education, labor policy, and government research Open-weight models and standardized embodiment checkpoints could support shared academic–industry curricula on robot adaptation, teleoperation, dataset collection, safety engineering, and human oversight. Policymakers and funding agencies could use the results to encourage reproducible benchmarks and responsible evaluation of adaptive physical AI systems. Dependencies: Broader adoption depends on licensing, hardware access, compute costs, data governance, worker consultation, cybersecurity, and clear standards for accountability when a continuously adapting robot causes damage or injury.

Glossary

  • AdaLN-Zero: An adaptive layer-normalization method whose modulation and residual gates are initialized to zero for stable conditioning. “The flow timestep conditions the expert through DiT-style adaLN-Zero”
  • Action chunking: Predicting a sequence of future control actions together rather than one action at a time. “The model predicts chunks of future actions rather than individual actions”
  • Action expert: A neural module specialized in generating continuous robot actions from multimodal representations. “The action expert is trained to transform Gaussian noise into continuous robot action chunks”
  • Autoregressive cross-entropy: A loss that trains a model to predict each output token conditioned on preceding tokens. “train the backbone with autoregressive cross-entropy on visual-question-answering or pointing targets”
  • Bimanual manipulation: Robot manipulation that coordinates two arms or end effectors. “Rho's physical dual-arm robot embodiments”
  • Bfloat16: A reduced-precision floating-point format commonly used to accelerate neural-network training. “We train with AdamW in bfloat16”
  • Causal language decoder: A language-model decoder that predicts tokens using only preceding context. “a Phi-family causal language decoder”
  • Chunk-relative delta end-effector representation: An action representation encoding future end-effector displacement relative to the current pose. “a common chunk-relative delta end-effector (EEF) representation”
  • Compounding error: The accumulation of small prediction errors over a sequence of sequential decisions. “which can reduce compounding error”
  • Conditional velocity field: A function that specifies how a generated sample should move over time given conditioning information. “the wide action stream gives the flow-matching expert capacity to represent high-fidelity conditional velocity fields”
  • Continuous 6D rotation representation: A six-dimensional numerical encoding of 3D orientation designed to avoid discontinuities. “a relative orientation in the continuous 6D rotation representation”
  • Cross-attention: An attention mechanism in which one sequence attends to representations from another sequence. “cross-attention to the VLM context”
  • Cross-modal projector: A learned transformation that maps representations from one modality into the representation space of another. “it has a two-layer cross-modal projector maps visual features into the language embedding space”
  • Delta end-effector control: Robot control that specifies changes in an end effector’s position or orientation rather than absolute poses. “a common chunk-relative delta end-effector (EEF) representation”
  • DiT: A diffusion-transformer architecture that uses transformer blocks for generative modeling. “DiT-style adaLN-Zero”
  • Domain randomization: Training with randomized environmental or visual parameters to improve robustness to variation. “all simulated data incorporates domain randomization across object color, camera positions, and focal lengths”
  • Embodiment: A particular physical robot configuration, including its sensors, morphology, and control interface. “embodiment midtraining improves downstream adaptation”
  • Embodiment midtraining: Intermediate training that adapts a general model to a specific robot platform before task-specific fine-tuning. “The embodiment-midtraining stage is key”
  • End effector: The tool, gripper, or terminal component at the end of a robot arm. “The target contains a three-dimensional translation delta”
  • Explicit Euler steps: Numerical integration steps using the current derivative to approximate the next point in a dynamical system. “the default configuration uses 10 explicit Euler steps”
  • Flow inversion: Recovering the initial latent condition that produces a desired output under a learned flow. “we invert the frozen flow to recover a latent target”
  • Flow matching: A generative-model training method that learns a vector field transporting noise toward target data. “we train the model to predict the velocity ϵ−a\epsilon-a with a mean-squared flow-matching loss”
  • Flow-matching action generator: A model that converts an initial noise or latent state into a robot action trajectory using flow matching. “Let Gθ(o,z)G_\theta(o,z) denote its flow-matching action generator”
  • Foundation model: A broadly pretrained model intended to support adaptation to many downstream tasks. “The Rho family derives from a single foundation”
  • Grouped-query attention: An attention variant in which multiple query heads share key and value heads. “With 12 blocks, shared modulation, and grouped-query attention”
  • Grounding: Connecting language or visual descriptions to specific objects, locations, or regions in perceptual input. “Multi-image grounding, pointing, OCR, and counting data”
  • Latent action space: An internal representation in which actions are encoded as latent variables before being generated. “Online adaptation in latent action space”
  • Latent policy: A policy that predicts internal latent variables used by another model to generate actions. “an optional internal latent policy”
  • Long-horizon task: A task requiring a sequence of actions over an extended duration. “high-dexterity long-horizon tasks”
  • Mean-squared error (MSE): The average squared difference between predicted and target values. “velocity-prediction MSE”
  • Mid-fusion: A multimodal architecture that integrates visual features into language processing at an intermediate stage. “Following the mid-fusion design studied in Phi-4-reasoning-vision-15B”
  • Multimodal: Involving and integrating multiple data modalities, such as images, language, and robot state. “The observation sequence combines one or more camera views with the task instruction and proprioceptive state”
  • NaFlex: A vision-encoder design that preserves image aspect ratio while varying the number of visual tokens. “The NaFlex encoder preserves image aspect ratio”
  • Offline adaptation: Model adaptation performed using a fixed dataset rather than live interaction. “Offline adaptation fine-tunes either pretrained Rho”
  • Online adaptation: Model adaptation performed during deployment through interaction and feedback. “Rho's built-in capacity for online adaptation”
  • Open-weights model: A model whose trained parameters are publicly released for reuse or further adaptation. “a family of open-weights VLA models”
  • Proprioceptive state: Internal measurements of a robot’s configuration, such as joint positions, velocities, or gripper states. “Proprioceptive state is kept outside this sequence”
  • Regularized progressive adaptation: A staged adaptation procedure designed to acquire new capabilities while preserving previously learned ones. “Each of these Rho variants is built via a recipe we call regularized progressive adaptation”
  • Rollout: A sequence of actions generated or executed by a policy in an environment. “We evaluate all eight RoboEval tasks using 100 rollouts per task”
  • Self-attention: An attention mechanism in which elements of a sequence attend to other elements within that same sequence. “Each block applies action-stream self-attention”
  • Soft token: A continuous learned embedding inserted into a token sequence in place of a discrete vocabulary token. “The projected visual ``soft tokens'' are interleaved with the text-token embeddings”
  • Spatial affordance: A location or geometric property indicating how an object or region can be interacted with. “RoboPoint supplies spatial-affordance pointing examples”
  • Synthetic trajectory: A simulated sequence of observations and actions generated computationally rather than collected from a physical robot. “It also contains a smaller set of simulated trajectories”
  • Task adaptation: Specializing a general or embodiment-specific model for a particular task. “The embodiment stage adapts the Rho foundation to a target robot type horizontally across many tasks and motions”
  • Temporal offset: The time difference between a reference point and a predicted or observed event. “chunk-relative displacement magnitudes generally grow with temporal offset”
  • Token embedding: A learned vector representation associated with a discrete input token. “The language token-embedding table remains frozen”
  • Vision-language-action (VLA) model: A model that jointly processes visual and linguistic inputs to produce robot actions. “We introduce Rho, a family of open-weights VLA models”
  • Vision-LLM (VLM): A model trained to represent or reason over both images and language. “Rho begins from a vision-LLM developed within the Microsoft Phi model family”
  • Visual question answering (VQA): A task in which a model answers natural-language questions about visual inputs. “a smaller set of vision-language data”
  • Zero-shot: Performing a task without task-specific training examples. “zero-shot success rates of existing VLA and WAMs on unseen manipulation tasks”

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Tweets

Sign up for free to view the 2 tweets with 107 likes about this paper.