RhinoVLA Technical Report

Abstract: Vision-Language-Action (VLA) models have shown strong potential for robotic manipulation, but real-time deployment on edge hardware remains challenging. In this work, we identify VLM visual and context tokens as a major source of deployment latency: for GEMM-dominated projection operators, computation grows linearly with the number of input tokens when model dimensions are fixed. Motivated by this observation, we propose RhinoVLA, a deployment-oriented VLA model co-designed with the Huixi R1 edge SoC. RhinoVLA adopts a token-efficient Qwen3-VL backbone and a continuous Action Expert, reducing the VLM-side token and computation burden while preserving pretrained multimodal capability. To support cross-robot learning, RhinoVLA further introduces a unified interface that combines View Registry, 72D physical state-action slot space, and robotinstance LoRA, allowing heterogeneous robot observations and action schemas to be aligned under a shared policy. On the deployment side, RhinoVLA is optimized through hardware-aware compilation, mixed-precision execution, and parallel visual encoding. Experiments show that RhinoVLA achieves downstream performance comparable to π0.5 at a similar parameter scale, while reaching 11.69 Hz end-to-end inference on Huixi R1, meeting the 10 Hz real-time closedloop control target. The project will be open-sourced at https://github.com/HuixiAI/RhinoVLA.

• The paper details a deployment-oriented VLA model that reduces token burden using a Qwen3-VL backbone for real-time robotic control.
• It employs a unified 72D state-action interface with robot-instance LoRA modules to adapt policies across diverse robot embodiments.
• The system achieves 85.54 ms inference latency and 11.69 Hz closed-loop control, meeting stringent real-time requirements in both simulation and real-robot tests.

RhinoVLA: Deployment-Oriented Vision-Language-Action Model for Real-Time Robotic Control

Introduction

The RhinoVLA technical report presents a deployment-focused Vision-Language-Action (VLA) model addressing real-time inference and cross-embodiment policy learning requirements critical for robotic manipulation. The predominant challenge in VLA deployment is the inference latency arising from the token structure of vision-LLMs (VLMs), particularly the linear scaling of GEMM-dominated operators with the number of visual and context tokens. RhinoVLA incorporates a token-efficient Qwen3-VL backbone and a continuous Action Expert, jointly optimized for real-time performance on edge hardware, specifically the Huixi R1 SoC. A unified interface is designed to align heterogeneous robot observations and action schemas, leveraging a View Registry, 72D physical state-action slot space, and robot-instance LoRA modules for residual adaptation.

Figure 1: RhinoVLA achieves real-time edge control at 11.69 Hz through algorithm–system co-design and enables embodiment-agnostic deployment across diverse robots.

Performance Analysis and Deployment Bottleneck

The computational bottleneck in state-of-the-art VLA models is not merely model scale, but the token burden incurred during inference. End-to-end roofline analysis reveals that models like $\pi_{0.5}$ and RDT approach or exceed the compute limits of NVIDIA Jetson AGX Orin at only 5 Hz target frequencies, and cannot meet the minimum 10 Hz closed-loop control requirements that are standard for real-time robot deployment. Operator-level breakdowns indicate over 90% of inference latency is concentrated in the VLM backbone and Action Expert modules. Specifically, MLP projections (gate_proj, up_proj, down_proj) dominate latency, scaling linearly with token count—a characteristic fundamentally limiting existing VLA architectures regarding edge deployment.

Figure 2: End-to-end roofline analysis of representative VLA models on NVIDIA Jetson AGX Orin under FP16 precision.

Model Architecture and Unified Cross-Robot Interface

RhinoVLA adopts a modular pipeline comprising a Qwen3-VL 2.13B parameter vision-language backbone and a 0.40B parameter Action Expert. Qwen3-VL offers substantial visual-token compression (64 tokens per image, compared to 256 for PaliGemma-224), reducing the main GEMM computational load. The Action Expert is Qwen-compatible, conditions on robot state, action masks, visual-language context, and robot-instance identifier, and predicts masked flow velocities over a unified 72D action slot space.

Cross-robot heterogeneity in camera layouts, action schemas, and embodiment-specific behaviors is addressed by three mechanisms: (A) View Registry explicitly tags each image with camera role/modality, decoupling dataset-specific convention; (B) a unified 72D slot space with binary masks ensures shared physical semantics, with only valid dimensions supervised; (C) robot-instance LoRA modules enable low-cost residual adaptation without separate output heads, preserving shared policy structure and deployment efficiency.

Figure 3: Overview of RhinoVLA. The architecture aligns heterogeneous robot datasets through three interface mechanisms. The Action Expert conditions on Qwen3-VL visual-language memory and predicts masked flow velocities over active action slots.

Training Strategy and Instance LoRA Adaptation

RhinoVLA leverages a corpus of heterogeneous robot demonstrations—each mapped to the unified interface—optimizing three components: VLM LoRA, shared Action Expert, and robot-instance LoRA. Pretraining is stabilized with a masked flow-matching loss supervising valid action slots, and residual regularization constrains LoRA adapters. Dataset sampling follows a power law, ensuring balanced learning across platforms.

Loss diagnostics confirm that robot-instance LoRA modules effectively absorb embodiment-specific residuals, reflected in the distinct loss curves per robot. Correlation analysis between action-mask Hamming distance and LoRA residual similarity validates that adapters encode physically relevant corrections, rather than mere dataset identity.

Figure 4: Pre-training loss diagnostics. The left panel shows the global masked flow-matching loss, while the right panel reports the full objective loss and per-instance losses on AgiBot G1 and AgiBot G2.

Figure 5: Small-scale diagnostic comparing instance-LoRA residual similarity with action-mask distance. The left matrix reports residual correlations between instance adapters on the same probe set, while the right matrix reports Hamming distances between their active action masks.

Deployment Optimization on Huixi R1 Edge SoC

Deployment on the Huixi R1 leverages three major optimizations: hardware-aware compilation with operator, graph, and runtime-level scheduling; mixed-precision quantization (INT8 weights, FP16 activations) with a custom W8A16 GEMM kernel for reduced memory bandwidth; and parallel encoding of visual streams to improve compute occupancy. These optimizations incrementally raise the end-to-end inference rate, culminating in 11.69 Hz closed-loop control, surpassing the real-time requirement.

Figure 6: Cumulative frame-rate improvement of RhinoVLA on Huixi R1. Bars for compilation, mixed precision, and parallel encoding denote incremental gains over the previous configuration.

Policy Performance: Simulation and Real-Robot Evaluation

RhinoVLA achieves strong downstream policy learning with a single checkpoint in the LIBERO benchmark: 90.0% average success rate—surpassing most direct policy and VLA baselines except for large-scale co-trained models like $\pi_{0.5}$. Importantly, the model is initialized solely from Qwen3-VL-2B-Instruct and does not rely on heterogeneous language–robot co-training, establishing the sufficiency of compact VLMs in action policy learning under limited pretraining.

Real-robot tasks on AgiBot G1/G2 and Galbot G1 demonstrate embodiment adaptation via instance LoRA, with robust results in both single-arm and bimanual tasks. On bimanual towel-folding, the model achieves 67% SR in seen settings and maintains competitive performance in unseen settings, validating the effectiveness in both rigid and deformable manipulation.

Figure 7: RhinoVLA performs a bimanual towel-folding task on AgiBot G1, demonstrating robustness in deformable object manipulation.

Inference Efficiency and Hardware Utilization

Post-optimization, RhinoVLA achieves end-to-end inference latency of 85.54 ms, with 11.69 Hz on the Huixi R1, aligning with real-time control thresholds. The efficiency gains result from cumulative improvements across visual encoding (parallel), VLM backbone (token-efficient, custom kernels), and Action Expert (modular, mask-conditioned).

Implications, Limitations, and Future Directions

Practically, RhinoVLA offers a scalable pipeline for deploying VLA models across diverse robot embodiments without compromising policy coherence or real-time responsiveness, unlocked by token-efficient architecture and unified interface design. Theoretically, its modular decomposition and residual adaptation framework provide a template for composable policy learning with cross-embodiment generalization and minimal robot-specific overhead.

The report advances the hypothesis that compact, token-efficient VLM backbones are sufficient for robust action policy learning and real-time deployment, contradicting the necessity of large-scale co-trained semantic models. Further, the unified 72D state-action interface allows for semantically aligned, physically interpretable policy transfer, pushing towards generalizable embodied intelligence.

Moving forward, the extension of training support directly onto the R1 platform would facilitate fully onboard reinforcement learning loops, potentially enabling closed-system online policy improvement. Continued evolution of VLM backbones, e.g., the Qwen-series, is anticipated to further reduce inference overhead while enhancing multimodal reasoning.

Conclusion

RhinoVLA provides a deployment-optimized VLA model co-designed with edge hardware, unifying cross-embodiment policy learning via a shared interface and LoRA-based residual adaptation. The system achieves task performance comparable to $\pi_{0.5}$ and delivers 11.69 Hz real-time inference on Huixi R1, with proven effectiveness across simulation and real-robot settings. Its architectural and deployment innovations point towards scalable, efficient, and generalizable robotic intelligence for diverse platforms (2606.07383).