StarVLA: Modular VLA Framework

• StarVLA is an open-source framework that modularly decomposes vision, language, and action into backbone and action head components for standardized evaluation.
• It enables reproducible experiments via reusable training strategies, cross-embodiment data mixing, and a consistent server–client API for both simulation and real-robot control.
• StarVLA-α serves as a simplified baseline to isolate design choices, assessing the effects of action modeling and pretraining across varied robotics benchmarks.

StarVLA is an open-source codebase for Vision-Language-Action research that addresses fragmentation across architectures, codebases, and evaluation protocols by organizing VLA development around a modular backbone–action-head abstraction, reusable training strategies, and a unified evaluation interface spanning simulation and real-robot deployment (Community, 6 Apr 2026). Within the same project line, StarVLA-$\alpha$ was introduced as a deliberately simplified baseline for studying VLA design choices under controlled conditions, using the same codebase and pipeline to test action modeling, robot-specific pretraining, and interface engineering with minimal architectural confounders (Ye et al., 13 Apr 2026).

1. Scope, motivation, and research role

StarVLA is positioned in the VLA regime, where a policy must integrate perception, language understanding, and action. Its stated target is principled comparison and reproducibility in a field where methods vary substantially in architectures, training data, embodiment configurations, and benchmark-specific engineering. To that end, the framework provides a modular backbone–action-head architecture, reusable training recipes such as supervised fine-tuning, multimodal co-training, and cross-embodiment mixtures, and a thin server–client evaluation API covering LIBERO, SimplerEnv, RoboTwin 2.0, RoboCasa-GR1, and BEHAVIOR-1K (Community, 6 Apr 2026).

The framework is explicitly deployment-driven. The same policy interface is used for both benchmark execution and real-robot control, with the model hosted as a lightweight WebSocket server and benchmark or robot-side components acting as clients. This design makes the abstraction operational rather than merely software-organizational: the model is trained and evaluated through the same predict_action contract, and no model code changes are required when switching from simulation to physical deployment (Community, 6 Apr 2026).

StarVLA-$\alpha$ narrows the scope from infrastructure to controlled scientific analysis. Its purpose is not to enlarge the design space, but to reduce complexity and re-evaluate several major axes—action modeling strategies, robot-specific pretraining, and interface engineering—under the same backbone, data, and hyperparameters. This suggests that the StarVLA project functions both as a software framework and as an experimental methodology for disentangling architectural choices from benchmark-specific implementation effects (Ye et al., 13 Apr 2026).

2. Backbone–action-head abstraction

The central StarVLA abstraction is a factorization between representation learning and action decoding. Every StarVLA policy exposes two methods with an identical I/O contract: forward(inputs) → {action_loss, aux_losses…} and predict_action(inputs) → {normalized_actions,…}. Internally, the policy is decomposed into a backbone $B$ that maps observations and language into hidden states, and an action head $H$ that maps those hidden states to an action chunk (Community, 6 Apr 2026):

$$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$

$H:h_t \longmapsto \widehat{\mathbf{a}}_{t:t+k}$

At inference time, the framework writes

$$h_t = B(x_{\le t}, \ell), \qquad \widehat a_{t:t+k} = H(h_t).$$

Because $B$ and $H$ communicate only through $h_t$, StarVLA supports both VLM backbones such as Qwen-VL and world-model backbones such as Cosmos-Predict2 under the same abstraction, and either component can be swapped independently in YAML configuration without code changes (Community, 6 Apr 2026).

Action head	Decoding paradigm	Description in StarVLA
FAST	Discrete autoregression	autoregressive discrete tokens
OFT	Parallel regression	parallel regression via MLP
$\alpha$0	Continuous generative decoding	flow-matching continuous denoising
GR00T	Hybrid reasoning–control	dual-system reasoning + flow

This decomposition is more than a software convenience. It makes VLM-based and world-model-based policies commensurable at the level of interfaces, losses, and evaluation hooks. A plausible implication is that StarVLA tries to standardize the VLA design space around a small number of orthogonal decisions: representation backbone, action-decoding paradigm, data mixture, and deployment target.

3. Training strategies, extension points, and evaluation interface

StarVLA treats training regimes as configurations over a shared objective,

$\alpha$1

where $\alpha$2 supervises predicted actions and $\alpha$3 may include VLM loss on web data, world-model loss, or language planning losses. One reusable recipe is multimodal co-training through a dual-loader loop: a forward pass on robot action data produces $\alpha$4, a forward pass on VLM web data produces $\alpha$5, and optimization uses the weighted sum

$\alpha$6

The coefficient $\alpha$7 is configured through config.loss_scale.vlm; the paper gives $\alpha$8 as an example for balancing robot and web gradients (Community, 6 Apr 2026).

A second reusable recipe is cross-embodiment learning through datasets.vla_data.data_mix, where LeRobotMixtureDataset uniformly draws trajectories from heterogeneous robot datasets such as LIBERO, RoboCasa-GR1, and RoboTwin 2.0 with specified weights. The same action loss is then applied across embodiments, enabling joint pre-training under a single pipeline (Community, 6 Apr 2026).

The framework is also designed for extensibility. New VLM or world-model backbones are added by subclassing BaseVLBackbone or BaseWorldModelBackbone in starvla/backbones/base.py, implementing forward(raw_inputs) → hidden_states, and registering the module in starvla/backbones/registry.py. New action heads subclass ActionHeadBase in starvla/action_heads/base.py, implement forward(hidden_states, action_gt) → loss_dict and predict(hidden_states) → normalized_actions, and are registered analogously (Community, 6 Apr 2026).

The evaluation path mirrors the training abstraction. StarVLA hosts a trained policy as a WebSocket server, while each benchmark or robot controller serializes observations as a client and invokes Framework.predict_action(). Because normalized actions are returned through the same API in all settings, the benchmark boundary becomes external to the model. In practice, this is how the framework unifies LIBERO, SimplerEnv, RoboTwin 2.0, RoboCasa-GR1, BEHAVIOR-1K, and real-robot deployment under one interface (Community, 6 Apr 2026).

4. Reported performance of the StarVLA framework

The framework paper reports that simple, fully reproducible single-benchmark recipes already match or surpass prior methods on multiple benchmarks with both VLM and world-model backbones. On LIBERO, with only 30 K steps of supervised fine-tuning, StarVLA-OFT with Qwen3-VL-4B reaches an average of 96.6, while the table reports 97.1 for OpenVLA-OFT at 175 K steps; other StarVLA heads report 95.4 for FAST, 95.7 for $\alpha$9, and 96.5 for GR00T (Community, 6 Apr 2026). On SimplerEnv WidowX visual-matching tasks, the paper reports averages of 64.6 for StarVLA-OFT and 65.3 for StarVLA-GR00T, compared with 51.3 for CogACT. On RoboCasa-GR1, StarVLA-OFT reports 48.8 average success over 24 tasks, compared with 37.0 for $B$0. On RoboTwin 2.0, jointly trained StarVLA variants reach up to 88.8 under randomization (Community, 6 Apr 2026).

Benchmark	Reported StarVLA result	Comparative figure in the same table
LIBERO	OFT: 96.6 average at 30 K steps	OpenVLA-OFT: 97.1 at 175 K
SimplerEnv WidowX	GR00T: 65.3 average	CogACT: 51.3 average
RoboCasa-GR1	OFT: 48.8 average success	$B$1: 37.0
RoboTwin 2.0	$B$2: 88.8 random	FAST: 83.2 random

These results are presented as evidence that the shared framework is not merely a benchmarking wrapper. The reported numbers are obtained with simple training recipes and minimal data engineering, which the paper uses to argue that reproducibility and competitive performance need not be in tension. This suggests that StarVLA’s main research contribution is infrastructural standardization combined with enough modeling breadth to span representative VLA paradigms (Community, 6 Apr 2026).

5. StarVLA-$B$3: minimal architecture and controlled benchmark analysis

StarVLA-$B$4 adopts a minimal two-stage pipeline: a single RGB frame $B$5, usually resized to $B$6, and a tokenized instruction are fed into Qwen3-VL, which treats image patches and text tokens as one long sequence, prepended with special tokens and appended with a single “action” token. After $B$7 Transformer blocks, the hidden state of the action token is extracted and passed to a two-layer MLP that predicts a chunk of $B$8 continuous control values, with $B$9 typically and action padding used to reach a uniform dimension across robots (Ye et al., 13 Apr 2026).

Its default training objective is a simple mean-squared error loss on continuous actions:

$H$0

For alternative heads, the paper also states the corresponding cross-entropy loss for FAST-style discrete tokenization and the standard flow-matching loss for $H$1-style heads. No additional regularization or auxiliary losses are introduced; weight decay and dropout follow standard AdamW recipes and are not tuned per benchmark (Ye et al., 13 Apr 2026).

Under specialist training, the paper reports the following benchmark averages for StarVLA-$H$2: LIBERO 98.8%, SimplerEnv WidowX 64.6%, Google VM 76.0%, RoboTwin clean 50.3%, RoboTwin +rand 88.3%, and RoboCasa 53.8%. The same table reports $H$3 at 96.9%, 46.9%, 72.7%, 60.2%, 76.8%, and 37.0%, respectively. For the public real-world RoboChallenge benchmark on ARX5, the paper reports $H$4 with average success rate 12.7% and score 27.6, versus StarVLA-$H$5 with success rate 33.6% and score 54.5, i.e. +20.9% absolute success rate (Ye et al., 13 Apr 2026).

The framework is also used to train a single generalist StarVLA-$H$6 model jointly on LIBERO, SimplerEnv, RoboTwin, and RoboCasa with no benchmark-specific fine-tuning. The reported hyperparameters are backbone learning rate $H$7, head learning rate $H$8, batch size 256, total steps 100 000, action dimension pad $H$9, AdamW with weight decay $$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$0, cosine decay to 0, and 64 $$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$1 A100. Under this regime, the paper states that the generalist model matches specialist performance on most benchmarks and surpasses $$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$2 by 3.5% on RoboCasa-GR1, reporting 57.3% versus 53.8% (Ye et al., 13 Apr 2026).

6. Interpretive significance, recurring misconceptions, and open technical lessons

A recurring assumption in VLA research is that strong manipulation performance requires increasingly elaborate action heads, benchmark-specific interface engineering, or broad robot-specific pretraining. StarVLA-$$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$3 explicitly tests these claims. Its ablations report that continuous heads all achieve approximately 98–99% on LIBERO and about 65% on SimplerEnv WidowX, whereas FAST discrete drops to approximately 60%; among continuous heads, the simplest MLP is within 0.1–1% of the best alternatives. The paper’s summary is that continuous regression is crucial, while MLP, diffusion, and flow are close under the controlled setting (Ye et al., 13 Apr 2026).

The same controlled analysis characterizes robot-specific pretraining as a double-edged sword. The reported examples state that in low-data regimes, in-domain pretraining helps—for example, InternData-A1 gives +13% on RoboTwin clean 50$$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$450—but out-of-domain tasks such as RoboCasa can suffer, with performance dropping from 53.8% to approximately 28%. Interface engineering shows the same conditional pattern: adding proprioceptive states, stacking history frames, predicting delta versus absolute actions, or changing coordinate conventions yields +2–10% when demonstrations are far below 1K per task, but beyond approximately 500 demos per task, variants collapse back within $$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$5 of the clean baseline (Ye et al., 13 Apr 2026).

These findings do not eliminate the value of architectural diversity within StarVLA; the framework explicitly supports FAST, OFT, $$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$6, and GR00T, and its empirical tables show that different heads can dominate on different benchmarks. Instead, the controlled results qualify where complexity appears necessary. A plausible implication is that StarVLA’s broader significance lies in separating two questions that are often conflated: whether a method is strong, and whether its strength comes from the core policy design or from benchmark-specific engineering. By turning backbone choice, action head, data mixture, and evaluation target into explicit, swappable components, StarVLA makes that distinction experimentally tractable (Community, 6 Apr 2026).

The project’s trajectory therefore has two complementary meanings in the VLA literature. As a framework, it standardizes model construction, training, evaluation, and extension. As a research program, especially through StarVLA-$$B:\{o^{vis},o^{depth},\ldots\}_{\le t}, \ell \longmapsto h_t \in \mathbb{R}^d$$7, it argues that a strong pretrained VLM backbone, continuous action regression, and large-batch joint training can already produce highly competitive specialist and generalist policies without heavy auxiliary machinery (Ye et al., 13 Apr 2026).