Wall-OSS-0.5: Open VLA for Robot Control

• The paper demonstrates that pretraining yields executable robot behavior by directly evaluating a 4B-parameter VLA model on physical tasks without task-specific adaptation.
• Wall-OSS-0.5 is a 4B-parameter system built on a 3B VLM backbone augmented with discrete action-token prediction and continuous flow matching to bridge control gradients.
• The model shows strong zero-shot and fine-tuning performance on diverse real-robot tasks, evidencing enhanced robotic competence and deployment efficiency.

Searching arXiv for the primary Wall-OSS-0.5 report and closely related VLA pretraining context. Wall-OSS-0.5 is an open-source Vision-Language-Action (VLA) model for robot control, presented as a 4B-parameter system built from the 3B-parameter vision-LLM backbone Qwen2.5-VL-3B-Instruct plus action-generation components (Yu et al., 29 May 2026). It was introduced to address a specific question in large-scale VLA research: whether pretraining itself yields executable robot behavior, or whether it mainly provides a better initialization for downstream fine-tuning. The technical report argues for the former position by directly evaluating the pretrained checkpoint on physical robots before task-specific adaptation, reporting non-trivial zero-shot real-robot behavior and subsequent gains after fine-tuning (Yu et al., 29 May 2026).

1. Definition and research setting

Wall-OSS-0.5 is described as an open-source 4B VLA built upon a 3B VLM backbone augmented with action-generation components, with the original Qwen2.5-VL-3B-Instruct retained as the model’s VL Expert and added modules expanding the system to over 4B parameters (Yu et al., 29 May 2026). The work is organized around a deployment-oriented criterion: the pretrained checkpoint is tested directly on physical robots before any task-specific adaptation. This distinguishes it from prior VLA reporting patterns summarized in the paper, where evidence for pretrained VLAs is almost invariably reported after task-specific fine-tuning (Yu et al., 29 May 2026).

The central empirical claim is that pretraining yields directly measurable robotic competence. According to the report, the pretrained model already exhibits non-trivial zero-shot real-robot behavior, including on a held-out deformable manipulation task, and the same checkpoint later serves as a stronger fine-tuning prior (Yu et al., 29 May 2026). This suggests a shift in how VLA pretraining may be evaluated: not only as initialization quality, but as a directly testable source of robot capability.

Three points are emphasized as defining features of the system. First, the work reports direct zero-shot physical-robot execution from the pretrained checkpoint rather than only post-fine-tuning performance. Second, it is explicitly presented as open-source. Third, it advances a training view in which discrete action prediction is used primarily as a gradient bridge into the pretrained VLM backbone, while continuous flow matching remains the deployment interface (Yu et al., 29 May 2026).

2. Architecture and action interfaces

Wall-OSS-0.5 extends Qwen2.5-VL-3B-Instruct with a Mixture-of-Transformers (MoT) backbone comprising a VL Expert and an Action Expert (Yu et al., 29 May 2026). The routing scheme is functional rather than a sparse-MoE capacity device. Vision tokens, language tokens, proprioception tokens, and discrete action tokens are routed through the VL Expert, whereas noisy continuous action tokens are routed through the Action Expert (Yu et al., 29 May 2026). The two experts share sequence-level attention context, so gradients can flow end-to-end, including from flow matching back into the VL backbone through shared attention. At the same time, the attention mask makes discrete and continuous action tokens mutually invisible in the forward pass, allowing the two pathways to be trained and used independently (Yu et al., 29 May 2026).

The discrete action pathway uses a Vision-Aligned Residual Vector Quantization Action Tokenizer. This replaces rule-based FAST tokenization with a learned tokenizer operating in delta-action space using an Encoder–RVQ–Decoder design (Yu et al., 29 May 2026). The encoder compresses observation-conditioned action chunks with temporal cross-attention, the RVQ codebooks represent coarse-to-fine motion residuals, and the decoder reconstructs action sequences conditioned on observations. The token space is shaped by three objectives: visual-action alignment, next-frame prediction, and DCT-domain reconstruction to suppress high-frequency jitter (Yu et al., 29 May 2026). In the report’s interpretation, this discrete next-token interface is the most native way to inject control-relevant supervision into a pretrained VLM backbone.

The deployment-time interface is instead continuous flow matching. The model predicts continuous action chunks using multi-step denoising from a noisy action chunk toward the clean action along the probability path

$$\mathbf{A}_t^\tau = \tau \mathbf{A}_t + (1-\tau)\epsilon, \quad \epsilon \sim \mathcal{N}(\mathbf{0}, \mathbf{I}),$$

with timestep sampling

$$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$

The report defines Action-Space Supervision via

$$\hat{A} = A^\tau + (1-\tau)\cdot f_\theta(A^\tau, \tau),$$

$$\mathcal{L}_A = \mathbb{E}_{\tau, \epsilon}\left[\left\|\hat{A} - A\right\|^2\right],$$

and notes that this is equivalent to a $(1-\tau)^2$-weighted velocity loss:

$$\mathcal{L}_A = \mathbb{E}_{\tau, \epsilon}\left[(1-\tau)^2\left\|f_\theta(A^\tau, \tau) - (A-\epsilon)\right\|^2\right].$$

The paper argues that this weighting emphasizes high-noise timesteps that are especially important for low-dimensional smooth robot trajectories because those timesteps determine the global trajectory shape (Yu et al., 29 May 2026).

The total training objective combines continuous flow matching, discrete action-token cross-entropy, and multimodal next-token cross-entropy:

$\mathcal{L} = \mathcal{L}_{\text{flow} + \lambda_{\text{act} \cdot \mathcal{L}_{\text{act-CE} + \lambda_{\text{mm} \cdot \mathcal{L}_{\text{mm-CE},$

with $\lambda_{\text{act}=\lambda_{\text{mm}=0.01$ as described in the report’s prose, despite typographic corruption in the PDF excerpt (Yu et al., 29 May 2026). The paper states that $\mathcal{L}_{\text{flow}$ is roughly two orders of magnitude smaller than the CE terms under Action-Space Supervision, motivating the 0.01 weighting for the CE losses (Yu et al., 29 May 2026).

The model uses a VLM-style conversation sequence with embodiment prompt, image tokens, instruction, proprioception tokens, and then action tokens on the assistant side (Yu et al., 29 May 2026). It supports an arbitrary number of camera views, samples either goal-level or step-level instruction granularity during training, uses language paraphrases to reduce overfitting, and converts proprioception into text-numerical tokens that are randomly dropped or perturbed during training (Yu et al., 29 May 2026).

The action space is 26-dimensional. For each arm, it includes relative 3D position, relative 6D rotation, and 1D gripper state, giving 20D total across two arms; additional channels are 3D mobile base velocity, 1D lift height, and 2D head actuation (Yu et al., 29 May 2026). The model predicts relative actions and uses 6D rotations rather than Euler angles or quaternions. Both discrete and continuous pathways predict a one-second action horizon, with frame count adjusted to each data source’s control frequency (Yu et al., 29 May 2026).

3. Training recipe and data mixture

The methodological core is gradient-bridged co-training, in which three objectives play distinct and complementary roles (Yu et al., 29 May 2026). Discrete action prediction provides a strong VLM-native gradient signal to the backbone because it uses standard autoregressive next-token cross-entropy. Multimodal next-token prediction preserves instruction following, visual grounding, and embodied scene understanding. Continuous flow matching trains the Action Expert for the deployment-time continuous action interface (Yu et al., 29 May 2026).

The report argues that continuous flow matching alone does not strongly shape the large pretrained VLM backbone. Beyond early training, flow matching contributes only a small but persistent approximately 5% share of the backbone update, whereas the dominant backbone updates come from the two CE losses (Yu et al., 29 May 2026). In this account, discrete action-token prediction is the bridge: it is aligned with control while still matching the autoregressive token interface that the backbone already understands well. The paper also states that discrete gradients are positively aligned with flow-matching gradients, while multimodal CE is largely orthogonal and stabilizing (Yu et al., 29 May 2026).

Pretraining uses a three-source mixture: self-collected robot manipulation data, curated open-source multi-embodiment robot datasets, and a large multimodal corpus (Yu et al., 29 May 2026). The model is pretrained across more than 20 embodiments and processes over one million robot trajectories per epoch. After sampling, one epoch contains approximately 60% self-collected and approximately 40% open-source robot trajectories (Yu et al., 29 May 2026).

The self-collected corpus includes tabletop bimanual systems, mobile manipulators, and an embodiment-free collection device, XRZero-G0 (Yu et al., 29 May 2026). It covers thousands of distinct tasks across household, industrial, office, and controlled collection-room settings, with trajectory labels for manipulation complexity, trajectory duration, and special attributes such as spatial reasoning or deformable interaction (Yu et al., 29 May 2026). Each trajectory is annotated with both brief goal-level instructions and detailed stepwise instructions, and LLM paraphrasing is used to expand linguistic coverage (Yu et al., 29 May 2026).

The open-source robot component retains 10 subsets after curation: RoboMIND v1, RoboMIND v2.0, AgiBotWorld Beta, RoboCOIN, RoboChallenge, Galaxea Open-World, RealOmin, DROID, BRIDGE v2, and Fractal/Google Robot (Yu et al., 29 May 2026). The multimodal corpus totals approximately 90 million samples, comprising 78M open-source multimodal samples and 12M embodied bridge samples (Yu et al., 29 May 2026). These are grouped into general vision-language data, embodied perception data, and embodied cognition data. Examples named in the report include COCO, VQAv2, CapsFusion, Cambrian, PixMo-Cap, PixMo-Point, RoboPoint, SpaceThinker, OpenSpaces, SpaceOm, RefSpatial, CrossPoint, SenseNova-SI, Robo2VLM, EO-Data, RoboVQA, and Cosmos-Reason1 (Yu et al., 29 May 2026).

A substantial preprocessing pipeline standardizes all robot sources into a common semantic schema with fields for bimanual end-effector poses, joint positions, gripper states, mobile base motion, lift or waist actuation, and head motion (Yu et al., 29 May 2026). If only joint states are provided, end-effector poses are recovered through forward kinematics from URDF. Canonicalized conventions include $x$ forward, $$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$0 left, $$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$1 upward, zero rotation as a forward-facing gripper with horizontal opening, and larger gripper values indicating wider opening (Yu et al., 29 May 2026). Euler-angle annotations are converted to 6D rotations. Video and state/action streams are synchronized by timestamp using nearest-timestamp pairing where necessary, anomalous trajectories are repaired or discarded, and near-stationary frames are removed to reduce supervision noise and redundant pauses at inference (Yu et al., 29 May 2026).

Sampling across sources and tasks uses power sampling:

$$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$2

with $$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$3, i.e. square-root sampling (Yu et al., 29 May 2026). This is one of the few explicit appearances of 0.5 in the report, but here it denotes the exponent in the data-balancing scheme rather than a model version parameter.

Optimization uses an effective global batch size of 8192, bf16 mixed precision, gradient clipping 1.0, cosine learning-rate schedule with linear warmup, and peak learning rate $$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$4 (Yu et al., 29 May 2026). Muon is applied to 2D parameters of each expert, while AdamW handles visual embeddings and the LM head. The report states that DMuon reduces optimizer overhead from roughly $$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$5 forward+backward time in a naive implementation to about $$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$6, roughly a 100× reduction (Yu et al., 29 May 2026). Fine-tuning uses learning rate $$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$7, keeps all modules trainable, and preserves the same joint discrete-plus-continuous objective as pretraining (Yu et al., 29 May 2026). Images are resized with aspect ratio preserved and long side fixed at 448 px (Yu et al., 29 May 2026).

4. Evaluation protocols and embodied benchmarks

The evaluation emphasizes direct physical-robot testing. For zero-shot assessment, the pretrained checkpoint is evaluated on a 17-task real-robot suite comprising 12 seen tasks from within the pretraining distribution and 5 unseen tasks that are held-out task configurations not collected as identical tasks on the current embodiment (Yu et al., 29 May 2026). The task dimensions are semantic understanding, rigid-object manipulation, deformable-object manipulation, fine-grained manipulation, and long-horizon multi-step manipulation (Yu et al., 29 May 2026). Each task is evaluated over 10 trajectories.

The 12 seen tasks listed in the appendix are Block Sorting, Fruit Sorting, Number Ordering, Switch Pressing, Ring Stacking, Cup Grasping, Towel Folding, Table Setting, Paper Shredding, Charger Plugging, Flower Arranging, and Package Sorting (Yu et al., 29 May 2026). The 5 unseen tasks are Toy Basket Placement, Rope Tightening, Bean Pouring, Table Wiping, and Pot Lid Covering (Yu et al., 29 May 2026). The held-out deformable task Rope Tightening is particularly emphasized in the report because it tests generalization beyond exact task memorization (Yu et al., 29 May 2026).

For adaptation performance, the authors evaluate on 15 real-robot tasks under identical fine-tuning data and protocol across models, with approximately 500 demonstration trajectories per task (Yu et al., 29 May 2026). These are divided into 10 manipulation tasks and 5 reasoning tasks. The manipulation tasks are Color Block Sorting, Ring Stacking, Spoon-in-Bowl, Object-to-Basket, Glasses Rack, Cup Triangle, Drawer Organization, Power Cord, Water Pouring, and Pencil Case; the reasoning tasks are Fruit Basket, Earphone Sorting, Object Matching, Shape Sorting, and Sequential Button Press (Yu et al., 29 May 2026).

The principal reported metric is task progress rather than binary success. The appendix defines a unified step-wise protocol in which each task has a maximum score of 10, points are awarded for milestone substeps, and

$$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$8

Each task is again evaluated over 10 trajectories (Yu et al., 29 May 2026). The report argues that this metric better captures foundational capability because a zero-shot pretrained model may show partial competence even when it does not fully complete the task (Yu et al., 29 May 2026).

The system is also evaluated on five multimodal benchmarks against its backbone Qwen2.5-VL-3B. These are RealWorld VQA, ERQA, EO-Bench, Embodied Grounding, and Where2Place (Yu et al., 29 May 2026). Here the purpose is to examine whether action training erodes general grounded vision-language competence or instead preserves and redirects it toward robot-relevant grounding.

5. Empirical performance

The strongest empirical claim is that the pretrained checkpoint alone already produces non-trivial physical-robot behavior (Yu et al., 29 May 2026). Across checkpoints from 50k to 400k, average task progress on the 17-task zero-shot suite rises as follows:

Checkpoint	Seen avg. (12)	Unseen avg. (5)	Overall avg. (17)
50k	26.1	24.2	25.5
100k	31.7	41.0	34.5
200k	40.1	38.8	39.8
300k	40.4	34.8	38.7
350k	48.1	47.6	47.9
400k	50.0	53.6	51.1

At 400k, the pretrained checkpoint achieves 50.0 average task progress on seen tasks, 53.6 on unseen tasks, and 51.1 overall across all 17 tasks (Yu et al., 29 May 2026). The report highlights six tasks with task progress at or above 60% at this checkpoint: Block Sorting at 100%, Fruit Sorting at 96, Ring Stacking at 86, Rope Tightening at 82, Cup Grasping at 64, and Bean Pouring at 60 (Yu et al., 29 May 2026). Rope Tightening, as an unseen deformable manipulation task, is singled out as evidence against pure memorization (Yu et al., 29 May 2026).

The paper is equally explicit about zero-shot limitations. At 400k, Towel Folding remains at 10, Table Setting at 9, and Charger Plugging at 9 (Yu et al., 29 May 2026). These are characterized as tasks requiring deformable handling, high precision insertion, or long sequential dependencies, and the report states that these remain beyond current zero-shot reach (Yu et al., 29 May 2026).

After fine-tuning, Wall-OSS-0.5 reaches 61.1 on the 10-task manipulation subset, 59.3 on the 5-task reasoning subset, and 60.5 overall on the 15-task benchmark (Yu et al., 29 May 2026). The baseline comparison table reports:

Model	Manipulation (10)	Reasoning (5)	Overall (15)
Wall-OSS-0.5	61.1	59.3	60.5
$$u \sim \mathrm{Beta}(1.5, 1), \qquad \tau = s(1-u), \quad s = 0.999.$$9	35.0	58.9	43.0
DreamZero	33.7	32.7	33.4

Thus Wall-OSS-0.5 outperforms $$\hat{A} = A^\tau + (1-\tau)\cdot f_\theta(A^\tau, \tau),$$0 by 17.5 points overall and DreamZero by 27.1 points overall, with the report also noting that the margin widens to 26 points on the 10-task manipulation subset (Yu et al., 29 May 2026). Per-task examples against $$\hat{A} = A^\tau + (1-\tau)\cdot f_\theta(A^\tau, \tau),$$1 include Color Block Sorting at 96 versus 42, Ring Stacking at 91 versus 60, Spoon-in-Bowl at 80 versus 43, Drawer Organization at 52 versus 7, and Power Cord at 50 versus 21, while Glasses Rack is one task where $$\hat{A} = A^\tau + (1-\tau)\cdot f_\theta(A^\tau, \tau),$$2 performs better at 87 versus 66 (Yu et al., 29 May 2026). Pencil Case Packing remains hard for Wall-OSS-0.5 at 18.5 (Yu et al., 29 May 2026).

The report also studies multi-task fine-tuning scaling. When the model is fine-tuned jointly on 5, 10, and 19 tasks, performance on shared tasks improves rather than degrading, with shared 5-task subset averages reported as 73.96, 74.75, and 83.75, and the shared 10-task subset improving from 59.98 to 64.78 (Yu et al., 29 May 2026). Newly added 9 out-of-distribution tasks under the 19-task configuration average 65.59 (Yu et al., 29 May 2026). The authors interpret this as evidence that broader fine-tuning improves reusable primitives and language-state-action coverage (Yu et al., 29 May 2026).

Multimodal evaluation presents a more nuanced picture. Relative to Qwen2.5-VL-3B, Wall-OSS-0.5 scores 44.2% versus 59.2% on RealWorld VQA, 32.8% versus 38.3% on ERQA, 24.7% versus 20.8% on EO-Bench, 30.8% versus 9.0% on Embodied Grounding, and 15.0% versus 4.0% on Where2Place (Yu et al., 29 May 2026). The report therefore states that general VQA regresses, while embodied grounding and placement reasoning improve strongly, especially with gains of +21.8 on Embodied Grounding and +11.0 on Where2Place (Yu et al., 29 May 2026). This suggests specialization toward robot-relevant perception rather than universal multimodal improvement.

6. Ablations, interpretation, and limitations

The most important ablation concerns training strategy. Four variants are compared, trained from scratch for 70k steps on five real-robot ablation tasks: co-training, stop-gradient, stop-gradient to co-training, and flow-only (Yu et al., 29 May 2026). Their reported scores are 57.0, 31.9, 49.6, and 36.6 respectively (Yu et al., 29 May 2026). The ranking supports the report’s central claim that the full three-objective co-training recipe is necessary: removing the action-token bridge or blocking flow gradients both degrade performance substantially (Yu et al., 29 May 2026).

A second ablation evaluates Action-Space Supervision against a standard velocity-space flow loss in LIBERO simulation (Yu et al., 29 May 2026). The action-space loss reaches peak success 96.5% at 25k, while the velocity-space loss peaks at 90.3% after 35k; the action-space variant also reaches 95.8% at 20k (Yu et al., 29 May 2026). The report interprets this as evidence that action-space supervision improves convergence speed, peak performance, and stability (Yu et al., 29 May 2026).

A tokenizer ablation replaces only the tokenizer under otherwise identical co-training settings. VQA improves from 75.7% to 77.5%, while average task progress on four real-robot tasks improves from 29.3% to 48.1% (Yu et al., 29 May 2026). Because evaluation still uses continuous flow-generated actions rather than discrete decoded tokens, the report treats this as evidence that the tokenizer improves the continuous policy indirectly by providing better gradients during co-training (Yu et al., 29 May 2026).

The evidence that pretraining yields executable behavior is therefore empirical rather than formal. It consists of direct deployment of the pretrained checkpoint on a 17-task physical robot suite, multiple zero-shot tasks above 80, held-out deformable manipulation at 82, and parallel improvement of seen and unseen progress over training (Yu et al., 29 May 2026). The report is cautious on the meaning of “unseen,” noting that these tasks are not collected as identical tasks on the current embodiment, while semantically related experience may still exist in the broader training corpus (Yu et al., 29 May 2026).

Several limitations are explicitly stated. The gradient-bridge dynamics have only been validated on a 3B VLM backbone. The model uses single-frame image inputs, which likely limits long-horizon memory and state tracking. The tokenizer and training pipeline are tied to a fixed 26D action representation, limiting direct extension to dexterous high-DoF hands. Evaluation still depends on manually designed scoring rubrics. The benchmark does not yet cover multi-robot collaboration, long-duration deployment, or broader open-world settings (Yu et al., 29 May 2026). The report also identifies persistent zero-shot weaknesses on towel folding, table setting, and charger plugging (Yu et al., 29 May 2026).

A common misconception would be to read the model as a general multimodal improvement over its backbone. The paper does not support that reading. Instead, it reports regressions on RealWorld VQA and ERQA alongside gains on embodied grounding and placement reasoning (Yu et al., 29 May 2026). Another possible misconception would be that the discrete action pathway is the deployment mechanism; the report is explicit that it is not decoded into executable actions at inference, and that deployment uses the continuous flow-matching Action Expert (Yu et al., 29 May 2026).

7. Deployment characteristics and open-source status

The technical report places unusual emphasis on deployment efficiency. Standard deployment runs at 15 Hz at high input resolution, and with optimized implementation on a single RTX 5090 and three-view input, the model runs at approximately 21 Hz at 224×224 and approximately 15 Hz at 448×448 using the standard denoising step $$\hat{A} = A^\tau + (1-\tau)\cdot f_\theta(A^\tau, \tau),$$3 (Yu et al., 29 May 2026). Both discrete and continuous pathways predict a one-second action horizon, while training uses long-side 448 px image preprocessing and inference benchmarks are reported at both 224×224 and 448×448 (Yu et al., 29 May 2026).

Inference optimization is reported as a 4× end-to-end speedup over a PyTorch eager baseline, achieved by combining CUDA Graph capture for denoising steps with custom fused CUDA kernels for RoPE, RMSNorm, and elementwise or reduction patterns (Yu et al., 29 May 2026). The report states that fused kernels provide 2–10× speedups over native implementations for those components (Yu et al., 29 May 2026).

Several normalization and representation choices are treated as deployment-relevant: relative action representation, 6D rotations, standardized action semantics across datasets, stationary-frame filtering to avoid idle behavior, and randomly dropped or perturbed proprioception tokens during training (Yu et al., 29 May 2026). These choices are presented as part of the system’s robustness and cross-embodiment portability strategy.

Wall-OSS-0.5 is explicitly presented as open-source (Yu et al., 29 May 2026). The report lists the code repository at https://github.com/X-Square-Robot/wall-x and the project page at https://x2robot.com/en/oss, and states that the authors will continue releasing model weights, training code, and evaluation tools (Yu et al., 29 May 2026). This suggests an intended role not only as a reported benchmark system but as a reproducible research artifact for open VLA development.

In the literature position implied by the report, Wall-OSS-0.5 is best understood as an open VLA that argues for a specific interpretation of pretraining. Pretraining is not treated merely as a better initializer for downstream policy learning; it is presented as an already useful source of executable robot capability that can be measured directly on hardware (Yu et al., 29 May 2026). The broader significance of that claim remains bounded by the paper’s stated limitations, but within those bounds the report offers a concrete and extensively quantified case for zero-shot deployability in large-scale VLA pretraining.