DSPv2: Whole-Body Mobile Manipulation Policy

• The paper introduces DSPv2 as an improved dense policy for whole-body mobile manipulation that fuses 3D geometry with 2D semantic features using a Q-former alignment mechanism.
• It leverages multimodal inputs from multi-view RGB, depth point clouds, and robot state to generate coherent, autoregressive action sequences for a 25-DOF platform.
• Experimental evaluations on the Astribot-S1 demonstrate that DSPv2 achieves robust generalization and inter-component coordination across diverse lighting, spatial, and object variations.

Searching arXiv for DSPv2 and closely related works to ground the article in current papers. DSPv2 is a whole-body mobile manipulation policy for imitation learning that targets three coupled difficulties in household robotics: processing complex multi-view observations, achieving robust generalization across environmental variation, and generating coherent actions for a high-DOF mobile manipulator. It was introduced as “DSPv2: Improved Dense Policy for Effective and Generalizable Whole-body Mobile Manipulation” (Su et al., 19 Sep 2025). The method combines a 3D spatial encoder with a multi-view 2D semantic encoder, aligns them through a Q-former, and feeds the fused representation to an autoregressive Dense Policy head extended to whole-body control. In the reported formulation, the policy operates on an Astribot-S1 platform with 25 DOF and is trained by behavior cloning from task-specific demonstrations (Su et al., 19 Sep 2025).

1. Definition and problem setting

DSPv2 is designed for whole-body mobile manipulation in cluttered, multi-camera environments, where the robot must coordinate base motion, torso, dual arms, and grippers under partial observability and domain shift (Su et al., 19 Sep 2025). The paper frames the central challenge as threefold: effective processing of complex observation, robust generalization across scenes, lighting, and object changes, and generation of coherent whole-body actions.

The policy’s central design principle is to align 3D spatial features with multi-view 2D semantic features. This uses geometry-aware 3D structure from depth while preserving semantically rich 2D representations from a pre-trained visual backbone. The resulting fused features are then used by a Dense Policy head that generates coordinated action sequences across effectors and timesteps (Su et al., 19 Sep 2025).

The target platform is the Astribot-S1, a 25-DOF robot comprising a 3-DOF chassis, 4-DOF torso, dual 7-DOF arms, a 2-DOF head, and two grippers. Control is issued as per-component pose commands relative to the base, while the chassis action is represented as an offset from its previous state (Su et al., 19 Sep 2025). This representation reflects the whole-body setting more directly than arm-only visuomotor policies.

A central claim of the work is that DSPv2 is the first policy-level generalization approach demonstrated on whole-body downstream tasks, with robustness to lighting variation, spatial rearrangements, object color and shape changes, and novel scenes, even with limited task-specific data (Su et al., 19 Sep 2025). This suggests that the contribution is not only architectural but also methodological, positioning generalization as a primary evaluation axis rather than a secondary property.

2. Observation model and multimodal representation

DSPv2 consumes three input categories: multi-view RGB, an uncolored depth point cloud, and robot state (Su et al., 19 Sep 2025). The RGB input comes from four cameras located on the head, torso, and two wrists. The depth point cloud is taken from the head camera and projected into the robot base frame for geometric stability against head pose changes. The state input contains current poses of robot components, with one deliberate omission: the chassis pose is excluded from the input so that navigation must rely on environmental cues rather than memorized self-state shortcuts. During training, the remaining pose inputs are randomly masked with 30% probability to prevent overfitting (Su et al., 19 Sep 2025).

The 3D branch begins by transforming the head point cloud into the base frame using the known extrinsic transform

$$X_{\text{base}} = T_{\text{base,head}} X_{\text{head}}.$$

The paper also gives the camera projection equations

$$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$

with pixel coordinates

$u = f_x x / z + c_x, \qquad v = f_y y / z + c_y.$

After projection, the point cloud is voxelized as

$$v = \left\lfloor \frac{X_{\text{base}} - o}{s} \right\rfloor,$$

where $s$ is the voxel size and $o$ is the grid origin (Su et al., 19 Sep 2025). Sparse convolutions then extract per-voxel features. The sparse 3D encoder produces fewer than 300 voxel-level tokens $F_P$, and each voxel center receives a sine-cosine positional embedding $Pos_P$ (Su et al., 19 Sep 2025).

The 2D branch uses a DINOv2-base backbone fine-tuned with LoRA to produce patch-level features from each RGB view. For $n$ views, this yields $F_I^{v_1-v_n}$, augmented with learnable positional embeddings $$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$0 per patch (Su et al., 19 Sep 2025). The role of LoRA is explicit: it preserves general semantic priors while adapting to task-specific data, which the paper identifies as important for generalization.

The decoupling of 3D geometry and 2D semantics is central. The 3D branch is color-agnostic and stable under visual shifts, while the 2D foundation model contributes semantic robustness to appearance changes. A plausible implication is that this factorization reduces interference between geometric precision and semantic invariance, rather than forcing both burdens onto a single visual representation.

3. Q-former alignment and fused visual features

DSPv2 aligns 3D voxel features and multi-view 2D patch features through a Q-former mechanism (Su et al., 19 Sep 2025). The model maintains 300 learnable query tokens $$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$1, made spatially aware by adding 3D voxel positional embeddings:

$$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$2

These queries attend to the 2D feature maps, using $$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$3 as keys and values. The attention operator is the standard form

$$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$4

The fused visual representation is then defined as

$$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$5

This construction transfers positional structure from the 3D voxel grid to the 2D patch grids and binds semantic evidence to geometric locations (Su et al., 19 Sep 2025).

The paper’s interpretation is explicitly geometry-semantic complementarity. Voxelization preserves spatial detail and is immune to appearance changes, while multi-view 2D foundation features provide semantics robust to lighting, color, and background variation. Cross-view attention selects informative patches for each voxel and mitigates occlusions and view-dependent artifacts, which is reported to support precise localization in tasks such as narrow-tolerance stacking and edge-specific grasping (Su et al., 19 Sep 2025).

Relative to policies that pool or sparsely sample point clouds, the method claims to preserve spatial detail while also using pre-trained 2D semantics for generalization (Su et al., 19 Sep 2025). This is one of the main distinctions between DSPv2 and point-cloud-centric or purely image-centric imitation learning pipelines.

4. Dense Policy extension to whole-body control

DSPv2 extends the Dense Policy paradigm to whole-body mobile manipulation by generating a future action sequence $$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$6 with bidirectional, coarse-to-fine autoregression (Su et al., 19 Sep 2025). The paper defines keyframe levels

$$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$7

and factorizes the action distribution as

$$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$8

Here $$x_{\text{cam}} = [R \mid t] X_{\text{world}}, \qquad x_{\text{img}} = K x_{\text{cam}},$$9 is the encoded robot state, produced by an MLP after the specified masking procedure (Su et al., 19 Sep 2025).

The distinguishing property of the head is bidirectional attention across the temporal dimension, allowing future and past keyframes to condition on one another. The paper argues that this lets action dimensions for one component observe raw, already-decided actions of other components at critical timesteps, rather than noised values as in diffusion-based heads. The reported effect is better inter-component coherence, for example alignment of torso height, base offset, and arm path when reaching to grasp a bottle (Su et al., 19 Sep 2025).

The authors also state that the Dense Policy head has logarithmic inference complexity in horizon length, reducing latency versus hierarchical diffusion, although concrete frequency figures are not reported (Su et al., 19 Sep 2025). In the presented interpretation, dense autoregression also mitigates hierarchical error amplification, especially under domain shift, because the model conditions on clean keyframe actions rather than iteratively denoised action samples.

This suggests that DSPv2’s control strategy is not merely a different decoder, but a particular temporal factorization tailored to whole-body coordination. The paper attributes improvements in coherence and robustness partly to this head design, not only to the visual encoder.

5. Training protocol, hardware, and empirical results

DSPv2 is trained end-to-end by behavior cloning on expert demonstrations, maximizing $u = f_x x / z + c_x, \qquad v = f_y y / z + c_y.$0 under a supervised objective (Su et al., 19 Sep 2025). The paper gives the optimization target as

$u = f_x x / z + c_x, \qquad v = f_y y / z + c_y.$1

For continuous pose commands, $u = f_x x / z + c_x, \qquad v = f_y y / z + c_y.$2 is an $u = f_x x / z + c_x, \qquad v = f_y y / z + c_y.$3 loss, possibly component-weighted:

$u = f_x x / z + c_x, \qquad v = f_y y / z + c_y.$4

If some action subspaces are discrete, such as binary gripper states, cross-entropy can be used for those terms (Su et al., 19 Sep 2025). No diffusion or adversarial losses are used.

The dataset comprises 100 demonstrations per task, collected via VR teleoperation. The robot runs on an onboard Intel i7-1370PE CPU. Sensors include an Orbbec Femto Bolt on the head, an Orbbec Gemini 335 on the torso, and Intel RealSense D401 cameras on each wrist; two Livox MID-360 LiDARs are mounted but not used (Su et al., 19 Sep 2025).

Evaluation covers five tasks with 20 trials per method under matched randomized initial scenes: Pick and Place, Deliver, Sort, Bowling, and Cart Pushing (Su et al., 19 Sep 2025). Generalization is assessed under Light, Spatial Arrangement, Object Color, Object Shape, and Scene perturbations.

Task or condition	DSPv2 result	Reported status
Pick and Place	pick 80%, place 60%	best
Deliver	100%	best
Sort	Stack-I 90%, Stack-II 25%	best
Bowling	grasp 80%, hit 50%	best
Cart Pushing	90%	best or tied

Under generalization, DSPv2 is reported to show minimal performance drop relative to the original setups and to outperform the baselines WB-WIMA, DP, and DP3 (Su et al., 19 Sep 2025). The paper gives several concrete examples: Pick and Place under Spatial Arrangement drops from pick 80% and place 60% to pick 50% and place 20%; under Light it reaches pick 60% and place 40%; Deliver under unseen object color reaches 85% versus 100% in the original setting; Sort under novel object shape reaches Stack-I 80% versus 90% originally; Cart Pushing in a new scene reaches 80% versus 90% originally (Su et al., 19 Sep 2025).

A reported failure mode is also informative: in the reduced-height second stage of Sort, all methods failed, which the authors attribute to a distribution gap in demonstrated actions, specifically the absence of low-height right-arm picks in the demonstrations (Su et al., 19 Sep 2025). This places a clear boundary on the policy’s claimed generalization.

6. Ablations, limitations, and nomenclature

The ablations isolate both the action head and the vision fine-tuning strategy. When DSPv2 is paired with a diffusion head rather than the Dense Head, the Dense Head performs better in both original and generalization settings. The paper gives examples including Pick and Place place success of 60% versus 30%, Sort Stack-I of 90% versus 60%, and generalization scene success of 80% versus 60% (Su et al., 19 Sep 2025). The stated interpretation is that dense autoregression makes different components mutually visible through raw keyframe actions and better controls hierarchical error growth under domain shift.

For the DINOv2 fine-tuning strategy, the Cart Pushing task yields 90% success for both full fine-tuning and LoRA in the original scene, versus 70% for frozen features. In a new scene, LoRA reaches 80%, while full fine-tuning and frozen features each reach 60% (Su et al., 19 Sep 2025). The authors interpret this as evidence that LoRA retains upstream semantic priors while still adapting to task-level detail.

The paper identifies several limitations. High-frequency or highly constrained actions remain challenging, and incorporating tactile sensing or higher-rate control may be necessary. Like most imitation learning methods, DSPv2 does not generalize to action modes absent from demonstrations. The authors also note that extending to general Vision-Language-Action models for long-horizon, multi-stage tasks on whole-body platforms will require large-scale datasets and designs adapted to whole-body control (Su et al., 19 Sep 2025).

A final point of clarification concerns nomenclature. DSPv2 is distinct from the 2024 method “DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers,” which does not mention a “DSPv2” (Zhao et al., 2024). It is also distinct from “DPU-v2,” the DAG Processing Unit v2 used as a baseline in a 2024 SpTRSV accelerator paper; that work explicitly compares against DPU-v2 and states that “DSPv2” does not appear in the paper (Chen et al., 2024). In current arXiv usage represented here, DSPv2 refers to the whole-body mobile manipulation policy introduced in 2025 (Su et al., 19 Sep 2025).