LocoMamba: Vision-Driven Quadrupedal Control

• LocoMamba is a vision-driven, cross-modal framework employing selective state-space models to efficiently model long-range dependencies for quadrupedal locomotion.
• It integrates proprioceptive data and depth vision into a unified token space, replacing quadratic attention with near-linear recurrent processing.
• Empirical results show significant gains in reward, collision reduction, and distance over baselines, highlighting improved learning efficiency and stability.

Searching arXiv for papers on “LocoMamba” and the naming-related “MambaLoc” to ground the article in current literature. LocoMamba denotes a selective state-space-model-based research line centered on efficient long-range dependency modeling with Mamba. In its primary usage, it is a vision-driven, cross-modal deep reinforcement learning framework for quadrupedal locomotion that replaces attention-based fusion backbones with stacked selective state-space models, jointly processing proprioception and depth vision and training end-to-end with Proximal Policy Optimization under terrain and appearance randomization and an obstacle-density curriculum (Wang et al., 16 Aug 2025). In the arXiv record, the label also appears as a naming variant of MambaLoc, a distinct single-image 6-DoF visual localization framework that inserts a bidirectional selective SSM block, the Global Information Selector, into an absolute pose regression pipeline (Wang et al., 2024).

1. Terminology and scope

The term LocoMamba is unambiguous only within the locomotion paper that introduces it as a vision-driven cross-modal DRL framework for quadrupedal control (Wang et al., 16 Aug 2025). That system is formulated around a torque-controlled quadruped, egocentric depth sensing, proprioceptive state estimation, and end-to-end policy optimization.

A separate naming clarification is necessary because the camera-localization paper explicitly states that “LocoMamba” may be encountered as a synonym for MambaLoc (Wang et al., 2024). MambaLoc is not a locomotion controller; it addresses single-image 6-DoF visual localization, or absolute pose regression, estimating camera position and orientation directly from an input image without requiring an explicit 3D map or SfM model at inference. The two systems therefore inhabit different problem domains—robot locomotion versus camera localization—even though both import selective SSMs to obtain efficient global context modeling.

This dual usage suggests that the common conceptual core is not the application domain but the use of selective SSMs as a substitute for more expensive global-context mechanisms such as quadratic self-attention or Non-local operations.

2. Locomotion problem setting and motivation

LocoMamba is designed for a torque-controlled quadruped with 12 actuated joints operating in PyBullet. The observation at time $t$ comprises a 93-dimensional proprioceptive vector $s^{prop}_t$ and a stack of the four most recent depth frames $$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$, each of resolution $64\times 64$ (Wang et al., 16 Aug 2025). The proprioceptive stream includes IMU readings, joint angles and velocities via recent actions, and base-state information.

The benchmark includes six map types with different terrain and obstacle characteristics: Wide Obstacle, Thin Obstacle, Wide Obstacle + spheres, Thin Obstacle + spheres, Moving Obstacle, and Mountain. Obstacles and spheres are randomized at episode reset, while the Moving Obstacle scenario updates obstacle positions during an episode. The Mountain map uses a goal at the summit, and reward is aligned with progress toward that goal.

The stated motivation is threefold. First, proprioception-only controllers are “blind”: they rely on reactive control and heavy domain randomization rather than exteroceptive foresight. Second, RNN-based controllers such as LSTM and GRU are limited by vanishing gradients, reduced capacity over long horizons, and optimization difficulties in extended sequences. Third, Transformer policies provide strong long-range modeling but incur quadratic memory and compute in token count, which restricts sequence length and visual resolution. LocoMamba replaces attention-based fusion with Mamba-style selective SSMs, whose token-dependent recurrent updates provide near-linear-time sequence modeling, low memory footprint, and an input-gated, exponentially decaying inductive bias that mitigates overfitting.

3. Cross-modal architecture and selective SSM fusion

LocoMamba encodes proprioception and depth into a shared token space of width $d=128$ (Wang et al., 16 Aug 2025). The 93-dimensional proprioceptive vector is mapped by a 2-layer MLP with hidden sizes $(256,256)$ and ReLU activations, followed by a learned linear projection to produce the token $z^{prop}_t\in\mathbb{R}^d$. Each $64\times 64$ depth frame is processed by a lightweight CNN that patchifies the image into $N=(H/P)(W/P)$ spatial tokens, which are then linearly projected to width $d$, per-token layer-normalized, and augmented with learned spatial positional codes and modality tags.

The resulting cross-modal sequence is

$s^{prop}_t$0

and after adding positional and modality embeddings and applying LayerNorm,

$s^{prop}_t$1

Fusion is performed by $s^{prop}_t$2 stacked Mamba SSM blocks with residual connections and LayerNorm. For token $s^{prop}_t$3, the selective scan maintains a hidden state $s^{prop}_t$4 and applies token-dependent updates:

$s^{prop}_t$5

$s^{prop}_t$6

with the scan proceeding causally over token index $s^{prop}_t$7, and the initial state at time $s^{prop}_t$8 carried from the previous step as $s^{prop}_t$9. Residual normalization is written as

$$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$0

with $$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$1.

After the final Mamba layer, the proprioceptive token and the average-pooled visual tokens are fused by a 2-layer MLP with hidden sizes $$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$2 and ReLU:

$$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$3

The control feature $$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$4 is then consumed by policy and value heads.

The complexity claim is central. Each Mamba layer requires $$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$5 time and memory, in contrast to the $$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$6 scaling of global self-attention. The paper attributes the practical benefit to causal streaming over tokens, reuse of the recurrent state across time steps, and robustness to token length and image resolution.

4. Policy optimization, reward design, and training protocol

The policy outputs joint-angle changes for all 12 joints by means of a Gaussian action distribution conditioned on $$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$7:

$$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$8

Joint torques $$\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$$9 are realized by the simulator’s low-level PD and dynamics and are used in energy regularization (Wang et al., 16 Aug 2025).

The MDP is written as $64\times 64$0 with return

$64\times 64$1

Training uses PPO with clipped surrogate

$64\times 64$2

$64\times 64$3

critic loss

$64\times 64$4

and total objective

$64\times 64$5

Advantages are computed with generalized advantage estimation:

$64\times 64$6

The reward is a compact state-centric combination of forward progress, energy, survival, and optional sparse sphere bonuses:

$64\times 64$7

with $64\times 64$8, $64\times 64$9, and $d=128$0. On flat terrains,

$d=128$1

whereas on mountain terrain,

$d=128$2

The energy term is

$d=128$3

and the alive reward is $d=128$4 until termination due to falls or unrecoverable collisions.

Training includes physics randomization at episode reset, with joint stiffness $d=128$5 N·m/rad, damping $d=128$6 N·m·s/rad, link inertia in $d=128$7 default, lateral friction in $d=128$8 N·s/m, body mass in $d=128$9 default, motor friction in $(256,256)$0 N·m·s/rad, motor strength in $(256,256)$1 default, and sensor latency in $(256,256)$2 s. Depth noise is injected by saturating $(256,256)$3 random pixels per depth frame to maximum sensor range. Obstacle density is linearly ramped from an easier setting to the target distribution over training iterations.

The training hyperparameters are explicitly reported: horizon $(256,256)$4, discount $(256,256)$5, samples per iteration $(256,256)$6, minibatch size $(256,256)$7, PPO epochs per iteration $(256,256)$8, clip parameter $(256,256)$9, policy and value learning rates $z^{prop}_t\in\mathbb{R}^d$0, optimizer Adam, standard Gaussian initialization, advantage normalization, and gradient clipping. The implementation uses PyTorch 2.4.1 on Ubuntu 22.04, with an Intel Xeon Gold 6430 CPU and an NVIDIA RTX 4090 GPU.

5. Empirical performance, learning efficiency, and generalization

On the main Thin Obstacle scenario, LocoMamba reports mean $z^{prop}_t\in\mathbb{R}^d$1 standard deviation across seeds of reward $z^{prop}_t\in\mathbb{R}^d$2, collisions $z^{prop}_t\in\mathbb{R}^d$3, and distance $z^{prop}_t\in\mathbb{R}^d$4 m (Wang et al., 16 Aug 2025). The comparable baselines are: Proprio-Only at reward $z^{prop}_t\in\mathbb{R}^d$5, collisions $z^{prop}_t\in\mathbb{R}^d$6, distance $z^{prop}_t\in\mathbb{R}^d$7 m; Transformer Vision-Only at reward $z^{prop}_t\in\mathbb{R}^d$8, distance $z^{prop}_t\in\mathbb{R}^d$9 m; Mamba Vision-Only at reward $64\times 64$0, distance $64\times 64$1 m; and Transformer Proprio-Vision at reward $64\times 64$2, collisions $64\times 64$3, distance $64\times 64$4 m. The reported gains over Transformer Proprio-Vision are $64\times 64$5 reward, $64\times 64$6 collisions, and $64\times 64$7 distance; over Proprio-Only they are $64\times 64$8 reward, $64\times 64$9 collisions, and $N=(H/P)(W/P)$0 distance.

Under matched training budgets, the learning-efficiency analysis reports final reward over the last 120 epochs of approximately 2M samples as: Proprio-Only $N=(H/P)(W/P)$1, Transformer Vision-Only $N=(H/P)(W/P)$2, Mamba Vision-Only $N=(H/P)(W/P)$3, Transformer Proprio-Vision $N=(H/P)(W/P)$4, and LocoMamba $N=(H/P)(W/P)$5. Early learning slope, measured as reward gain per epoch over the first 120 epochs, is $N=(H/P)(W/P)$6, $N=(H/P)(W/P)$7, $N=(H/P)(W/P)$8, $N=(H/P)(W/P)$9, and $d$0, respectively. Learning efficiency, defined as overall reward gain per epoch, is $d$1, $d$2, $d$3, $d$4, and $d$5. AUC per epoch is $d$6, $d$7, $d$8, $d$9, and $s^{prop}_t$00. These numbers are used to argue both faster convergence and higher asymptotic performance for cross-modal Mamba fusion.

The reported stability analysis uses the coefficient of variation over the last 200 epochs. LocoMamba attains the lowest CoV for value loss, $s^{prop}_t$01, and for advantages, $s^{prop}_t$02. Relative to Transformer Proprio-Vision, the reductions are $s^{prop}_t$03 in value-loss CoV, from $s^{prop}_t$04, and $s^{prop}_t$05 in advantage CoV, from $s^{prop}_t$06.

Zero-shot generalization is evaluated by training on Thin Obstacle and testing on unseen conditions. LocoMamba reports reward $s^{prop}_t$07, collisions $s^{prop}_t$08, and distance $s^{prop}_t$09 m. The corresponding baselines are Proprio-Only at reward $s^{prop}_t$10, collisions $s^{prop}_t$11, distance $s^{prop}_t$12 m; Transformer Vision-Only at reward $s^{prop}_t$13, distance $s^{prop}_t$14 m; Mamba Vision-Only at reward $s^{prop}_t$15, distance $s^{prop}_t$16 m; and Transformer Proprio-Vision at reward $s^{prop}_t$17, collisions $s^{prop}_t$18, distance $s^{prop}_t$19 m. The reported gains are $s^{prop}_t$20 reward, $s^{prop}_t$21 collisions, and $s^{prop}_t$22 distance over Proprio-Only, and $s^{prop}_t$23 reward, $s^{prop}_t$24 collisions, and $s^{prop}_t$25 distance over Transformer Proprio-Vision.

The paper interprets these outcomes through three factors: near-linear scaling with token length, long-horizon temporal context maintained in a compact recurrent state, and the regularizing inductive bias of input-gated, exponentially decaying dynamics. A plausible implication is that the gains are not only a consequence of replacing attention with a cheaper operator, but of changing the optimization landscape seen by PPO.

6. Relation to MambaLoc in camera localization

The naming-overlap paper, MambaLoc, addresses single-image 6-DoF visual localization by importing selective SSMs into absolute pose regression (Wang et al., 2024). Its pipeline begins with a shared CNN backbone and bifurcated heads for translation and rotation. Given an RGB image $s^{prop}_t$26, the backbone produces activation maps $s^{prop}_t$27 and $s^{prop}_t$28, which are mapped by a $s^{prop}_t$29 convolution to unified channel depth $s^{prop}_t$30, flattened, and prepended with a learnable task token. Positional encodings are factorized into one-dimensional $s^{prop}_t$31- and $s^{prop}_t$32-axis encodings. Each branch then uses an independent 6-layer Transformer encoder with multi-head attention, a 2-layer GeLU MLP, pre-LayerNorm, residuals, and dropout, yielding a task summary at the special token.

The distinctive module is the Global Information Selector, or GIS, a bidirectional single-layer selective SSM inserted after the encoder. For input $s^{prop}_t$33, the method constructs $s^{prop}_t$34 and

$s^{prop}_t$35

The selective SSM then uses structured $s^{prop}_t$36, initialized with HIPPO/Legendre polynomials, together with content-dependent $s^{prop}_t$37, $s^{prop}_t$38, and $s^{prop}_t$39 to discretize and scan the 2-step sequence. Gating is implemented by splitting a linear projection of $s^{prop}_t$40 into hidden and gate parts, activating the gate with SiLU, and modulating the selective-scan output elementwise. The final global features $s^{prop}_t$41 and $s^{prop}_t$42 feed two MLP heads with hidden size $s^{prop}_t$43 that regress 3D translation $s^{prop}_t$44 and a 4D quaternion $s^{prop}_t$45, with quaternion normalization before the rotation loss.

The camera-localization paper positions GIS against both Transformer attention and Non-local blocks. Transformer attention and Non-local operations scale as $s^{prop}_t$46, whereas Mamba’s selective scan is linear in sequence length. GIS adds complexity

$s^{prop}_t$47

with $s^{prop}_t$48, $s^{prop}_t$49, and $s^{prop}_t$50. In practice, the paper reports average training time of 14 minutes on 7Scenes and 38 minutes on Cambridge Landmarks. On 7Scenes, the average MambaLoc pose error is $s^{prop}_t$51 m / $s^{prop}_t$52 with 14 minutes training time; on Cambridge, the average is $s^{prop}_t$53 m / $s^{prop}_t$54 with 38 minutes training time. It also reports that GIS is a plug-in block improving training speed and accuracy across PoseNet, LSTM-PN, and TransPoseNet, while adding only modest model-size overhead.

The relation between the two papers is methodological rather than task-specific. Both use selective SSMs as linear-complexity substitutes for more expensive global-context operators; both emphasize parameter efficiency, memory efficiency, and robustness; and both present the Mamba-derived module as a modular insertion point inside a broader perception-and-decision stack. This suggests a transferable design pattern for vision-heavy robotics workloads in which the main computational bottleneck is global token interaction rather than local encoding.

7. Limitations, deployment status, and open directions

The locomotion version of LocoMamba has not yet been validated in the real world (Wang et al., 16 Aug 2025). The paper explicitly states that sim-to-real transfer, latency, and safety under field conditions remain future work. It also notes that extremely dense or rapidly moving obstacles may stress the controller, and that depth sensor artifacts beyond the modeled salt-like saturation—such as systematic biases or motion blur—could degrade performance. The proposed future directions include more comprehensive perception noise models, multi-sensor fusion such as LiDAR, multi-camera setups, higher-resolution vision, and tactile feedback. No code or model artifacts are released, although the paper states that the described training loop, hyperparameters, and architecture settings are sufficient for re-implementation in PyTorch and PyBullet.

MambaLoc, the camera-localization system sometimes referred to as LocoMamba, has a different limitation profile (Wang et al., 2024). The paper notes occasional rotation-error increases when GIS is integrated into certain fine-tuned Transformer pipelines such as TransPoseNet, and it places multi-scene and NeRF-augmented approaches outside the scope of its fair speed comparisons. It reports public availability of code and models, but no repository URL is included in the text. It also highlights open challenges involving more detailed reporting of hardware-aware algorithm internals and code availability.

Taken together, the two papers place LocoMamba within a broader selective-SSM trend: replacing quadratic token-mixing mechanisms with compact recurrent scanning while retaining long-range dependency modeling. In locomotion, that strategy is used for cross-modal policy learning; in camera localization, it is used for absolute pose regression. The shared claim is that selective SSMs can provide global information capture with lower complexity, lower memory footprint, and improved training behavior, but both application areas still leave deployment-specific questions—especially hardware latency, safety, and robustness beyond controlled evaluation conditions—only partially resolved.