---
title: 'LocoMamba: Vision-Driven Quadrupedal Control'
url: https://www.emergentmind.com/topics/locomamba
type: topic
---

# LocoMamba: Vision-Driven Quadrupedal Control

Searching arXiv for recent papers on “LocoMamba” and the naming-related “MambaLoc” to ground the article in current literature.
LocoMamba denotes a selective state-space-model-based research line centered on efficient long-range dependency modeling with Mamba. In its primary usage, it is a vision-driven, cross-modal deep reinforcement learning framework for quadrupedal locomotion that replaces attention-based fusion backbones with stacked selective state-space models, jointly processing proprioception and depth vision and training end-to-end with Proximal Policy Optimization under terrain and appearance randomization and an obstacle-density curriculum [2508.11849]. In the arXiv record, the label also appears as a naming variant of MambaLoc, a distinct single-image 6-DoF visual localization framework that inserts a bidirectional selective SSM block, the Global Information Selector, into an absolute pose regression pipeline [2408.09680].

## 1. Terminology and scope

The term *LocoMamba* is unambiguous only within the locomotion paper that introduces it as a vision-driven cross-modal DRL framework for quadrupedal control [2508.11849]. That system is formulated around a torque-controlled quadruped, egocentric depth sensing, proprioceptive state estimation, and end-to-end policy optimization.

A separate naming clarification is necessary because the camera-localization paper explicitly states that “LocoMamba” may be encountered as a synonym for *MambaLoc* [2408.09680]. MambaLoc is not a locomotion controller; it addresses single-image 6-DoF visual localization, or absolute pose regression, estimating camera position and orientation directly from an input image without requiring an explicit 3D map or SfM model at inference. The two systems therefore inhabit different problem domains—robot locomotion versus camera localization—even though both import selective SSMs to obtain efficient global context modeling.

This dual usage suggests that the common conceptual core is not the application domain but the use of selective SSMs as a substitute for more expensive global-context mechanisms such as quadratic self-attention or Non-local operations.

## 2. Locomotion problem setting and motivation

LocoMamba is designed for a torque-controlled quadruped with 12 actuated joints operating in PyBullet. The observation at time $t$ comprises a 93-dimensional proprioceptive vector $s^{prop}_t$ and a stack of the four most recent depth frames $\left[I^{depth}_{t-3}, I^{depth}_{t-2}, I^{depth}_{t-1}, I^{depth}_t\right]$, each of resolution $64\times 64$ [2508.11849]. The proprioceptive stream includes IMU readings, joint angles and velocities via recent actions, and base-state information.

The benchmark includes six map types with different terrain and obstacle characteristics: Wide Obstacle, Thin Obstacle, Wide Obstacle + spheres, Thin Obstacle + spheres, Moving Obstacle, and Mountain. Obstacles and spheres are randomized at episode reset, while the Moving Obstacle scenario updates obstacle positions during an episode. The Mountain map uses a goal at the summit, and reward is aligned with progress toward that goal.

The stated motivation is threefold. First, proprioception-only controllers are “blind”: they rely on reactive control and heavy domain randomization rather than exteroceptive foresight. Second, RNN-based controllers such as LSTM and GRU are limited by vanishing gradients, reduced capacity over long horizons, and optimization difficulties in extended sequences. Third, Transformer policies provide strong long-range modeling but incur quadratic memory and compute in token count, which restricts sequence length and visual resolution. LocoMamba replaces attention-based fusion with Mamba-style selective SSMs, whose token-dependent recurrent updates provide near-linear-time sequence modeling, low memory footprint, and an input-gated, exponentially decaying inductive bias that mitigates overfitting.

## 3. Cross-modal architecture and selective SSM fusion

LocoMamba encodes proprioception and depth into a shared token space of width $d=128$ [2508.11849]. The 93-dimensional proprioceptive vector is mapped by a 2-layer MLP with hidden sizes $(256,256)$ and ReLU activations, followed by a learned linear projection to produce the token $z^{prop}_t\in\mathbb{R}^d$. Each $64\times 64$ depth frame is processed by a lightweight CNN that patchifies the image into $N=(H/P)(W/P)$ spatial tokens, which are then linearly projected to width $d$, per-token layer-normalized, and augmented with learned spatial positional codes and modality tags.

The resulting cross-modal sequence is
$$
U_t\in\mathbb{R}^{(1+N)\times d}=[z^{prop}_t; Z^{vis}_t],
$$
and after adding positional and modality embeddings and applying LayerNorm,
$$
\hat{U}_t = LN(U_t + E^{spat}_{pos} + E_{mod}).
$$

Fusion is performed by $L_m=2$ stacked Mamba SSM blocks with residual connections and LayerNorm. For token $u_{t,k}\in\mathbb{R}^d$, the selective scan maintains a hidden state $x_{t,k}\in\mathbb{R}^h$ and applies token-dependent updates:
$$
x_{t,k+1} = \bar{A}_{t,k}(u_{t,k})\, x_{t,k} + \bar{B}_{t,k}(u_{t,k})\, u_{t,k},
$$
$$
y_{t,k} = \bar{C}_{t,k}(u_{t,k})\, x_{t,k} + \bar{D}_{t,k}(u_{t,k})\, u_{t,k},
$$
with the scan proceeding causally over token index $k$, and the initial state at time $t$ carried from the previous step as $x_{t,1}\leftarrow x_{t-1,1+N}$. Residual normalization is written as
$$
H^{(\ell+1)}_t = LN(Y^{(\ell)}_t + H^{(\ell)}_t), \qquad \ell=0,\dots,L_m-1,
$$
with $H^{(0)}_t=\hat{U}_t$.

After the final Mamba layer, the proprioceptive token and the average-pooled visual tokens are fused by a 2-layer MLP with hidden sizes $(256,256)$ and ReLU:
$$
\bar{y}^{vis}_t=\frac{1}{N}\sum_{i=1}^{N} y^{vis}_{t,i}, \qquad
h_t = f_{head}([y^{prop}_t;\bar{y}^{vis}_t])\in\mathbb{R}^{d_h}.
$$
The control feature $h_t$ is then consumed by policy and value heads.

The complexity claim is central. Each Mamba layer requires $O((1+N)L_m)$ time and memory, in contrast to the $O(((1+N)^2)L_m)$ scaling of global self-attention. The paper attributes the practical benefit to causal streaming over tokens, reuse of the recurrent state across time steps, and robustness to token length and image resolution.

## 4. Policy optimization, reward design, and training protocol

The policy outputs joint-angle changes for all 12 joints by means of a Gaussian action distribution conditioned on $h_t$:
$$
\tilde{a}_t \sim \mathcal{N}\!\big(\mu_\theta(h_t),\,\operatorname{diag}(\sigma_\theta^2(h_t))\big), \qquad
a_t = a_{\max}\tanh(\tilde{a}_t).
$$
Joint torques $\tau_t$ are realized by the simulator’s low-level PD and dynamics and are used in energy regularization [2508.11849].

The MDP is written as $\mathcal{M}=(\mathcal{S},\mathcal{A},P,r,\gamma)$ with return
$$
J(\theta)=\mathbb{E}\!\left[\sum_{t=0}^{T-1}\gamma^t r_t\right].
$$
Training uses PPO with clipped surrogate
$$
\rho_t(\theta)=\frac{\pi_\theta(a_t\mid h_t)}{\pi_{\theta_{\text{old}}}(a_t\mid h_t)},
$$
$$
\mathcal{L}_{clip}(\theta)=\mathbb{E}\Big[\min\big(\rho_t(\theta)A_t,\ \operatorname{clip}(\rho_t(\theta),1-\epsilon,1+\epsilon)A_t\big)\Big],
$$
critic loss
$$
\mathcal{L}_V(\phi)=\mathbb{E}\big[(V_\phi(h_t)-\hat{R}_t)^2\big],
$$
and total objective
$$
\mathcal{J}(\theta,\phi)=-\mathcal{L}_{clip}(\theta)+\beta_V\mathcal{L}_V(\phi)-\beta_H\mathbb{E}[\mathcal{H}_t].
$$
Advantages are computed with generalized advantage estimation:
$$
\delta_t = r_t + \gamma V_\phi(h_{t+1}) - V_\phi(h_t), \qquad
A_t = \sum_{l=0}^{T-1-t} (\gamma \lambda)^l\, \delta_{t+l}.
$$

The reward is a compact state-centric combination of forward progress, energy, survival, and optional sparse sphere bonuses:
$$
R_t=\alpha_{fwd}R^{fwd}_t+\alpha_{energy}R^{energy}_t+\alpha_{alive}R^{alive}_t+K_tR^{sphere}_t,
$$
with $\alpha_{fwd}=1$, $\alpha_{energy}=0.005$, and $\alpha_{alive}=0.1$. On flat terrains,
$$
R^{fwd}_t=\langle v_t,e_x\rangle,
$$
whereas on mountain terrain,
$$
R^{fwd}_t=\langle v_t,u^{goal}_t\rangle.
$$
The energy term is
$$
R^{energy}_t=-\|\tau_t\|_2^2,
$$
and the alive reward is $1$ until termination due to falls or unrecoverable collisions.

Training includes physics randomization at episode reset, with joint stiffness $K_P\in[40,90]$ N·m/rad, damping $K_D\in[0.4,0.8]$ N·m·s/rad, link inertia in $[0.5,1.5]\times$ default, lateral friction in $[0.5,1.25]$ N·s/m, body mass in $[0.8,1.2]\times$ default, motor friction in $[0.0,0.05]$ N·m·s/rad, motor strength in $[0.8,1.2]\times$ default, and sensor latency in $[0,0.04]$ s. Depth noise is injected by saturating $K\sim U\{3,\dots,30\}$ random pixels per depth frame to maximum sensor range. Obstacle density is linearly ramped from an easier setting to the target distribution over training iterations.

The training hyperparameters are explicitly reported: horizon $T=1000$, discount $\gamma=0.99$, samples per iteration $8192$, minibatch size $256$, PPO epochs per iteration $3$, clip parameter $\epsilon=0.2$, policy and value learning rates $1\times 10^{-4}$, optimizer Adam, standard Gaussian initialization, advantage normalization, and gradient clipping. The implementation uses PyTorch 2.4.1 on Ubuntu 22.04, with an Intel Xeon Gold 6430 CPU and an NVIDIA RTX 4090 GPU.

## 5. Empirical performance, learning efficiency, and generalization

On the main Thin Obstacle scenario, LocoMamba reports mean $\pm$ standard deviation across seeds of reward $762.34 \pm 156.53$, collisions $72.53 \pm 79.47$, and distance $32.41 \pm 5.23$ m [2508.11849]. The comparable baselines are: Proprio-Only at reward $145.64 \pm 89.55$, collisions $487.80 \pm 114.13$, distance $6.13 \pm 2.49$ m; Transformer Vision-Only at reward $187.45 \pm 93.88$, distance $6.75 \pm 2.90$ m; Mamba Vision-Only at reward $28.16 \pm 34.47$, distance $2.92 \pm 1.01$ m; and Transformer Proprio-Vision at reward $511.96 \pm 247.30$, collisions $141.83 \pm 158.47$, distance $24.85 \pm 7.34$ m. The reported gains over Transformer Proprio-Vision are $+48.9\%$ reward, $-48.9\%$ collisions, and $+30.4\%$ distance; over Proprio-Only they are $+423.4\%$ reward, $-85.1\%$ collisions, and $+428.7\%$ distance.

Under matched training budgets, the learning-efficiency analysis reports final reward over the last 120 epochs of approximately 2M samples as: Proprio-Only $27.2$, Transformer Vision-Only $577.8$, Mamba Vision-Only $676.0$, Transformer Proprio-Vision $714.1$, and LocoMamba $737.4$. Early learning slope, measured as reward gain per epoch over the first 120 epochs, is $1.69$, $3.69$, $4.22$, $3.66$, and $6.41$, respectively. Learning efficiency, defined as overall reward gain per epoch, is $0.27$, $1.07$, $1.15$, $1.28$, and $1.44$. AUC per epoch is $36.5$, $448.3$, $443.8$, $529.8$, and $601.7$. These numbers are used to argue both faster convergence and higher asymptotic performance for cross-modal Mamba fusion.

The reported stability analysis uses the coefficient of variation over the last 200 epochs. LocoMamba attains the lowest CoV for value loss, $0.215$, and for advantages, $0.708$. Relative to Transformer Proprio-Vision, the reductions are $-61\%$ in value-loss CoV, from $0.550$, and $-24\%$ in advantage CoV, from $0.932$.

Zero-shot generalization is evaluated by training on Thin Obstacle and testing on unseen conditions. LocoMamba reports reward $583.76 \pm 154.57$, collisions $470.27 \pm 108.15$, and distance $24.04 \pm 6.18$ m. The corresponding baselines are Proprio-Only at reward $106.45 \pm 74.94$, collisions $779.40 \pm 119.85$, distance $4.94 \pm 1.97$ m; Transformer Vision-Only at reward $151.75 \pm 82.28$, distance $5.75 \pm 2.68$ m; Mamba Vision-Only at reward $28.53 \pm 29.25$, distance $2.79 \pm 0.96$ m; and Transformer Proprio-Vision at reward $257.38 \pm 431.52$, collisions $593.73 \pm 135.62$, distance $16.45 \pm 8.25$ m. The reported gains are $+448\%$ reward, $-39.7\%$ collisions, and $+386.6\%$ distance over Proprio-Only, and $+126.8\%$ reward, $-20.8\%$ collisions, and $+46.1\%$ distance over Transformer Proprio-Vision.

The paper interprets these outcomes through three factors: near-linear scaling with token length, long-horizon temporal context maintained in a compact recurrent state, and the regularizing inductive bias of input-gated, exponentially decaying dynamics. A plausible implication is that the gains are not only a consequence of replacing attention with a cheaper operator, but of changing the optimization landscape seen by PPO.

## 6. Relation to MambaLoc in camera localization

The naming-overlap paper, *MambaLoc*, addresses single-image 6-DoF visual localization by importing selective SSMs into absolute pose regression [2408.09680]. Its pipeline begins with a shared CNN backbone and bifurcated heads for translation and rotation. Given an RGB image $I\in\mathbb{R}^{H\times W\times 3}$, the backbone produces activation maps $M_x$ and $M_q$, which are mapped by a $1\times 1$ convolution to unified channel depth $C_t=256$, flattened, and prepended with a learnable task token. Positional encodings are factorized into one-dimensional $X$- and $Y$-axis encodings. Each branch then uses an independent 6-layer Transformer encoder with multi-head attention, a 2-layer GeLU MLP, pre-LayerNorm, residuals, and dropout, yielding a task summary at the special token.

The distinctive module is the Global Information Selector, or GIS, a bidirectional single-layer selective SSM inserted after the encoder. For input $G_{in}\in\mathbb{R}^{B\times 1\times D}$, the method constructs $G_{flip}=\operatorname{flip}(G_{in}, time)$ and
$$
G_{concat}=\operatorname{concat}(G_{flip},G_{in})\in\mathbb{R}^{B\times 2\times D}.
$$
The selective SSM then uses structured $A\in\mathbb{R}^{D\times N}$, initialized with HIPPO/Legendre polynomials, together with content-dependent $B$, $C$, and $\Delta$ to discretize and scan the 2-step sequence. Gating is implemented by splitting a linear projection of $G_{concat}$ into hidden and gate parts, activating the gate with SiLU, and modulating the selective-scan output elementwise. The final global features $\hat{G}_x$ and $\hat{G}_q$ feed two MLP heads with hidden size $1024$ that regress 3D translation $x$ and a 4D quaternion $q$, with quaternion normalization before the rotation loss.

The camera-localization paper positions GIS against both Transformer attention and Non-local blocks. Transformer attention and Non-local operations scale as $O(L^2)$, whereas Mamba’s selective scan is linear in sequence length. GIS adds complexity
$$
O(BLDN),
$$
with $L=2$, $D=C_t$, and $N=16$. In practice, the paper reports average training time of 14 minutes on 7Scenes and 38 minutes on Cambridge Landmarks. On 7Scenes, the average MambaLoc pose error is $0.17$ m / $8.71^\circ$ with 14 minutes training time; on Cambridge, the average is $0.95$ m / $3.84^\circ$ with 38 minutes training time. It also reports that GIS is a plug-in block improving training speed and accuracy across PoseNet, LSTM-PN, and TransPoseNet, while adding only modest model-size overhead.

The relation between the two papers is methodological rather than task-specific. Both use selective SSMs as linear-complexity substitutes for more expensive global-context operators; both emphasize parameter efficiency, memory efficiency, and robustness; and both present the Mamba-derived module as a modular insertion point inside a broader perception-and-decision stack. This suggests a transferable design pattern for vision-heavy robotics workloads in which the main computational bottleneck is global token interaction rather than local encoding.

## 7. Limitations, deployment status, and open directions

The locomotion version of LocoMamba has not yet been validated in the real world [2508.11849]. The paper explicitly states that sim-to-real transfer, latency, and safety under field conditions remain future work. It also notes that extremely dense or rapidly moving obstacles may stress the controller, and that depth sensor artifacts beyond the modeled salt-like saturation—such as systematic biases or motion blur—could degrade performance. The proposed future directions include more comprehensive perception noise models, multi-sensor fusion such as LiDAR, multi-camera setups, higher-resolution vision, and tactile feedback. No code or model artifacts are released, although the paper states that the described training loop, hyperparameters, and architecture settings are sufficient for re-implementation in PyTorch and PyBullet.

MambaLoc, the camera-localization system sometimes referred to as LocoMamba, has a different limitation profile [2408.09680]. The paper notes occasional rotation-error increases when GIS is integrated into certain fine-tuned Transformer pipelines such as TransPoseNet, and it places multi-scene and NeRF-augmented approaches outside the scope of its fair speed comparisons. It reports public availability of code and models, but no repository URL is included in the text. It also highlights open challenges involving more detailed reporting of hardware-aware algorithm internals and code availability.

Taken together, the two papers place *LocoMamba* within a broader selective-SSM trend: replacing quadratic token-mixing mechanisms with compact recurrent scanning while retaining long-range dependency modeling. In locomotion, that strategy is used for cross-modal policy learning; in camera localization, it is used for absolute pose regression. The shared claim is that selective SSMs can provide global information capture with lower complexity, lower memory footprint, and improved training behavior, but both application areas still leave deployment-specific questions—especially hardware latency, safety, and robustness beyond controlled evaluation conditions—only partially resolved.

Source: https://www.emergentmind.com/topics/locomamba