Mamba-HMR: 3D Human Mesh Recovery

• Mamba-HMR is a state-space sequence modeling approach that reconstructs dense 3D human meshes from single images and video using structured state-space models.
• It employs efficient scanning and innovative token serialization to overcome limitations of transformer-based methods, delivering improved FPS and accuracy.
• The framework features distinct pipelines—one for non-parametric vertex-based recovery and one for geometry-aware, motion-guided video reconstruction—to ensure spatial and temporal consistency.

Mamba-HMR denotes a Mamba-based approach to 3D human mesh recovery in which human reconstruction is formulated as state-space sequence modeling over ordered geometric tokens rather than only as image-to-parameter regression. In the supplied arXiv literature, the designation appears in two closely related forms: a single-image, non-parametric, vertex-based whole-body regressor introduced within MeshMamba (Yoshiyasu et al., 21 Jul 2025), and the video-based HMRMamba framework for geometry-aware and motion-guided human mesh recovery (Chen et al., 29 Jan 2026). Both systems use Structured State Space Models (SSMs) to exploit long-range dependencies with efficient scanning, but they differ substantially in input modality, tokenization strategy, and the role of temporal information.

1. Scope, task formulation, and nomenclature

Human Mesh Recovery (HMR) seeks to reconstruct a dense triangular mesh of a person from visual input. In the single-image formulation, the input is an RGB image $I\in\mathbb{R}^{H\times W\times 3}$ and the output is a dense mesh $V\in\mathbb{R}^{N\times 3}$ with fixed connectivity inherited from a template such as SMPL or SMPL-X; the conditional target is the distribution $p(V\mid I)$ (Yoshiyasu et al., 21 Jul 2025). In the video formulation, the goal is to recover a temporally coherent mesh sequence from tracked 2D joints and image features, with particular emphasis on reliable 3D pose anchors and motion-aware reconstruction (Chen et al., 29 Jan 2026).

The two main instantiations differ in representation. The MeshMamba version of Mamba-HMR is explicitly non-parametric and vertex-based, operating at full mesh resolution and decoding per-vertex offsets added to a canonical template shape (Yoshiyasu et al., 21 Jul 2025). HMRMamba instead introduces a two-stage video pipeline: a Geometry-Aware Lifting Module that produces a 3D pose anchor sequence, followed by a Motion-guided Reconstruction Network that regresses the final SMPL mesh sequence (Chen et al., 29 Jan 2026).

This division is significant because it isolates two different technical problems within HMR. The single-image setting emphasizes serialization of very large vertex sets and efficient whole-body inference. The video setting emphasizes the instability of intermediate pose anchors, temporal coherence, and robustness under occlusion and motion blur. A plausible implication is that “Mamba-HMR” is best understood not as a single canonical architecture, but as a class of SSM-based HMR systems sharing an efficiency-oriented sequential design.

2. State-space foundations

The video-based HMRMamba paper grounds its sequence modules in a continuous linear time-invariant system

$h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$

which under zero-order hold discretization with time-step $\Delta$ yields

$$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$

The per-step recurrence becomes

$h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$

and admits a single global convolution

$y=x\circledast \bar K,$

with kernel

$$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$

This mechanism underlies the “Global Scan” passes in the Dual-Scan Mamba blocks (Chen et al., 29 Jan 2026).

The single-image Mamba-HMR paper adopts a different but related state-space view. It casts feed-forward recovery as a $T$-step state-space inference over a latent mesh code $V\in\mathbb{R}^{N\times 3}$0:

$V\in\mathbb{R}^{N\times 3}$1

followed by

$V\in\mathbb{R}^{N\times 3}$2

Training does not explicitly optimize a Gaussian negative-log-likelihood; instead, the model subsumes this view through standard $V\in\mathbb{R}^{N\times 3}$3, geometric, and joint regression losses (Yoshiyasu et al., 21 Jul 2025).

The common premise is that reconstruction quality depends on how effectively the model propagates information across long token sequences. In HMRMamba, this propagation is organized around spatial and temporal scans over joints and frames. In the MeshMamba formulation, it is organized around scans over serialized vertices and joint queries. This suggests that the main conceptual contribution of Mamba-HMR lies less in a specific output parameterization than in the replacement of generic sequence mixing with task-structured SSM recurrences.

3. Single-image Mamba-HMR within MeshMamba

The MeshMamba paper introduces Mamba-HMR as a 3D human mesh recovery model that reconstructs a human body shape and pose from a single image and extends previous non-parametric HMR approaches from body-only poses with around 500 vertex tokens to the whole-body setting with face and hands, while achieving competitive performance in (near) real time (Yoshiyasu et al., 21 Jul 2025).

A central difficulty is that Mamba blocks operate over 1D sequences of length $V\in\mathbb{R}^{N\times 3}$4, so the mesh vertices must be serialized. Two complementary orderings are used. The first is a template-coordinate sort, where vertices from the template mesh in T-pose are sorted by coordinate orderings such as “xyz”, “-xyz”, or “yzx”. The second is a DensePose part-IUV sort, where each vertex is assigned to one of 24 body parts and sorted by $V\in\mathbb{R}^{N\times 3}$5 then $V\in\mathbb{R}^{N\times 3}$6, while the parts themselves are ordered by 3D segment centroids. A 2-strategy setup alternates these permutations across Mamba layers: all but one layer use $V\in\mathbb{R}^{N\times 3}$7, and the single mid-layer uses $V\in\mathbb{R}^{N\times 3}$8 (Yoshiyasu et al., 21 Jul 2025).

The architecture contains a CNN encoder, described as a ResNet/HRNet backbone producing a spatial feature map $V\in\mathbb{R}^{N\times 3}$9. It then prepares $p(V\mid I)$0 body-joint queries and $p(V\mid I)$1 vertex queries, each embedded to $p(V\mid I)$2 and augmented with standard “sin-cos” positional encodings. The Mamba stack is divided into shallow, mid, and deep groups with skip connections from shallow to deep groups. Each block applies a selective SSM layer $p(V\mid I)$3, two feed-forward MLPs, a small 1D convolution, and GELU activations. The input to block $p(V\mid I)$4 is the concatenation $p(V\mid I)$5 permuted by the chosen serialization. The final decoder is an MLP that maps the Mamba output $p(V\mid I)$6 to per-vertex 3D offsets added to a canonical template shape to obtain $p(V\mid I)$7 (Yoshiyasu et al., 21 Jul 2025).

Inference proceeds by encoding the image, initializing $p(V\mid I)$8 from the projected queries, iterating the Mamba blocks, and decoding the final vertex set. Each block processes all $p(V\mid I)$9 tokens in $h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$0 time by convolutional SSM scans. On a single NVIDIA V100 GPU with $h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$1 vertices, the reported speed is approximately $h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$2 FPS, whereas a pure transformer baseline runs at approximately $h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$3 FPS on the same hardware (Yoshiyasu et al., 21 Jul 2025).

4. Geometry-aware and motion-guided video HMRMamba

HMRMamba addresses video-based 3D HMR, where prior methods are described as producing physically implausible results because of flawed intermediate 3D pose anchors and ineffective modeling of complex spatiotemporal dynamics. Its first component, the Geometry-Aware Lifting Module, takes 2D joint tracks $h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$4 and image features $h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$5, and produces a grounded 3D anchor sequence

$h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$6

The pipeline consists of encoder fusion, a per-frame SpatialMamba stage, deformable attention that ties image cues to joint features, a TemporalMamba stage across frames, and a lifting head MLP that outputs final 3D joints (Chen et al., 29 Jan 2026).

Its core sequence block is the Dual-Scan Mamba. The global scan imposes a linear ordering over joints or time to capture long-range dependencies. The local, or kinematic, scan traverses the human skeletal tree in a non-sequential order such as torso $h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$7 shoulder $h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$8 elbow, thereby injecting anatomy priors. Their outputs are fused as

$h'(t)=A\,h(t)+B\,x(t), \qquad y(t)=C\,h(t),$9

This is a distinctive design choice because it combines the efficiency of SSM scanning with an explicitly human-structured traversal (Chen et al., 29 Jan 2026).

The second component is the Motion-guided Reconstruction Network. It takes the full 3D anchor sequence $\Delta$0 and $\Delta$1, and applies motion-aware attention before mesh regression. Explicit motion is defined as

$\Delta$2

while implicit motion is learned via correction of $\Delta$3 by motion cues. Queries are $\Delta$4, and keys and values come from linear projections of $\Delta$5, giving

$\Delta$6

A final mesh regressor $\Delta$7 outputs

$\Delta$8

with $\Delta$9 SMPL vertices (Chen et al., 29 Jan 2026).

The paper characterizes this design as an architectural advance because the 2D-to-3D lifting is grounded in image geometry and the reconstruction stage receives explicitly temporally coherent motion cues. The stated effect is improved coherence and robustness, particularly under occlusion and motion blur (Chen et al., 29 Jan 2026).

5. Objectives, benchmarks, and quantitative performance

The single-image Mamba-HMR model is trained end-to-end with

$$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$0

The losses comprise a vertex loss, 3D joint loss, 2D joint losses on projected joints, edge length loss, Laplacian loss, and normal loss. The geometric terms compare edge lengths, cotangent-Laplacian responses, and per-face normals between prediction and ground truth (Yoshiyasu et al., 21 Jul 2025).

The video HMRMamba model separates supervision into a lifting objective and a mesh-recovery objective. The lifting loss is

$$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$1

with $$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$2, $$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$3, and $$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$4, where $$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$5 is MPJPE, $$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$6 is temporal consistency, $$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$7 is MPJVE, and $$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$8 is 2D reprojection error. The mesh recovery loss is

$$\bar A=\exp(\Delta A), \qquad \bar B=(\Delta A)^{-1}(\exp(\Delta A)-I)(\Delta B).$$9

with $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$0, $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$1, $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$2, and $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$3 (Chen et al., 29 Jan 2026).

On the UBody test set without fine-tuning, the single-image Mamba-HMR reports PA-MVE $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$4 mm, PA-MVE (Hands) $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$5 mm, PA-MVE (Face) $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$6 mm, MVE $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$7 mm, and FPS $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$8. After fine-tuning on UBody, it improves to PA-MVE $h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,$9 mm and MVE $y=x\circledast \bar K,$0 mm (Yoshiyasu et al., 21 Jul 2025).

Setting	Reported result
UBody, no fine-tuning	PA-MVE $y=x\circledast \bar K,$1 mm; Hands $y=x\circledast \bar K,$2 mm; Face $y=x\circledast \bar K,$3 mm; MVE $y=x\circledast \bar K,$4 mm; FPS $y=x\circledast \bar K,$5
UBody, fine-tuning	PA-MVE $y=x\circledast \bar K,$6 mm; MVE $y=x\circledast \bar K,$7 mm

For HMRMamba, the reported benchmark results are as follows.

Benchmark	Ours-S	Ours-L
3DPW	MPJPE $y=x\circledast \bar K,$8, PA-MPJPE $y=x\circledast \bar K,$9, MPVPE $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$0, Accel $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$1	MPJPE $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$2, PA-MPJPE $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$3, MPVPE $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$4, Accel $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$5
MPI-INF-3DHP	MPJPE $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$6, PA-MPJPE $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$7, Accel $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$8	MPJPE $$\bar K=[C\bar B,\; C\bar A\bar B,\; \ldots,\; C\bar A^{(L-1)}\bar B]^\top.$$9, PA-MPJPE $T$0, Accel $T$1
Human3.6M	MPJPE $T$2, PA-MPJPE $T$3, Accel $T$4	MPJPE $T$5, PA-MPJPE $T$6, Accel $T$7

The accompanying efficiency figures on Human3.6M are PMCE at $T$8 M parameters and $T$9 GFlops, ARTS at $V\in\mathbb{R}^{N\times 3}$00 M parameters and $V\in\mathbb{R}^{N\times 3}$01 GFlops, Ours-S at $V\in\mathbb{R}^{N\times 3}$02 M parameters and $V\in\mathbb{R}^{N\times 3}$03 GFlops, and Ours-L at $V\in\mathbb{R}^{N\times 3}$04 M parameters and $V\in\mathbb{R}^{N\times 3}$05 GFlops. The paper states that the S-model is $V\in\mathbb{R}^{N\times 3}$06 smaller than PMCE and $V\in\mathbb{R}^{N\times 3}$07 smaller than ARTS, with comparable or lower GFlops while delivering $V\in\mathbb{R}^{N\times 3}$08–$V\in\mathbb{R}^{N\times 3}$09 relative MPJPE improvement (Chen et al., 29 Jan 2026).

6. Limitations, significance, and sources of confusion

The limitations of the single-image Mamba-HMR model are stated directly. Fixed topology prevents modeling loose clothing or hairstyles. Generalization to in-the-wild clothing, wild backgrounds, and out-of-distribution poses such as sports or acrobatics remains challenging. The approach also relies on large-scale supervised datasets including COCO, Human3.6M, and UBody, and the paper identifies self-supervised or weakly supervised extensions as a direction that would broaden applicability (Yoshiyasu et al., 21 Jul 2025).

The video HMRMamba work is positioned against two recurring failure modes in prior video HMR: flawed anchors and static reconstructions. Its stated contribution is to overcome these through geometry-aware 2D-to-3D lifting and explicit motion injection into mesh recovery, leading to improvements in reconstruction accuracy, temporal consistency, and computational efficiency (Chen et al., 29 Jan 2026). Taken together with the single-image results, this indicates a broader transition in HMR from attention-heavy token interaction toward structure-aware SSM designs.

A common source of confusion is nomenclature. In the supplied material, a separate paper titled “HGP-Mamba: Integrating Histology and Generated Protein Features for Mamba-based Multimodal Survival Risk Prediction” includes the phrase that it is “also referred to as Mamba-HMR in some contexts,” even though its declared task is survival risk prediction rather than human mesh recovery (Dai et al., 17 Mar 2026). This suggests that the label “Mamba-HMR” is not fully standardized across preprints. In the human-mesh literature, however, the term refers to the Mamba-based HMR systems described above rather than to multimodal pathology models.

The broader significance of Mamba-HMR is therefore methodological. The MeshMamba paper concludes that state-space models with careful mesh serialization can match or exceed transformer-based HMR in both accuracy and speed, while HMRMamba reports state-of-the-art results with superior computational efficiency in video HMR (Yoshiyasu et al., 21 Jul 2025, Chen et al., 29 Jan 2026). Within the supplied evidence, Mamba-HMR is best situated as a state-space reformulation of dense human reconstruction, spanning both real-time whole-body single-image recovery and temporally coherent video mesh estimation.