WildPose: Robust Monocular Pose Estimation

Updated 4 July 2026

The paper presents a unified framework combining a frozen MASt3R encoder with a DROID-SLAM-style backend to achieve state-of-the-art monocular pose estimation in dynamic environments.
It introduces a novel motion mask detector that integrates with differentiable bundle adjustment, effectively mitigating errors from moving objects in dynamic scenes.
Empirical evaluations across dynamic and static benchmarks demonstrate WildPose’s improved accuracy and speed, outperforming or matching competing methods.

Searching arXiv for the WildPose paper and closely related methods cited in the provided material. WildPose is a unified monocular pose estimation framework for dynamic environments that is designed to remain robust when the static-world assumption fails, while maintaining state-of-the-art performance on static and low-ego-motion datasets. Its central design is to fuse the rich perceptual frontend of a feedforward SfM model with the end-to-end optimization of differentiable bundle adjustment, specifically by combining a frozen, pre-trained MASt3R feature backbone with a DROID-SLAM-style backend, a 3D-aware update operator, and a high-capacity motion mask detector (Zheng et al., 12 May 2026).

1. Problem setting and scope

WildPose addresses camera pose estimation in monocular video under dynamic scene conditions. The motivating observation is that dynamic scenes violate the static-world assumption at the core of most monocular SLAM and SfM methods. Moving objects produce outlier correspondences, inflate reprojection residuals, and force bundle adjustment to explain away motion via erroneous camera-pose or depth updates, which can lead to tracking failure (Zheng et al., 12 May 2026).

The framework is positioned against three families of remedies described as incomplete when used in isolation. Semantic-based approaches are characterized as brittle, per-sequence optimization methods are described as failing on short sequences, and purely feed-forward learned models are described as losing precision on static scenes. WildPose is therefore presented as a unified alternative rather than a dynamic-only specialization (Zheng et al., 12 May 2026).

A common misconception is that robustness to dynamic content must come at the expense of static-scene accuracy. The reported evaluation is explicitly framed to counter that view: WildPose is evaluated not only on dynamic benchmarks such as Wild-SLAM and Bonn, but also on static and low-ego-motion benchmarks including TUM, 7-Scenes, and Sintel, where it remains competitive or best (Zheng et al., 12 May 2026).

2. System architecture

WildPose extends the DROID-SLAM pipeline by replacing its scratch-trained CNN frontend with a frozen MASt3R encoder and by adding a learned motion-mask module. MASt3R is used as a ViT-based feed-forward SfM backbone whose encoder produces multi-level, 3D-aware features at $1/16$ image resolution with 1024 channels. All MASt3R weights remain fixed throughout training and inference (Zheng et al., 12 May 2026).

Two lightweight adapters, each described as a pair of residual conv layers, up-sample and reshape ViT patches into $1/8$-resolution feature maps. These are used for two distinct roles: a flow feature extractor for the correlation volume and a context feature extractor for the ConvGRU state. This separation preserves the use of frozen 3D-aware features while allowing trainable downstream components to specialize to iterative pose optimization (Zheng et al., 12 May 2026).

The 3D-aware update operator works on image pairs $(i,j)$ using the RGB images $I_i, I_j$ , the current disparity $d_i$ , previous optical flow $\tilde{\mathbf f}_{ij}$ , and a hidden state from the previous iteration. Its pipeline is executed for 15 iterations. At each iteration, WildPose extracts flow features, builds a 4D correlation volume, updates the hidden state through a ConvGRU, and decodes that state into updated flow $\hat{\mathbf f}_{ij}^{t+1}$ , confidence weights $\hat{\mathbf w}_{ij}^{t+1}$ , damping factors $\hat \eta^{t+1}$ , and disparity-upsampling masks $\hat{\mathbf u}^{t+1}$ . A Gauss–Newton step then refines the camera poses $1/8$0, disparities $1/8$1, and flows $1/8$2 (Zheng et al., 12 May 2026).

A separate motion-mask detector operates on edges in the frame graph rather than on isolated frames. For each new edge $1/8$3, it predicts a mask $1/8$4 that identifies pixels whose motion cannot be explained by pure ego-motion. The detector fuses multi-level decoder tokens from MASt3R via a DPT-style fusion layer and uses a CNN-based motion head to produce $1/8$5. The stated advantage is that it masks moving objects only when they are inconsistent between $1/8$6 and $1/8$7, thereby avoiding the “global object alive” ambiguity of per-frame segmentation (Zheng et al., 12 May 2026).

3. Optimization formulation

WildPose formulates inference as a weighted reprojection plus depth-prior bundle adjustment problem. The objective is

$1/8$8

where $1/8$9 is the flow induced by the current $(i,j)$ 0, $(i,j)$ 1 is the metric depth prior from MoGe-2, and $(i,j)$ 2 regularizes disparity toward $(i,j)$ 3. The weighting term is defined by

$(i,j)$ 4

so the BA residuals are modulated jointly by predicted flow confidence and the learned motion mask (Zheng et al., 12 May 2026).

The paper also gives a more general robust BA form,

$(i,j)$ 5

where $(i,j)$ 6 is a robustifier such as Huber and $(i,j)$ 7 is a pose-regularization term. This suggests that the WildPose objective can be interpreted as a concrete instantiation of robust geometric optimization in which dynamic-scene uncertainty is pushed directly into the weighting structure rather than handled only by post hoc outlier rejection (Zheng et al., 12 May 2026).

The Gauss–Newton update is written as

$(i,j)$ 8

with $(i,j)$ 9 the Jacobian of residuals with respect to pose parameters, $I_i, I_j$ 0 the weight matrix including $I_i, I_j$ 1 and $I_i, I_j$ 2, and $I_i, I_j$ 3 a damping term. In WildPose, this classical optimizer is embedded in an iterative learned pipeline rather than used as a purely hand-designed backend (Zheng et al., 12 May 2026).

4. Training curriculum and parameterization

Training follows a three-stage curriculum. Stage 1 pretrains the update operator on static sequences from TartanAir V2, TartanGround, and Kubric static. The loss is

$I_i, I_j$ 4

where $I_i, I_j$ 5 is geodesic pose error, $I_i, I_j$ 6 is an $I_i, I_j$ 7 loss between ground-truth and geometry-induced flow, and $I_i, I_j$ 8 is an $I_i, I_j$ 9 loss between predicted $d_i$ 0 and $d_i$ 1. Data augmentation includes color jitter, grayscale, blur, and center resize plus crop to $d_i$ 2, with per-pair flow magnitude constrained to $d_i$ 3 pixels. This stage is trained for 200 K iterations with learning rate $d_i$ 4 (Zheng et al., 12 May 2026).

Stage 2 finetunes the update operator on mixed static and dynamic data by adding Dynamic Replica, OmniWorld-Game, and Kubric dynamic. Ground-truth motion masks $d_i$ 5 are injected into the BA weights. This stage runs for 100 K iterations with learning rate $d_i$ 6 (Zheng et al., 12 May 2026).

Stage 3 trains the motion mask detector while freezing the update operator and the backbone. The loss is

$d_i$ 7

where $d_i$ 8 is binary cross-entropy with available annotated masks. This stage is trained for 15 K iterations with learning rate $d_i$ 9 (Zheng et al., 12 May 2026).

The backbone remains frozen throughout. Only the lightweight adapters, dual encoders for flow and context, ConvGRU and decoders in the update operator, and the motion head are trained. This fixed-backbone design is integral to the framework’s stated synthesis of large-scale pre-trained 3D-aware features with differentiable BA (Zheng et al., 12 May 2026).

5. Empirical performance

WildPose is evaluated on dynamic, static, and low-ego-motion benchmarks. On Wild-SLAM MoCap, measured by ATE RMSE in centimeters, WildPose reports $\tilde{\mathbf f}_{ij}$ 0 average, compared with $\tilde{\mathbf f}_{ij}$ 1 for WildGS-SLAM, $\tilde{\mathbf f}_{ij}$ 2 for MegaSaM, and $\tilde{\mathbf f}_{ij}$ 3 for ViPE, and it is reported as best on all 10 sequences. On Bonn RGB-D Dynamic it reports $\tilde{\mathbf f}_{ij}$ 4, listed as second best behind WildGS at $\tilde{\mathbf f}_{ij}$ 5, and ahead of MegaSaM at $\tilde{\mathbf f}_{ij}$ 6 and ViPE at $\tilde{\mathbf f}_{ij}$ 7. On TUM Dynamic it reports $\tilde{\mathbf f}_{ij}$ 8, ahead of WildGS at $\tilde{\mathbf f}_{ij}$ 9, MegaSaM at $\hat{\mathbf f}_{ij}^{t+1}$ 0, and ViPE at $\hat{\mathbf f}_{ij}^{t+1}$ 1 (Zheng et al., 12 May 2026).

On low-ego-motion and static benchmarks, the same system remains competitive. On MPI Sintel, using scale-normalized ATE in meters, WildPose reports $\hat{\mathbf f}_{ij}^{t+1}$ 2, compared with $\hat{\mathbf f}_{ij}^{t+1}$ 3 for MegaSaM and $\hat{\mathbf f}_{ij}^{t+1}$ 4 for DROID. On TUM static it reports $\hat{\mathbf f}_{ij}^{t+1}$ 5 and is described as best. On 7-Scenes it reports $\hat{\mathbf f}_{ij}^{t+1}$ 6, described as second best by $\hat{\mathbf f}_{ij}^{t+1}$ 7 (Zheng et al., 12 May 2026).

Benchmark	WildPose	Comparative note
Wild-SLAM MoCap	0.39 cm	best on all 10 sequences
Bonn RGB-D Dynamic	2.36 cm	2nd best
TUM Dynamic	1.57 cm	best
MPI Sintel	0.017 m	best
TUM static	0.027 m	best
7-Scenes	0.049 m	2nd best by 0.001 m

The paper also distinguishes WildPose from keyframe-only methods such as MASt3R-SLAM, VGGT, and $\hat{\mathbf f}_{ij}^{t+1}$ 8, noting that such methods are inherently easier, while claiming that WildPose still outperforms them on dynamic data and matches them on static scenes (Zheng et al., 12 May 2026).

Runtime and memory measurements on Wild-SLAM sequences with an RTX 4090 place WildPose at $\hat{\mathbf f}_{ij}^{t+1}$ 9 FPS and $\hat{\mathbf w}_{ij}^{t+1}$ 0 GiB, compared with $\hat{\mathbf w}_{ij}^{t+1}$ 1 FPS and $\hat{\mathbf w}_{ij}^{t+1}$ 2 GiB for WildGS-SLAM, $\hat{\mathbf w}_{ij}^{t+1}$ 3 FPS and $\hat{\mathbf w}_{ij}^{t+1}$ 4 GiB for MegaSaM, $\hat{\mathbf w}_{ij}^{t+1}$ 5 FPS and $\hat{\mathbf w}_{ij}^{t+1}$ 6 GiB for DROID-SLAM, and $\hat{\mathbf w}_{ij}^{t+1}$ 7 FPS and $\hat{\mathbf w}_{ij}^{t+1}$ 8 GiB for ViPE. The reported interpretation is that WildPose strikes a balance: it is far faster than per-sequence optimizers and more accurate on dynamic scenes than lightweight SLAM systems (Zheng et al., 12 May 2026).

6. Ablations, limitations, and relation to broader pose-estimation research

The ablation study isolates the effects of mixed dynamic finetuning, motion masking, and depth regularization. Average ATE in centimeters on Wild-SLAM, Bonn, and TUM-Dyn is reported as $\hat{\mathbf w}_{ij}^{t+1}$ 9 for the baseline without mixed finetuning and without masks; $\hat \eta^{t+1}$ 0 for mixed finetuning only; $\hat \eta^{t+1}$ 1 for mask only; and $\hat \eta^{t+1}$ 2 for the combination of mixed finetuning and mask while keeping depth regularization in the final global BA, which is the best setting in the table. Removing the final depth prior is reported to slightly degrade static performance. The authors further state that pre-training on static data is critical, mixing dynamic data yields large gains, and the motion mask detector gives consistent improvements on all dynamic benchmarks, including $\hat \eta^{t+1}$ 3 on Wild-SLAM (Zheng et al., 12 May 2026).

These results clarify two points often blurred in discussions of dynamic-scene SLAM. First, the motion mask is not a semantic segmentation proxy; it is edge-dependent and enters directly into the BA weighting. Second, depth regularization is not presented as a universal constraint to keep enabled at every stage; it improves scale drift and low-ego-motion accuracy but can be switched off for final global BA to refine poses (Zheng et al., 12 May 2026).

Within the broader research landscape, WildPose belongs to a family of “pose in the wild” methods, but it solves a distinct problem. Related work has addressed lifting animal 2D keypoints to 3D for avatar retargeting from a single camera in the wild (Debnath et al., 2 Jan 2025), direct reward fine-tuning for single-image to 3D human reconstruction with challenging poses (Do et al., 3 Mar 2026), and shape-conditioned arbitrary-object pose estimation that generalizes to new object types and even animals (Xiao et al., 2019). This suggests that WildPose occupies the monocular camera-pose and trajectory-estimation regime, whereas those methods focus on articulated body reconstruction or relative object pose.

Taken together, WildPose is characterized by a hybrid design—frozen 3D-aware perceptual features, learned differentiable BA updates, and explicit motion-mask weighting—that targets robust monocular pose estimation across highly dynamic, purely static, and low-motion sequences (Zheng et al., 12 May 2026).