Again-Pose: Anchor-Guided Adaptive Inter-Frame Motion Cues Propagating for High-quality Human Pose Reconstruction

Abstract: Reconstructing continuous 3D human poses from unconstrained videos is challenging, especially in extreme motion scenarios involving severe motion blur and occlusion. Current state-of-the-art methods typically rely on implicit temporal attention to aggregate features across frames. However, under severe visual degradation, input features often suffer from collapse, rendering them indistinguishable from noise. In such cases, implicit aggregation fails to distinguish valid signals, leading to catastrophic reconstruction errors. To address this robustness gap, we propose a simple yet effective framework called Anchor-guided adaptive inter-frame motion cues propagating (Again-Pose), reformulating pose estimation in degraded frames as a motion-guided recovery task. Instead of blindly smoothing features, we explicitly identify high-quality Anchor Frames based on feature saliency and propagate reliable kinematic cues to "inpaint" the poses of degraded intermediate frames. Specifically, a Dual-path Motion-aware Module captures fine-grained inter-frame dynamics, while a Difference-weighted Fusion Module adaptively propagates these cues to suppress drift. Extensive experiments on standard benchmarks (Human3.6M, 3DPW, PoseTrack) and the challenging FineDiving dataset demonstrate that Again-Pose significantly outperforms state-of-the-art methods in robustness and stability, effectively recovering plausible poses where other methods fail.

• The paper introduces Again-Pose, an anchor-guided framework that identifies reliable frames, propagates inter-frame SMPL motion cues, and reconstructs poses degraded by blur, occlusion, and rapid movement.
• The method achieves 70.7 mm MPJPE and 44.5 mm PA-MPJPE on 3DPW without 3DPW training, improving substantially over HMR 2.0 while adding only about 0.6 GFLOPs per frame with cached features.
• The results show that explicit motion recovery improves downstream FineDiving action-quality assessment, although robustness without reliable anchors and sensitivity to hand-tuned temporal parameters remain open challenges.

Motivation and problem statement

Reconstructing continuous 3D human pose sequences from unconstrained video remains fragile under extreme motion conditions such as high-speed diving, gymnastics, and figure skating, where severe motion blur, abrupt pose changes, and self-occlusion degrade per-frame visual features. The paper's central diagnosis is that state-of-the-art video-based methods (VIBE, GLoT, TRAM) rely on implicit temporal attention to aggregate features across frames, an operation that presupposes the input features are reliable. When blurry frames undergo "feature collapse"—visual cues becoming indistinguishable from noise—implicit aggregation cannot separate valid signals from corrupted ones and propagates errors across the sequence, producing collapsed or chaotic SMPL meshes (2606.29230).

Again-Pose reframes pose estimation in degraded frames as a motion-guided recovery (inpainting) task: explicitly identify high-fidelity Anchor Frames, regress accurate SMPL parameters for them, and propagate kinematic cues through inter-frame dynamics to reconstruct degraded intermediate frames.

Method

The framework builds on HMR 2.0's ViT-H backbone (frozen during training) and comprises three components:

• Intelligent Anchor-frame Selector: computes a per-frame quality score combining a temporal-softmax L2-norm saliency term $s_{base}$ with a learnable dynamic weight $w_{dynamic}$ predicted by an MLP from the Transformer decoder token ($\lambda = 0.3$). A filter-and-fill algorithm selects TOP-K candidates subject to a MIN-DISTANCE constraint, with recursive filling of excessively long blank intervals to bound interpolation error.
• Dual-path Motion-aware Module: Path One regresses SMPL parameter differences ($\Delta\theta, \Delta\beta, \Delta\pi$) from feature pairs via cross-attention; Path Two (training only) aligns these difference representations with I3D appearance features through cascaded cross/self-attention supervised by a Temporal Matching Loss.
• Difference-weighted Fusion Module: accumulates differences bidirectionally from adjacent anchors; within overlap regions (OVERLAP-SIZE), forward and backward predictions are fused with distance-based weights after conversion to continuous 6D rotation representation to avoid interpolation artifacts on rotation matrices.

Training uses full-sequence supervision—single-frame losses on all frames and difference losses on all consecutive pairs—plus five loss terms (3D/2D joint L1, SMPL parameter MSE, Temporal Matching Smooth-L1, and an Anchor-weight loss coupling dynamic weights to anchor reconstruction error).

Quantitative results

Under strict cross-dataset generalization (no 3DPW training), Again-Pose reports 70.7 mm MPJPE / 44.5 mm PA-MPJPE on 3DPW versus 81.3/54.3 for HMR 2.0—an improvement of roughly 10 mm over the previous best non-fine-tuned approach—and matches single-frame SOTA on PoseTrack ([email protected] of 0.90). On Human3.6M it reaches 48.8/30.7, surpassing both single-frame baselines and video methods including VIBE (93.5/56.5 on 3DPW) and GLoT (80.7*/50.6*).

Benchmark	Metric	HMR 2.0	GLoT	Again-Pose
3DPW	MPJPE ↓	81.3	80.7*	70.7
3DPW	PA-MPJPE ↓	54.3	50.6*	44.5
Human3.6M	MPJPE ↓	50.0	67.0	48.8
Human3.6M	PA-MPJPE ↓	32.4	46.3	30.7

On the downstream AQA task using FineDiving with the pose-based pipeline of Zhu et al., Again-Pose attains $\rho$ = 0.9503, $R_{\ell2}$ = 0.2113, and [email protected] = 0.9902, exceeding the base method and pose variants built on HMR 2.0 or HSMR. Notably, substituting HMR 2.0 poses degrades AQA relative to the base model due to blur-induced sudden pose changes—a result underscoring that per-frame accuracy does not transfer to temporal stability in extreme motion.

Ablations on Human3.6M confirm each component's contribution: removing the I3D visual supervision path causes the largest degradation (55.2 vs. 48.8 MPJPE), followed by removing full-sequence supervision (52.4), replacing distance-weighted fusion with hard switching (49.9), average fusion (49.5), uniform selection (50.8), and random selection (51.2). Hyperparameter analysis shows sharp sensitivity: MIN-DISTANCE = 6 degrades MPJPE to 58.6 mm, while TOP-K requires a trade-off between anchor quality and coverage (optimal at 6 for 16-frame inputs).

Efficiency

With ViT feature caching, the temporal modules add only ~0.6 GFLOPs per frame (~0.48% overhead) and +70M parameters over the 680M baseline, since the backbone processes each frame exactly once. The authors position the method for offline high-fidelity applications (AQA, film production) rather than real-time use.

Limitations and open questions

The paper concedes several constraints. Reconstruction is "not yet flawless" in the most extreme frames, and qualitative superiority over baselines rests partly on visual inspection rather than quantitative metrics on FineDiving poses. Hyperparameters (TOP-K, MIN-DISTANCE, OVERLAP-SIZE, $\lambda$) are tuned empirically and their interaction with sequence length beyond the fixed 16-frame setting is unexplored. The anchor selection heuristic depends on feature saliency being informative under degradation—the very condition it is meant to diagnose—and the paper does not analyze failure cases when all frames in a window are blurred. Finally, evaluation of the fusion strategy against learned propagation alternatives (e.g., diffusion-based inpainting) is absent, leaving open whether explicit geometric weighting remains optimal as sequence length grows.

Conclusion

Again-Pose replaces implicit temporal aggregation with explicit anchor identification and motion-cue propagation, reformulating degraded-frame pose estimation as recovery rather than smoothing. It delivers state-of-the-art cross-dataset generalization results—notably a ~10 mm MPJPE gain on 3DPW—and improved stability on FineDiving-driven AQA, at negligible computational overhead. Its main open questions concern robustness when no reliable anchors exist and generalization of its hand-tuned selection hyperparameters.