HyperDiff: 3D Human Pose Estimation

• HyperDiff is a 3D human pose estimation method that combines a denoising diffusion model with a multi-granularity Hypergraph Convolutional Network to address occlusion and depth ambiguity.
• It leverages DDIM-style iterative sampling and hypergraph-based representations to capture both local and global skeletal joint correlations efficiently.
• Evaluations on Human3.6M and MPI-INF-3DHP demonstrate state-of-the-art performance, offering configurable accuracy–efficiency trade-offs in real-time scenarios.

HyperDiff is a 3D human pose estimation method that integrates a denoising diffusion probabilistic model with a multi-granularity Hypergraph Convolutional Network (HyperGCN) for monocular 2D-to-3D pose lifting. It is formulated to address depth ambiguity and occlusion during the lifting process while explicitly modeling high-order joint correlations through skeleton structures at multiple scales. In the reported system, a HyperGCN-based denoiser predicts the clean 3D pose from a noised pose conditioned on 2D keypoints, and DDIM-style iterative sampling produces the final reconstruction at inference. The method is evaluated on Human3.6M and MPI-INF-3DHP, where the reported results show state-of-the-art performance together with configurable accuracy–efficiency trade-offs (Han et al., 20 Aug 2025).

1. Problem setting and conceptual scope

HyperDiff is situated in monocular 3D human pose estimation, specifically the 2D-to-3D lifting setting in which an input pose $x$ in image coordinates is mapped to a 3D pose $y_0$. The method is motivated by two difficulties stated explicitly in its formulation: depth ambiguity and occlusion, both of which can degrade the reliability of direct regression from 2D joints to 3D joints. A second stated motivation is that traditional methods may overlook multi-scale skeleton features when utilizing skeleton structure information, which can negatively impact pose-estimation accuracy (Han et al., 20 Aug 2025).

The framework combines two components. The first is a diffusion model, used to capture data uncertainty during denoising. The second is HyperGCN, used as the denoiser and designed around multi-granularity skeletal structures so that high-order joint correlations can be modeled more accurately, particularly for complex poses. This division of labor is central to the method’s identity: diffusion handles stochastic ambiguity, while the hypergraph architecture injects structured inductive bias over the human body (Han et al., 20 Aug 2025).

The paper presents HyperDiff as the first integration of diffusion models with a multi-granularity Hypergraph GCN for 3D human pose lifting. This suggests that its novelty is not diffusion alone, nor hypergraph reasoning alone, but the coupling of iterative denoising with a hierarchy of skeletal relations spanning local and global body structure (Han et al., 20 Aug 2025).

2. Diffusion formulation for pose lifting

HyperDiff follows a DDPM-style forward noising process with a cosine noise schedule $\{\beta_t\}_{t=1}^T$. If $y_0$ denotes the ground-truth 3D pose, then the forward transition is

$$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$

with the closed form

$$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$

This provides a standard corruption path from a clean pose to progressively noisier states (Han et al., 20 Aug 2025).

The reverse process differs from the common noise-prediction parameterization in that the denoiser directly predicts the clean pose:

$\hat y_0 = \mathfrak{D}(y_t, x, t).$

Here, $\mathfrak{D}$ is conditioned jointly on the noised 3D pose $y_t$, the input 2D pose $x$, and the diffusion timestep $y_0$0. At inference, HyperDiff follows DDIM sampling rather than a purely DDPM ancestral scheme, iteratively updating the sample from Gaussian noise $y_0$1 toward a final 3D pose estimate. The paper also states that multiple hypotheses $y_0$2 and sampling iterations $y_0$3 can be run and aggregated (Han et al., 20 Aug 2025).

Training uses a single Mean Squared Error objective on the clean pose estimate:

$y_0$4

The published version does not add hypergraph-specific regularization terms. This is a notable design choice because it places all structural supervision in the denoiser architecture rather than in auxiliary losses (Han et al., 20 Aug 2025).

3. Multi-granularity hypergraph representation of the skeleton

The skeleton is represented with vertices $y_0$5, where each vertex corresponds to a body joint. HyperDiff constructs three structural scales: a joint-scale graph, a part-scale hypergraph, and a body-scale hypergraph. The part-scale hypergraph $y_0$6 contains hyperedges $y_0$7, each grouping a local chain of joints. The body-scale hypergraph $y_0$8 contains hyperedges $y_0$9, each grouping a larger body region (Han et al., 20 Aug 2025).

Scale	Construction
Joint scale	Standard adjacency $\{\beta_t\}_{t=1}^T$0 connecting physically adjacent joints
Part scale	Hypergraph $\{\beta_t\}_{t=1}^T$1 with hyperedges $\{\beta_t\}_{t=1}^T$2 grouping local chains
Body scale	Hypergraph $\{\beta_t\}_{t=1}^T$3 with hyperedges $\{\beta_t\}_{t=1}^T$4 grouping large regions

The paper gives representative examples. At part scale, one hyperedge groups $\{\beta_t\}_{t=1}^T$5, while another groups $\{\beta_t\}_{t=1}^T$6. At body scale, one hyperedge groups $\{\beta_t\}_{t=1}^T$7. These constructions formalize the intuition that human pose is constrained simultaneously by immediate kinematic adjacency, local limb composition, and larger anatomical assemblies (Han et al., 20 Aug 2025).

For a chosen hypergraph, the incidence matrix is $\{\beta_t\}_{t=1}^T$8. The normalization terms are

$\{\beta_t\}_{t=1}^T$9

These support the symmetric normalization used in hypergraph convolution. The stated rationale is that multi-granularity structures improve denoising capability, especially for complex poses, by capturing high-order relations that a simple joint graph can miss (Han et al., 20 Aug 2025).

4. HyperGCN denoiser architecture

The denoiser takes a noisy pose $y_0$0 and concatenates it with 2D keypoints $y_0$1, producing $y_0$2 inputs. A linear embedding lifts these features to model dimension $y_0$3, after which learnable timestep and spatial embeddings are added to obtain $y_0$4. The network then stacks $y_0$5 HyperGCN blocks and projects the final representation to $y_0$6 (Han et al., 20 Aug 2025).

Within a single block, three parallel branches are computed from $y_0$7. The joint-scale branch applies a graph convolution with normalized adjacency

$y_0$8

The part-scale branch applies hypergraph convolution

$y_0$9

and the body-scale branch is defined analogously with its own hypergraph. The three outputs are fused by a learnable weighted sum,

$$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$0

followed by a residual connection, batch normalization, and ReLU:

$$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$1

This block design operationalizes the paper’s claim that joint-scale, part-scale, and body-scale hyperedges capture complementary local and global correlations (Han et al., 20 Aug 2025).

The published implementation uses embedding dimension $$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$2 and $$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$3 HyperGCN blocks. The diffusion process uses $$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$4 timesteps with a cosine schedule. These settings place HyperDiff in the regime of full diffusion training rather than shallow iterative refinement (Han et al., 20 Aug 2025).

5. Experimental protocol, results, and efficiency

HyperDiff is evaluated on Human3.6M and MPI-INF-3DHP. For Human3.6M, training uses subjects 1, 5, 6, 7, and 8, and testing uses subjects 9 and 11. Metrics are MPJPE and P-MPJPE. For MPI-INF-3DHP, training uses 8 actors and testing uses the evaluation set; metrics are MPJPE, PCK@150 mm, and AUC. Optimization uses AdamW with learning rate $$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$5, weight decay $$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$6, and 50 training epochs on 8 A100 GPUs (Han et al., 20 Aug 2025).

Dataset / setting	Sampling config	Reported result
Human3.6M, detected 2D input	$$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$7	MPJPE 46.8 mm
Human3.6M, detected 2D input	$$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$8	MPJPE 46.0 mm
Human3.6M, GT 2D input	$$q(y_t \mid y_{t-1}) = \mathcal{N}\bigl(y_t;\,\sqrt{1-\beta_t}\,y_{t-1},\,\beta_t I\bigr),$$9	P-MPJPE 28.6 mm
Human3.6M, GT 2D input	$$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$0	P-MPJPE 28.1 mm
MPI-INF-3DHP	$$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$1	PCK 87.6%, AUC 57.0%, MPJPE 69.2 mm
MPI-INF-3DHP	$$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$2	PCK 88.4%, AUC 58.7%, MPJPE 68.5 mm

The paper characterizes the Human3.6M result of MPJPE 46.0 mm at $$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$3 as state of the art among non-image based methods. It also reports that with $$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$4, HyperDiff runs at approximately $$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$5 k FPS with $$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$6 M parameters and $$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$7 GFLOPs while obtaining MPJPE $$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$8 mm. Even with richer sampling such as $$y_t = \sqrt{\bar\alpha_t}\,y_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s), \quad \epsilon\sim\mathcal{N}(0,I).$$9, the method is reported to outperform the diffusion-based D3DP baseline in accuracy while preserving real-time flexibility under simpler settings (Han et al., 20 Aug 2025).

Qualitatively, the reported behavior is that HyperDiff recovers correct limb depths under severe self-occlusion on Human3.6M and handles faulty 2D detections, incomplete keypoints, and rare poses on in-the-wild videos. The paper attributes this robustness to the combination of uncertainty-aware diffusion and structure-aware HyperGCN denoising (Han et al., 20 Aug 2025).

6. Ablation findings, interpretation, and nomenclature

Ablation studies isolate the contribution of hypergraph granularity, feature-fusion strategy, and sampling configuration. On Human3.6M with the single-hypothesis setting, the paper reports the following MPJPE values for structural granularity (Han et al., 20 Aug 2025):

Configuration	MPJPE
Joint scale only	49.5 $\hat y_0 = \mathfrak{D}(y_t, x, t).$0
+ Part scale	48.2 $\hat y_0 = \mathfrak{D}(y_t, x, t).$1
+ Body scale	47.6 $\hat y_0 = \mathfrak{D}(y_t, x, t).$2
Joint + Part + Body	46.8 $\hat y_0 = \mathfrak{D}(y_t, x, t).$3

The fusion study reports $\hat y_0 = \mathfrak{D}(y_t, x, t).$4 mm for concatenation, $\hat y_0 = \mathfrak{D}(y_t, x, t).$5 mm for element-wise product, and $\hat y_0 = \mathfrak{D}(y_t, x, t).$6 mm for a learnable weighted sum. The paper also states that increasing the number of hypotheses $\hat y_0 = \mathfrak{D}(y_t, x, t).$7 and sampling iterations $\hat y_0 = \mathfrak{D}(y_t, x, t).$8 steadily improves MPJPE at the cost of FLOPs and latency. These results support the interpretation that the joint, part, and body branches provide complementary signals and that learnable weighted fusion is the most effective of the tested fusion rules (Han et al., 20 Aug 2025).

The article’s main interpretive claim is that hypergraph branches capture complementary local versus global joint correlations, while DDIM sampling with a learned HyperGCN denoiser enables flexible movement between real-time and high-precision regimes. A plausible implication is that HyperDiff’s performance depends not only on denoising capacity but also on how faithfully the skeletal hierarchy is encoded into the denoiser itself (Han et al., 20 Aug 2025).

A common source of confusion is nomenclature. HyperDiff should be distinguished from “HyperDiffusion,” which generates implicit neural fields by diffusing directly in MLP weight space for 3D shapes and 4D mesh animations (Erkoç et al., 2023); from “NeuMaDiff,” which applies hyperdiffusion to neural-field BRDF weights for material synthesis (Zhou et al., 2024); and from “HypDiff,” which uses hyperbolic geometric latent diffusion for graph generation (Fu et al., 2024). The shared lexical element “HyperDiff” therefore denotes different technical ideas across subfields: multi-granularity hypergraph denoising for pose lifting in HyperDiff, weight-space diffusion for implicit representations in HyperDiffusion and NeuMaDiff, and hyperbolic latent diffusion for graphs in HypDiff.