Depth-Consistent Human Modeling (DCHM)

• Depth-Consistent Human Modeling (DCHM) is a framework that ensures human geometry reconstruction remains aligned with depth observations across varying views, poses, and time.
• It employs diverse representations—from double-depth maps and neural SDFs to radiance fields—to enforce per-ray, canonical, and temporal consistency in 3D human models.
• DCHM underpins multiview pedestrian detection by integrating depth-consistent reconstruction to improve metric accuracy and localization in crowded surveillance scenes.

Depth-Consistent Human Modeling (DCHM) denotes a family of problems and methods in which human geometry is reconstructed, animated, or localized so that it remains consistent with depth observations across views, poses, and time. In the cited literature, the term appears both as a general objective and as the name of a specific multiview pedestrian-detection framework. Across these usages, depth consistency can mean per-ray front/back coherence, agreement between fused partial surfaces over time, canonical-space alignment of monocular depth observations, cross-view consistency in global coordinates, or metric and temporal stability in monocular video human mesh recovery (Ma et al., 19 Jul 2025).

1. Emergence and scope of the concept

An early depth-sensor formulation appears in "Deformable Modeling for Human Body Acquired from Depth Sensors" (Vegeshna, 2017), which presents an approach to reconstruct complete 3D deformable models over time by a single depth camera. Its abstract describes a pipeline in which partial surfaces reconstructed from various times of capture are assembled together to form a complete 3D surface, a mesh warping algorithm aligns different partial surfaces based on linear mesh deformation, and a volumetric method combines partial surfaces, fixes missing holes, and smooths alignment errors. The associated arXiv entry currently provides no PDF or source, so the paper’s specific equations, algorithmic parameters, and quantitative results are not accessible (Vegeshna, 2017).

A complementary single-image formulation appears in "Moulding Humans: Non-parametric 3D Human Shape Estimation from Single Images" (Gabeur et al., 2019). There, a full 3D human shape is encoded by a visible/front depth map and a hidden/back depth map, and the shape is reconstructed as a ray-wise “mould.” This representation treats the occupied interval along each camera ray as the segment bounded by the two depths, yielding a per-ray watertight encoding that is image-aligned and substantially lower-dimensional than voxel grids while remaining non-parametric enough to capture clothing, hair, and other un-modelled elements (Gabeur et al., 2019).

Later work generalized the notion beyond explicit depth-map fusion. "MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images" (Wang et al., 2021) frames the problem as learning controllable neural signed distance fields for clothed humans from monocular depth observations, while "Neural Surface Fields for Human Modeling from Monocular Depth" (Xue et al., 2023) formulates a continuous displacement field on a canonical base surface. This suggests that DCHM is best understood not as a single architecture, but as a unifying constraint: human geometry should remain faithful to observed depth while staying coherent under deformation, animation, and viewpoint change (Wang et al., 2021).

2. What “depth consistency” means

The literature assigns several precise meanings to depth consistency. In monocular video human mesh recovery, one formulation decomposes it into three facets: along-ray correctness, metric scale consistency, and temporally coherent dynamics. Along-ray correctness addresses failures in depth ordering, such as limbs that “pop behind” the torso; metric scale consistency addresses drift in bone lengths and person-camera distance; and temporally coherent dynamics addresses jitter without collapsing motion detail through over-smoothing (Cen et al., 4 Feb 2026).

In sparse-view neural reconstruction, depth consistency is also a corrective to RGB-only ambiguity. "HumanRecon: Neural Reconstruction of Dynamic Human Using Geometric Cues and Physical Priors" (Yin et al., 2023) argues that methods relying only on RGB color supervision are more prone to overfitting to color and to geometric ambiguity along rays. Its design therefore binds the rendered termination depth to explicit pseudo-supervision from monocular depth and normals, and further constrains the density field by maximizing density at the ray-surface intersection. In this formulation, depth consistency is not merely a smoother output; it is a geometric condition linking volumetric rendering, surface location, and local orientation (Yin et al., 2023).

A related monocular HMR interpretation appears in "Distribution and Depth-Aware Transformers for 3D Human Mesh Recovery" (Bright et al., 2024). That method does not define an explicit depth reconstruction loss, but fuses pseudo-depth with image features through a dual-stream transformer and measures the effect directly on the recovered mesh’s $z$-axis error. On 3DPW, the reported ablation gives $\mathrm{mPJPE}(z)=69.1$ and $\mathrm{PA\text{-}mPJPE}(z)=58.3$ without depth modeling, versus $\mathrm{mPJPE}(z)=65.4$ and $\mathrm{PA\text{-}mPJPE}(z)=53.6$ with depth modeling, making depth consistency an explicitly measured property of the recovered 3D geometry (Bright et al., 2024).

3. Representational families

Different DCHM systems impose consistency in different spaces: along rays, in canonical coordinates, on an extracted surface, or inside a radiance or Gaussian representation.

Setting	Representation	Consistency mechanism
Single-image non-parametric shape	Visible/front and hidden/back depth maps	Ray-wise occupancy bounded by two depths; Poisson reconstruction (Gabeur et al., 2019)
Few-shot animatable avatars	Controllable neural SDF with implicit forward/inverse skinning and a pose-conditioned hypernetwork	Canonicalization by inverse LBS; IGR on canonicalized depth points (Wang et al., 2021)
Monocular depth sequences	Neural field defined solely on a canonical base surface	Canonical fusion, continuous surface displacement, and shared mesh connectivity across frames (Xue et al., 2023)
Sparse multi-view dynamic reconstruction	Canonical radiance field with SMPL-anchored deformations	Depth and normal pseudo-supervision plus surface-density prior (Yin et al., 2023)

The double-depth representation of "Moulding Humans" (Gabeur et al., 2019) is the most direct ray-space instantiation. If $D_{\text{vis}}(u,v)$ and $D_{\text{hid}}(u,v)$ are the visible and hidden depths at pixel $(u,v)$, then the occupied segment along that ray is the interval between them. The details block writes this explicitly as

$$O(u,v,z)=1 \;\; \text{if} \;\; D_{\text{vis}}(u,v)\le z \le D_{\text{hid}}(u,v),$$

with front and back surface point sets obtained by back-projecting the two depth maps. The representation is therefore depth-consistent by construction at the per-ray level (Gabeur et al., 2019).

In MetaAvatar, consistency is imposed in canonical space rather than image space. A clothed human is represented by an SDF $f_\theta(x):[-1,1]^3\to \mathbb{R}$, while observed depth points are canonicalized through inverse linear blend skinning with learned implicit skinning networks $\mathrm{mPJPE}(z)=69.1$0 and $\mathrm{mPJPE}(z)=69.1$1. The method then applies Implicit Geometric Regularization so that canonicalized on-surface samples are driven to the zero level set and off-surface samples regularize empty space, while a pose-conditioned hypernetwork $\mathrm{mPJPE}(z)=69.1$2 maps bone transformations $\mathrm{mPJPE}(z)=69.1$3 to the SDF parameters $\mathrm{mPJPE}(z)=69.1$4 (Wang et al., 2021).

NSF relocates the field from the ambient volume to the canonical surface itself. It first learns a subject-specific canonical fusion shape $\mathrm{mPJPE}(z)=69.1$5 as an SDF, discretizes it once, and then defines a pose-conditioned displacement field on the base surface:

$\mathrm{mPJPE}(z)=69.1$6

followed by linear blend skinning

$\mathrm{mPJPE}(z)=69.1$7

Because the same canonical mesh connectivity is used at all frames, per-frame surface extraction is eliminated and vertex correspondences persist over time, which the paper identifies as the basis of mesh coherency (Xue et al., 2023).

HumanRecon keeps a radiance-field formulation but injects explicit geometry. Its expected termination depth is supervised by pseudo-depth, normals are obtained from density gradients, and the surface prior is written as

$\mathrm{mPJPE}(z)=69.1$8

This ties the ray depth selected by the renderer to a high-density surface location and suppresses multi-peak ambiguities along rays (Yin et al., 2023).

4. Monocular, temporal, and metric formulations

Several recent methods extend DCHM from static reconstruction to temporally stable and metrically meaningful video geometry. "GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion" (Kim et al., 29 May 2025) decomposes the problem into an Image-to-Geometry estimator for the first frame and a Video-to-Geometry estimator that propagates geometry through an image-to-video diffusion model conditioned on the first-frame geometry. Its key representation is root-relative depth,

$\mathrm{mPJPE}(z)=69.1$9

which preserves human-scale information while discarding global translation. On ActorsHQ, the reported metric-depth results include $\mathrm{PA\text{-}mPJPE}(z)=58.3$0 for moving subjects and $\mathrm{PA\text{-}mPJPE}(z)=58.3$1 for moving camera sequences, while for affine-invariant depth the reported temporal metrics include OPW $\mathrm{PA\text{-}mPJPE}(z)=58.3$2 and TC-RMSE $\mathrm{PA\text{-}mPJPE}(z)=58.3$3 (Kim et al., 29 May 2025).

In single-image HMR, D2A-HMR uses pseudo-depth more indirectly. The model tokenizes both image and pseudo-depth features, applies self-attention to each stream, uses cross-attention to couple them, and fuses them through learnable weights:

$\mathrm{PA\text{-}mPJPE}(z)=58.3$4

A residual log-likelihood regularizer implemented with RealNVP is then used to align predicted and ground-truth mesh distributions. The method is not defined by an explicit depth loss, yet the reduction in $\mathrm{PA\text{-}mPJPE}(z)=58.3$5-axis error indicates improved depth consistency of the recovered mesh (Bright et al., 2024).

Temporal DCHM in monocular video HMR is addressed explicitly by the 2026 depth-guided framework (Cen et al., 4 Feb 2026). It introduces a Depth-Guided Multi-Scale Fusion module, a Depth-guided Metric-Aware Pose and Shape estimator, and a Motion-Depth Aligned Refinement module. Bone lengths are calibrated by confidence-weighted temporal statistics,

$\mathrm{PA\text{-}mPJPE}(z)=58.3$6

fused with template statistics to form $\mathrm{PA\text{-}mPJPE}(z)=58.3$7, and then used for scale-consistent initialization. On 3DPW, the paper reports MPJPE $\mathrm{PA\text{-}mPJPE}(z)=58.3$8, PA-MPJPE $\mathrm{PA\text{-}mPJPE}(z)=58.3$9, MPVPE $\mathrm{mPJPE}(z)=65.4$0, and Accel $\mathrm{mPJPE}(z)=65.4$1, with an ablation showing that DGMF, D-MAPS, and MoDAR are synergistic (Cen et al., 4 Feb 2026).

Metric consistency also becomes a scene-level problem in multi-person HMR. "Towards Metric-Aware Multi-Person Mesh Recovery by Jointly Optimizing Human Crowd in Camera Space" (Wang et al., 17 Nov 2025) defines an affine mapping from relative depth to metric depth,

$\mathrm{mPJPE}(z)=65.4$2

and solves for $\mathrm{mPJPE}(z)=65.4$3 jointly across all persons under anthropometric height priors in a MAP formulation. This yields DTO-Humans, described as $\mathrm{mPJPE}(z)=65.4$4M high-quality, scene-consistent multi-person images with an average of $\mathrm{mPJPE}(z)=65.4$5 persons per image. On Relative Human, PCDR$\mathrm{mPJPE}(z)=65.4$6(all) improves from $\mathrm{mPJPE}(z)=65.4$7 for CameraHMR to $\mathrm{mPJPE}(z)=65.4$8 for CameraHMR + DTO, and fine-tuned MA-HMR reaches $\mathrm{mPJPE}(z)=65.4$9 (Wang et al., 17 Nov 2025).

5. DCHM as a multiview pedestrian-detection framework

The paper explicitly titled "DCHM: Depth-Consistent Human Modeling for Multiview Detection" (Ma et al., 19 Jul 2025) defines DCHM as a label-free multiview pedestrian detection framework. Its problem setting is two-stage: human modeling followed by pedestrian localization. The inputs are calibrated, synchronized multiview images and 2D pedestrian segmentation masks; the outputs are consistent per-view depth maps, fused global point clouds and 3D Gaussians representing pedestrians, multiview segmentation with aligned IDs, and final pedestrian localizations (Ma et al., 19 Jul 2025).

Its central representation is superpixel-wise Gaussian Splatting. Each pedestrian mask is partitioned into $\mathrm{PA\text{-}mPJPE}(z)=53.6$0 superpixels per pedestrian. Superpixel-wise Gaussians are optimized under a foreground reconstruction objective

$\mathrm{PA\text{-}mPJPE}(z)=53.6$1

where the terms are a superpixel photometric loss, a mask loss, a depth-variance constraint within each pedestrian mask, and an opacity regularization. DCHM then enforces multiview depth consistency through cross-view foreground filtering and cross-view depth-consistency filtering, retains reliable pseudo-depth labels, fine-tunes an off-the-shelf monocular depth model, scales monocular depth to metric depth using ground calibration, back-projects depths into world coordinates, reconstructs Gaussians, propagates cross-view IDs by visibility-aware label matching, and localizes pedestrians by DBSCAN clustering followed by NMS (Ma et al., 19 Jul 2025).

The reported results position human modeling quality as the principal determinant of detection quality. On Wildtrack, under the same localization head (“Our Loc.”), DCHM Reconstruction yields MODA $\mathrm{PA\text{-}mPJPE}(z)=53.6$2 and MODP $\mathrm{PA\text{-}mPJPE}(z)=53.6$3, compared with Depth Pro at $\mathrm{PA\text{-}mPJPE}(z)=53.6$4 and $\mathrm{PA\text{-}mPJPE}(z)=53.6$5. In label-free multiview detection, DCHM reports $\mathrm{PA\text{-}mPJPE}(z)=53.6$6 on Wildtrack for MODA/MODP/Precision/Recall, compared with UMPD at $\mathrm{PA\text{-}mPJPE}(z)=53.6$7. On Terrace, DCHM reports $\mathrm{PA\text{-}mPJPE}(z)=53.6$8, and on MultiviewX $\mathrm{PA\text{-}mPJPE}(z)=53.6$9. The implementation details further report human-modeling construction in $D_{\text{vis}}(u,v)$0 seconds on an RTX 4090, a label-free inference speed of $D_{\text{vis}}(u,v)$1 FPS, and a three-cycle iterative loop as the practical accuracy-cost tradeoff (Ma et al., 19 Jul 2025).

This use of the acronym is narrower than the broader methodological sense found in monocular reconstruction and avatar literature. Here, DCHM is not primarily an avatar or mesh-animation system; it is a geometry-first detection framework in which depth-consistent reconstruction serves pedestrian localization in sparse-view, large-scale, crowded surveillance scenes (Ma et al., 19 Jul 2025).

6. Limitations, misconceptions, and open directions

A recurrent misconception is that temporal smoothing alone is sufficient. The monocular video HMR literature argues that temporal smoothers help but cannot resolve the underlying 3D ambiguity; without along-ray and metric cues, systems remain prone to jitter, limb flips, and scale drift (Cen et al., 4 Feb 2026). A related misconception is that RGB supervision alone can reliably recover geometry. HumanRecon explicitly presents geometric cues and physical priors as a response to RGB-only overfitting and density ambiguity along rays (Yin et al., 2023).

Failure modes recur across representation families. MetaAvatar notes that depth consistency or cloth realism can degrade for complex, unseen garments such as blazer tails when fine-tuning data is extremely limited, and that accurate SMPL registrations are important because large pose-estimation errors or severe occlusions degrade canonicalization and thus the SDF training signal (Wang et al., 2021). NSF identifies thin structures, fast motions, heavy occlusions, and consumer depth noise as factors that can degrade SDF fusion and feature projection, while also requiring reliable pose estimates and inverse skinning to canonical space (Xue et al., 2023).

Video and metric formulations inherit additional dependencies. GeoMan points to fast motion, motion blur, and noisy root-depth estimation as sources of degradation in metric depth recovery, since errors in $D_{\text{vis}}(u,v)$2 propagate directly into the recovered metric depth (Kim et al., 29 May 2025). DTO and MA-HMR remain sensitive to monocular depth errors, extreme perspectives, and anthropometric-prior mismatch, particularly for non-standard individuals or kid-heavy scenes; the paper also notes that under-representation of minors in training may limit generalization (Wang et al., 17 Nov 2025).

The multiview detection variant has its own constraints. DCHM for multiview detection depends on calibrated cameras, ground-plane geometry for metric scaling, reliable 2D masks, and sufficiently informative cross-view overlap. The paper explicitly identifies noisy compensatory masks from inaccurate depth, single-view instability when only one camera sees a person, and extremely disjoint views as remaining failure cases, and proposes temporal information and stronger validity checks for compensatory masks as future improvements (Ma et al., 19 Jul 2025).

Taken together, these works suggest that DCHM is an organizing principle for human reconstruction systems that must reconcile observation fidelity, deformation coherence, and metric plausibility. Its concrete implementations differ sharply—double-depth maps, neural SDFs, neural surface fields, radiance fields, diffusion-based geometry predictors, and Gaussian splats—but each treats depth not as an auxiliary cue alone, but as a structural constraint on the geometry that the model is permitted to produce.