Appearance-Agnostic Skeletal Geometry

Updated 6 December 2025

The paper introduces robust descriptors that abstract human skeletal geometry independent of appearance factors using both analytical and deep learning methods.
It employs techniques like point cloud skeletonization, parametric skeleton coding, and graph-based descriptors for precise statistical modeling and alignment.
Empirical validations show these features enhance performance in medical imaging, action recognition, and virtual try-on applications.

Appearance-agnostic human skeletal geometry features are robust descriptors of human body morphology and kinematics designed to abstract geometric structure from sensor data or reconstructions, independent of superficial appearance details (lighting, texture, clothing, background, or photometric variance). Such features form the foundation for precise statistical modeling, recognition, generative synthesis, shape correspondence, and downstream tasks in computer vision, medical analysis, biometrics, action recognition, and simulation. The following sections provide an in-depth exposition of principles, extraction methodologies, model architectures, supervision strategies, geometric invariance, and validation of appearance-agnostic skeletal geometry features across contemporary literature.

1. Formal Representations and Mathematical Foundations

Appearance-agnostic skeletal geometry features encode the spatial arrangement and connectivity of anatomical landmarks (joints, bones, medial axes) in a manner decoupled from image- or voxel-level texture, intensity, or color. Key approaches include:

Medial/skeletal point-sheet models: Compact, discrete sets of skeletal points $S=\{s_j\}_{j=1}^M$ and associated radii $r(s_j)$ are constructed from convex combinations of surface points $P=\{p_i\}_{i=1}^N$ , typically via mapping matrices $W\in\mathbb{R}^{N\times M}$ , e.g., $s_j = \sum_i W_{ij}p_i$ and $r(s_j) = \sum_i W_{ij} d_i$ , where $d_i$ is the surface-to-skeleton Euclidean distance (Khargonkar et al., 2023).
Parametric skeleton codes: Separation of latent codes for shape ( $\beta^s$ ) and skeleton ( $\beta^k$ ) is realized in expressive body models such as ATLAS, in which $\beta^k$ exclusively controls joint scales and bone lengths while $\beta^s$ parameterizes soft tissue variation. The skeleton code is fit directly to 2D/3D keypoint clouds and is invariant to outer surface appearance (Park et al., 21 Aug 2025).
Graph-based and metric descriptors: Skeleton graphs $G=(V,E)$ encode pixel or vertex connections in medial axis skeletonization pipelines; associated features encompass per-segment lengths, angles at forks, global ratios (height, width), and normalized postural descriptors (Sengupta et al., 2015).
Spatio-temporal joint features: Temporal joint trajectories $\mathbf{J}_{t,k}\in\mathbb{R}^{T\times K\times3}$ , bone lengths $\mathbf{L}_n$ , and kinematic displacements $\mathbf{M}$ are disentangled to represent pose, body scale, and motion respectively, independent of video features or RGB context (Tang et al., 21 Oct 2024).

These representations are constructed solely from geometric and topological structure and are computed directly from the mesh, keypoint, or point-cloud data.

2. Extraction Methods and Geometric Losses

State-of-the-art appearance-agnostic skeletal extraction relies on either analytical geometric pipeline design or deep learning architectures with explicit geometric priors and loss terms:

PointNet++ Skeletonization + Geometric Deep Learning: An input 3D point cloud is encoded through hierarchical set-abstraction layers. The decoder produces a skeleton sheet by learning weights that convex-combine surface points. Weak supervision is provided via fitted s-reps (using SlicerSALT templates); geometric loss terms include the symmetric Chamfer loss for skeleton alignment, spread regularization to prevent collapse, and medial enforcement to guarantee approximate medial axis properties (Khargonkar et al., 2023).
Evolutionary s-rep via Fitted Frames: Boundary meshes are mapped diffeomorphically from ellipsoidal templates using CMCF and LDDMM flows, producing s-reps in object interiors. Local frames and onion-skin surfaces yield invariant features such as spoke length and direction, frame curvature, and positional spacing; all quantities are made Euclidean invariant through log, PNS, and alignment-free transformations (Pizer et al., 19 Jul 2024).
Contrastive Attack-Augmentation Learning: Adversarial perturbations (via entropy-maximizing attacks) and appearance augmentations generate boundary-level hard positives and negatives. The mixing-contrastive objective leverages these samples to force networks to focus on intrinsic geometric configurations, not superficial appearance (Xu et al., 2023).
Semi-analytical regressors: ARTS uses lifted 3D skeletons to decouple joint positions, bone lengths, and dynamics; temporal inverse kinematics and bone-guided shape fitting stages exploit these components analytically before permitting final appearance-driven refinement (Tang et al., 21 Oct 2024).

These extraction and supervision strategies leverage direct geometry-based criteria to produce representations insensitive to appearance.

3. Invariance Properties and Downstream Utility

Built-in normalization and geometric constraints provide the following invariances:

Feature Type	Translation-invariant	Scale-invariant	Appearance-invariant
Joint coordinates	Yes (origin-centering)	Yes (bone norm)	Yes (RGB, texture ignored)
Medial radii	Yes	Yes	Yes
Direction vectors	Yes	Yes (normed)	Yes
Skeleton codes	Yes	Yes	Yes

Normalization: Features (e.g. joint coordinates, bone lengths) are centered at anatomical roots, scaled to unit average bone length, eliminating effects of translation and uniform scaling (Tseng et al., 2022).
Alignment: Canonical poses (e.g. root facing -z axis) via rigid alignment eradicate viewpoint disparity, making geometric matching robust for few-shot recognition and clustering (Tseng et al., 2022).
Geometric regularity: Enforced by geometric losses and deep alignment, descriptors are robust to nonuniform segmentation, background, or texture (Khargonkar et al., 2023, Xu et al., 2023).
Appearance-agnostic transformations: All features depend exclusively on geometry (joint locations, bone segment lengths, angles) and ignore clothing, shading, or background. E.g., warping in virtual try-on is computed from keypoints and limb vectors, not pixel colors (Roy et al., 2022).

This robust invariance makes such features suitable for shape analysis, morphometry, classification, motion modeling, and generative evaluation.

4. Applications in Medical Imaging, Action Analysis, and Generative Evaluation

Appearance-agnostic skeletal features find application in:

Medical shape analysis and biomarker discovery: s-rep-based skeleton sheets yield compact yet informative features for statistical morphometry, disease classification, and population studies, outperforming raw surface representations in anatomical correspondence and segmentation robustness (Khargonkar et al., 2023, Pizer et al., 19 Jul 2024).
Human action recognition and video generation benchmarking: Explicit geometric (joint, bone, trajectory) descriptors provide higher action classification accuracy, improved few-shot generalization, and foundation for action plausibility metrics in generative modeling evaluation. Metrics formed in learned geometric manifolds (via Transformers and contrastive losses) correlate strongly with human assessment of motion plausibility (Thomas et al., 1 Dec 2025).
Virtual human modeling and parametric mesh fitting: Decoupled skeleton/shape latent codes in models such as ATLAS allow accurate fitting to keypoint clouds, facilitating control over body dimensions without soft-tissue confounds, and supporting downstream synthesis, animation, and mesh recovery (Park et al., 21 Aug 2025, Tang et al., 21 Oct 2024).
Virtual try-on and pose-robust garment warping: Handcrafted features based on limb segment angles, affinity weights, and radial coordinates support pixel-level warping of garment parts across extreme pose variation, avoiding the pitfalls of global non-rigid warps (Roy et al., 2022).
Real-time human detection and biometric recognition: Binary skeletonization pipelines, posture ratios, and limb orientation features yield fast, reliable detectors with invariance to appearance and high computational efficiency in surveillance (Sengupta et al., 2015).

Performance metrics in medical reconstruction, recognition, and generative plausibility are generally superior to pure appearance-based pipelines.

5. Quantitative Evaluation and Empirical Benchmarks

Key empirical measurements across studies include:

Application	Primary Metric	Appearance-agnostic Score	Reference
Skeleton point-sheet recon	Chamfer, Hausdorff	C.D.=0.004, H.D.=0.097 (hipp.)	(Khargonkar et al., 2023)
Parametric skeleton coding	Joint error	3 mm avg. bone length error	(Park et al., 21 Aug 2025)
Few-shot action recog.	Acc. (NTU xView, 5-shot)	70.0% (explicit aligned 3D)	(Tseng et al., 2022)
Generative action metric	Consistency, smoothness	+68% vs prior methods	(Thomas et al., 1 Dec 2025)
Surveillance recognition	False pos/neg	0%/0% (30 human/nonhuman trials)	(Sengupta et al., 2015)
Virtual try-on (poses)	SSIM, FID	SSIM=0.93, FID=16.38	(Roy et al., 2022)
Skeleton contrastive cls.	Single-stream accuracy	84.6% (multi-modal)	(Xu et al., 2023)

A plausible implication is that appearance-agnostic skeletal features retain discriminative power across extreme domain shifts, viewing conditions, and semantic tasks.

6. Current Limitations and Future Directions

Limitations identified across works:

Dependence on keypoint/skeleton extraction fidelity: Occlusion, motion blur, or poor sensor quality can degrade geometric descriptor quality, propagating errors to all downstream appearance-agnostic tasks (Tang et al., 21 Oct 2024, Thomas et al., 1 Dec 2025).
Inflexibility for complex topologies: Fixed-sheet s-reps or analytic skeletons may be insufficient to capture the subtleties of joints (epiphyseal complexity, multifurcations) in complex bones (Pizer et al., 19 Jul 2024).
Potential underweighting of certain error modalities: Shape models with fixed bone-length priors may miss subtle artifacts (e.g., limb stretching in generative models if SMPL fit fails) (Thomas et al., 1 Dec 2025).
Threshold and calibration drift in detection tasks: Static thresholds may not generalize well to all demographic segments or to nonhuman motion (Sengupta et al., 2015).

Future avenues include adaptive threshold learning, multi-view and multi-person fusion, integration of physical priors for contact and interaction dynamics, and embedding geometry-aware modules as sub-networks within multimodal learning frameworks.

Appearance-agnostic human skeletal geometry features form the backbone of modern geometric analysis in biometrics, medical imaging, action recognition, and generative evaluation. Their analytical and data-driven foundations yield robust, interpretable, and transferable representations, capable of transcending superficial appearance artifacts to capture the true structure and dynamics of human morphology and motion.