Skeletal Representation (S-Rep)

• Skeletal Representation (S-Rep) is a family of shape encodings that model objects using interior medial scaffolds and directional spokes for compact, correspondence-rich shape analysis.
• It integrates classical medial axis concepts with modern techniques like graph-based and deep learning methods to optimize shape reconstruction, symmetry, and geometric fidelity.
• S-Reps are pivotal in biomedical imaging and 3D graphics, providing stable correspondences and precise morphometric descriptors for detailed anatomical and geometric analysis.

A skeletal representation (S-Rep) is a family of shape encodings that models a solid or articulated object using low-dimensional geometric scaffolds—typically interior loci (medial sheets, points, or lines) equipped with associated geometric fields (spoke directions, radii, frames)—for the purpose of providing compact, correspondence-rich, and generatively interpretable models of shape in ℝ² or ℝ³. S-Reps unify principles of the Medial Axis Transform, geometric modeling, and statistical shape analysis, and have become foundational in both biomedical image analysis and 3D data-driven geometry processing.

1. Mathematical Foundations of Skeletal Representations

Skeletal representations generalize the medial axis transform (MAT), which encodes a closed domain Ω⊂ℝ³ by the locus of centers of maximal inscribed balls (medial points) and their radii. For classical MAT,

$$\mathrm{MAT}(\Omega) = \{(x, r(x)) \mid r(x) = \min_{b \in \partial\Omega}\|x-b\|,\, |\arg\min_b \|x-b\|| \ge 2\}$$

This medial locus is generically a union of sheets (for 3D objects), from which one can reconstruct both the object’s topology and local thickness.

Skeletal representations refine this schema:

• By discretizing the medial sheet (as a grid or mesh of points or frames),
• Attaching directional spokes $d(u,v)$ and lengths $r(u,v)$ at each skeletal location (parameterized by $(u,v)$ on the sheet),
• Optionally providing an orthonormal local frame $F(u,v,t)\in SO(3)$ at every position in the volume or on the boundary.

The boundary is reconstructed by shooting a spoke of length $r(u,v)$ from each skeleton point $s(u,v)$ in direction $d(u,v)$ so $b(u,v) = s(u,v) + r(u,v)d(u,v)$ (Pizer et al., 2024). For full generality, modern S-Reps allow both sheet-like and curve-like (skeleton) structures to coexist, supporting representation of both slabular and tubular geometries (Lin et al., 2020).

Validity constraints typically enforced in S-Reps include boundary adherence, local orthogonality (spoke direction normal to boundary at endpoint), and non-intersection of spokes (Gao et al., 22 Dec 2025).

2. Computational Construction of S-Reps

Multiple algorithmic paradigms exist for constructing S-Reps, depending on the nature and modality of available data (triangular mesh, point cloud, binary mask):

• Analytic Template Deformation: The evolutionary s-rep constructs a smooth diffeomorphism $\phi$ mapping a canonical ellipsoid $E$ to the object $M$ using staged LDDMM flows, carrying the analytic medial skeleton (s-sheet) and its spokes along the deformation (Pizer et al., 2024). Frames are pulled back and re-orthonormalized to provide interior correspondences. Spoke and frame parameters are optimized jointly to enforce fit, orthogonality, and geometric fairness.
• Data-Driven Learning from Point Clouds: Methods such as Point2Skeleton (Lin et al., 2020) predict skeletal points as a convex combination of input points using architectures such as PointNet++, then estimate radii and build a mesh connectivity via graph auto-encoders. The geometric loss combines reconstruction fidelity, medialness, and spread regularization.
• Graph-Based Learning from Masks: HybridVNet encodes the medial grid in a fixed graph template and uses graph convolutional, variational autoencoder decoders to deform a template S-Rep to a given object segmentation, inferring corresponding skeletal and boundary points (Gaggion et al., 2024).
• Classical and Modern Medial Geometry: Algebraic, Voronoi-based, and contraction flows extract medial axis points or sheets (or their curve skeleton reductions), sometimes stabilized by pruning or scale axis transforms (Tagliasacchi, 2013). Generalized enveloping primitives and sparseness optimization provide even more compact, yet highly precise, skeletal diagrams (Guo et al., 2023).
• Boundary-Based Fitting and Swept Sheets: For slabular objects, recent methods employ spectral clustering to split the object boundary, fit the central medial skeleton via 3D Voronoi operations, and locally parameterize the sheet with splines or polynomial surfaces. A combinatorial/optimization search balances model fit, symmetry, and smoothness (Taheri et al., 2024).

3. Geometric Features, Correspondence, and Statistical Shape Analysis

S-Reps are uniquely suited for shape analysis because they establish interior geometric correspondences that are stable across populations:

• Feature Extraction: At each skeletal node or grid point, features include spoke lengths, spoke direction (as unit vectors on $S^2$), rotation between neighboring frames (using the $\log_{\mathfrak{so}(3)}$ map), radial distances, and local curvatures. These are concatenated (often after sphere-to-Euclid embeddings) to yield high-dimensional Euclidean feature vectors (Pizer et al., 2024).
• Cross-Object and Longitudinal Correspondence: Sampling the s-reps on a common grid or template skeleton ensures that each parameter location (e.g., $(u,v)$ on the sheet) is consistently matched across subjects. Re-optimization of spoke lengths but not skeleton ensures stable longitudinal correspondence (Gao et al., 22 Dec 2025).
• Morphometric Descriptors: Derived quantities include local thickness (sum of superior/inferior spoke lengths), lamellar width, long-axis length, and curvature, enabling fine-scale characterization of local and global shape variability (Gao et al., 22 Dec 2025).
• Euclideanization for Statistics: Geometry is mapped from products of spheres and $SO(3)$ to $\mathbb{R}^n$ for application of standard multivariate tests (e.g., DiProPerm) and machine learning classifiers (Taheri et al., 2024, Pizer et al., 2024).

4. Applications and Comparative Evaluation

Skeletal representations are foundational in:

• Biomedical Shape Analysis: S-Reps support population correspondence for anatomical studies in the hippocampus, heart valves, and other organs, yielding improved classification accuracy for disease-related shape differences and enabling substructure-specific morphometry (Gao et al., 22 Dec 2025, Pizer et al., 2024).
• 3D Computer Vision and Graphics: In addition to geometric modeling and surface reconstruction (enabling near-watertight, topology-preserving results from sparse interior points), S-Reps power animation rigs, mesh decomposition, topology optimization, mesh alignment, and user-interactive design (Guo et al., 2023, Zhang et al., 2024, Han et al., 2016).
• Human Motion Representation: In articulated models, the skeletal graph provides both spatial and spatiotemporal encoding for pose, action, and identity recognition. Graph and manifold-based S-Reps capture both low-level geometry and high-level semantic dynamics (Han et al., 2016).

Empirical results demonstrate that evolutionary s-reps and discrete swept s-reps outperform previous methods in terms of correspondence accuracy, statistical power, and geometric fidelity. For instance, an AUC of 0.73 on hippocampal classification (vs. 0.53–0.58 for initial velocity/diffeomorphic momenta, and 0.60 for one-stage S-Rep) has been achieved (Pizer et al., 2024). HippMetric delivers cross-sectional and longitudinal correspondence errors ~1.6 mm, well below that of SPHARM-PDM or traditional cm-reps (Gao et al., 22 Dec 2025).

5. Extensions, Limitations, and Open Problems

Skeletal representation research remains highly active, with ongoing advances addressing:

• Representation Sparsity vs. Geometric Fidelity: The Medial Skeletal Diagram demonstrates that by shifting geometric complexity from discrete combinatorial elements to continuous primitive interpolation, one can achieve orders-of-magnitude compression without sacrificing reconstruction error (Guo et al., 2023).
• Robustness to Boundary Noise: Since classical MAT is highly unstable to small surface perturbations, practical S-Rep pipelines employ geometric, statistical, or learned denoising mechanisms (e.g., relaxed spoke orthogonality, energy regularization, or deep learning priors) (Lin et al., 2020, Khargonkar et al., 2023).
• Generalization Across Modalities: Progress includes robust S-Rep extraction from non-watertight point clouds, mask-based segmentations, and even heterogeneous, device-adapted, or multi-modal data streams (Lin et al., 2020, Gaggion et al., 2024, Han et al., 2016).
• Intrinsic Coordinate Systems: Axis-referenced morphometric models and the development of globally smooth skeletal coordinates (long-axis and lamellae) underlie advances in fine-grained anatomical analysis and cross-sectional/longitudinal studies (Gao et al., 22 Dec 2025).
• Limitations and Current Research: Challenges persist in parameter-free model selection, correspondence in highly twisted or branching structures, stability guarantees under sampling variation, and incorporation of semantic or functional priors beyond geometry (Taheri et al., 2024, Tagliasacchi, 2013).

6. Taxonomy and Unification Across Domains

Despite diverse technical frameworks, S-Reps in biomedical modeling, computer vision, and geometric deep learning can be classified by:

Paradigm	Medial Element	Construction Algorithm	Key Application Domains
Analytic/deformable	Sheets/frames	LDDMM + s-rep fitting	Shape statistics, anatomical morphometry
Discrete/geometric	Points, mesh	Voronoi, graph pruning	Graphics, 3D object modeling, tube reconstr.
Deep learning	Points/graph	PointNet++, GCN, autoencoder	Fast S-Rep from point clouds, segmentation
Articulated/kinematic	Skeletal graphs	Adjacency, Laplacian, GNN	Human pose/action, animation, tracking

This taxonomy reflects the unifying principle: S-Reps encode shape by interior scaffolding and geometric fields, enabling parametrically rich, analysis-ready, and correspondence-consistent representations suited to both generative and discriminative shape modeling across a broad array of domains (Pizer et al., 2024, Gao et al., 22 Dec 2025, Guo et al., 2023, Lin et al., 2020, Han et al., 2016, Tagliasacchi, 2013).