Structure-Aware Fine-grained Gaussian Splatting

• The paper introduces SFGS, integrating SMPL-X priors and triplane-hexplane features to achieve topology-aware, photorealistic human avatar reconstruction.
• It combines spatial and temporal representations to preserve fine details, such as hand articulation, while maintaining global structural coherence.
• Experimental results demonstrate improved PSNR, SSIM, and LPIPS metrics, underscoring its efficiency and superiority over previous methods.

Structure-aware Fine-grained Gaussian Splatting (SFGS) denotes a class of 3D Gaussian Splatting approaches in which Gaussian primitives are conditioned by explicit structural cues so that fine detail can be reconstructed without sacrificing global coherence. In the narrow sense, the term names a monocular expressive-avatar method that combines a spatial-only triplane, a time-aware hexplane, a structure-aware gaussian module, and a residual refinement module based on fine-grained hand reconstruction to recover photorealistic and topology-aware full-body humans from a monocular video sequence (Su et al., 10 Apr 2026). In a broader research sense, the same phrase has been used as an organizing lens for structure-conditioned Gaussian fields in domains such as bonsai generation, large-scale outdoor rendering, semantic urban reconstruction, and geometry-aware densification (Wu et al., 2 Apr 2025).

1. Definition and scope

SFGS emerges from a recurring limitation of conventional 3D Gaussian Splatting: photometric optimization alone often fails to preserve thin structures, articulated parts, or long-range organization when the scene or object contains strong topology, sparse supervision, or high-frequency appearance. In human-avatar reconstruction, this failure appears most clearly in hands and facial expressions; in text-to-3D bonsai generation it appears as broken branching topology and oversmoothed foliage; in large-scale scenes it appears as loss of global layout, local surface detail, or multi-view consistency (Su et al., 10 Apr 2026).

Current usage therefore spans two related meanings. First, SFGS is a specific method for expressive avatar reconstruction, centered on SMPL-X/MANO priors, triplane-hexplane features, and joint-aware Gaussian offsets (Su et al., 10 Apr 2026). Second, it functions as a descriptive label for methods that make Gaussian splatting “structure-aware” by introducing skeletons, semantic groups, depth priors, edge masks, hierarchical scales, or view-consistent densification as explicit constraints on where Gaussians are initialized, how they deform, and which details they are permitted to represent (Xiao et al., 23 May 2025).

A common misconception is that SFGS refers only to semantic labeling or only to a single architectural template. The literature instead shows multiple structural regimes: skeletal structure for avatars, procedural branch structure for bonsai, tri-plane or grid structure for outdoor scenes, and depth-, edge-, or city-model-guided structure for reconstruction and semantics. What unifies them is not a fixed module list but the principle that Gaussian support, attributes, or optimization should be governed by domain structure rather than by image loss alone (Tang et al., 2024).

2. Canonical Gaussian representation and spatiotemporal features

In the named avatar method, SFGS uses a canonical human mesh derived from SMPL-X and samples about 170k points on the canonical surface, with each sample associated to a canonical-space 3D Gaussian primitive (Su et al., 10 Apr 2026). Each Gaussian carries mean position, covariance, color, and opacity, and rendering follows the usual Gaussian-splatting image formation: $$\mathbf{C}(\mathbf{p}) = \sum_{i=1}^N w_i(\mathbf{p}) \cdot \mathbf{c}_i .$$ For dynamic motion, the canonical body is deformed by Linear Blend Skinning,

$$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$

and the posed Gaussians are rendered with the standard 3DGS pipeline (Su et al., 10 Apr 2026).

The central representational choice is the combination of a spatial-only triplane and a time-aware hexplane. The triplane tensor is

$$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$

and the hexplane tensor is

$$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$

with $C=32$ and $H=W=128$ (Su et al., 10 Apr 2026). For a canonical point $\mathbf{v}_i$, bilinear sampling over the $xy$, $yz$, and $xz$ planes yields a triplane feature $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$0. For a space-time coordinate $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$1, sampling over the $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$2, $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$3, $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$4, $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$5, $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$6, and $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$7 planes yields a hexplane feature $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$8, which is then aligned to the triplane feature distribution by an MLP (Su et al., 10 Apr 2026).

The two features are fused by a learned per-point weight: $$\mathbf{V}_{\mathrm{pose}} = \mathrm{LBS}(\bar{\mathbf{V}}, \theta, \mathbf{W}),$$9

$$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$0

This makes the representation simultaneously stable in canonical space and responsive to temporal change. The paper further uses separate higher-resolution triplane and hexplane tensors for the facial region, reflecting the need for greater capacity in mouth and eye dynamics (Su et al., 10 Apr 2026).

Within the broader SFGS literature, this coarse principle reappears in different forms. SplatCo uses a global tri-plane feature field plus local context grids with hierarchical compensation, rather than a triplane-hexplane pair, but the objective is analogous: combine global structural consistency with local detail modeling in the Gaussian attribute prediction process (Xiao et al., 23 May 2025).

3. Structure-aware deformation and fine-grained hand modeling

The structure-aware gaussian module is the defining mechanism of the avatar formulation. SFGS uses SMPL-X with 55 joints and assigns each Gaussian to a dominant joint by the argmax of its skinning weights: $$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$1 For each Gaussian $$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$2, a joint-aware feature is constructed from the dominant joint rotation $$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$3, joint position $$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$4, and a learnable joint embedding $$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$5: $$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$6 This feature is concatenated with the fused triplane-hexplane feature and passed through an MLP,

$$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$7

to predict a mean offset $$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$8 and a scale offset $$\mathbf{T} \in \mathbb{R}^{3 \times C \times H \times W},$$9. The geometry is then updated as

$$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$0

This construction makes Gaussian deformation explicitly pose-dependent and spatially coherent near the controlling joint (Su et al., 10 Apr 2026).

Appearance is also made structure-aware. A locally fused pose embedding is defined as

$$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$1

so each Gaussian receives a weighted mixture of all joint rotations that influence it. Combined with the local surface normal $$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$2 and the fused feature $$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$3, this drives a pose-aware color offset: $$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$4 The result is a color field that changes with local articulation rather than with a global pose code repeated uniformly over the body (Su et al., 10 Apr 2026).

Hands receive a dedicated residual refinement module because they exhibit the largest reconstruction error in standard Gaussian human avatars. SFGS uses the MANO hand model to obtain a higher-fidelity hand mesh,

$$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$5

and computes a residual with respect to the SMPL-X hand mesh: $\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$6 A pose-conditioned MLP then refines this residual,

$$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$7

and the corrected hand vertices are used to adjust the Gaussians attached to the hand region (Su et al., 10 Apr 2026). This module is not a post-processing step; it is integrated into the same single-stage optimization as the rest of the avatar.

4. Optimization, supervision, and reported performance

SFGS is trained in a single stage with the total objective

$$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$8

The full-body image term is

$$\mathbf{H} \in \mathbb{R}^{6 \times C \times H \times W},$$9

with the best reported weights $C=32$0, $C=32$1, and $C=32$2 (Su et al., 10 Apr 2026). A face-specific loss uses UV-aligned facial textures from FLAME, and the regularization block contains mean-offset, scale, Laplacian-smoothness, and joint-consistency terms (Su et al., 10 Apr 2026). All MLPs are 2-layer networks with hidden dimension 256 and ReLU, and training is reported on a single NVIDIA RTX 4090 (Su et al., 10 Apr 2026).

On NeuMan, averaged over the reported sequences, SFGS achieves PSNR 35.34, SSIM 0.985, and LPIPS 0.009, compared with ExAvatar at PSNR 34.80, SSIM 0.984, and LPIPS 0.009 (Su et al., 10 Apr 2026). On X-Humans subject 00028, SFGS reports PSNR 31.12, SSIM 0.983, and LPIPS 0.018, compared with ExAvatar at PSNR 30.58, SSIM 0.981, and LPIPS 0.018, while X-Avatar reports PSNR 28.57, SSIM 0.976, and LPIPS 0.026 (Su et al., 10 Apr 2026). Region-wise geometry evaluation further shows lower CD and CD-MAX together with higher NC and IoU than ExAvatar across all, face, and hand regions, with the paper highlighting a 3.8% reduction in hand CD and an IoU improvement of about 3.79% for hands (Su et al., 10 Apr 2026).

Ablation results identify the structure-aware offset as the strongest contributor. Removing structure offset reduces PSNR to 30.58, SSIM to 0.982, and LPIPS to 0.020, while removing color offset gives PSNR 30.83, removing HexPlane gives PSNR 31.03, and removing hand reconstruction gives PSNR 30.92 with SSIM dropping from 0.983 to 0.927 (Su et al., 10 Apr 2026). Temporal consistency is also improved: tc-LPIPS decreases when hexplane features are included, and the method renders at about 30 FPS at $C=32$3, compared with 26 FPS for ExAvatar (Su et al., 10 Apr 2026).

5. Broader formulations and neighboring methods

The broader literature uses SFGS as a useful abstraction for any Gaussian-splatting pipeline in which structure constrains initialization, feature fusion, densification, or supervision.

Paper	Domain	Structural mechanism
3DBonsai (Wu et al., 2 Apr 2025)	Text-to-3D bonsai	Trainable 3D SCA, skeleton/mesh/point-cloud prior, fine and coarse structure conditioned generation
SplatCo (Xiao et al., 23 May 2025)	Large-scale outdoor scenes	Global tri-plane, local context grids, hierarchical compensation, cross-view assisted training
DET-GS (Huang et al., 6 Aug 2025)	Sparse-view reconstruction	Hierarchical geometric depth supervision, edge-aware depth regularization, RGB-guided edge-preserving TV
HiSplat (Tang et al., 2024)	Generalizable sparse-view GS	Coarse-to-fine hierarchical Gaussians, Error Aware Module, Modulating Fusion Module
SA-GS (Xiong et al., 2024)	Large-scene semantic reconstruction	GroundingSAM masks, geometric complexity loss, semantic-aware Gaussian allocation
GeoTexDensifier (Jiang et al., 2024)	Photorealistic scene reconstruction	Texture-aware densification, geometry-aware splitting, Validation of Depth Ratio Change

In 3DBonsai, the structural prior is a trainable 3D space colonization algorithm that produces a bonsai skeleton, then a mesh, then a point cloud used as the support of the Gaussian field; the method reports an FID drop from 138 to 74 for the structural prior after training, and its text-conditioned pipelines distinguish between fine structure conditioned generation and coarse structure conditioned generation (Wu et al., 2 Apr 2025). In SplatCo, the phrase “structure-view collaborative Gaussian splatting” denotes a global tri-plane structural field fused with local context grids, plus multi-view gradient synchronization, visibility-aware densification, and structural-consistency-guided pruning; the paper reports PSNR improvements of 1–2 dB and SSIM gains of 0.1 to 0.2 on 13 large-scale scenes (Xiao et al., 23 May 2025).

DET-GS shows a different structural regime: it remains within standard 3DGS but adds hierarchical geometric depth supervision, edge-aware depth regularization guided by Canny masks, and RGB-guided edge-preserving Total Variation loss, achieving PSNR 28.29, SSIM 0.840, and LPIPS 0.175 on Mip-NeRF 360 (Huang et al., 6 Aug 2025). HiSplat, by contrast, is hierarchical rather than explicitly semantic: stage 1 predicts large coarse-grained Gaussians that form a skeleton of the scene, and later stages add smaller Gaussians guided by residual error, reaching PSNR 27.21, SSIM 0.881, and LPIPS 0.117 on RealEstate10K with only two reference views (Tang et al., 2024).

Other neighboring developments extend the same logic. AH-GS augments Scaffold-GS with Adaptive Frequency Encoding Module and high-frequency reinforce loss so that structurally complex regions obtain higher-frequency encodings, reporting PSNR 29.70, SSIM 0.871, and LPIPS 0.181 on Mip-NeRF360 at 30K iterations (Xu et al., 28 Mar 2025). GS4City transfers structured urban semantics from aligned LoD3 CityGML models into Gaussian identity codes and reports up to 15.8 IoU points in coarse building segmentation and 14.2 mIoU points in fine-grained semantic segmentation over 2D-driven semantic 3DGS baselines (Zhang et al., 13 Apr 2026). “Faster 3D Gaussian Splatting Convergence via Structure-Aware Densification” replaces gradient-based densification with a per-Gaussian, per-axis frequency violation metric $C=32$4, anisotropic splitting, and multiview consistency, effectively converging within 3k iterations on Mip-NeRF360 while preserving high-frequency detail (Lyu et al., 30 Apr 2026).

6. Limitations, misconceptions, and research directions

The most important misconception is that SFGS is already a standardized formulation. The literature instead shows a moving target: in avatars, structure is skeletal and articulation-aware; in bonsai synthesis it is procedural and topological; in urban semantics it is hierarchical and city-model-based; in reconstruction it may be depth-, edge-, texture-, or frequency-aware. This suggests that “structure-aware” currently names a research program more than a closed taxonomy (Su et al., 10 Apr 2026).

The named avatar method has clear limitations. It depends on reasonably accurate SMPL-X fitting and hand pose estimation, is designed for single-subject sequences, and can produce blurred boundaries in wide clothing or bulky regions because the upsampled point density may be insufficient there (Su et al., 10 Apr 2026). The broader variants exhibit analogous dependence on their own priors: 3DBonsai is tuned to Chinese-style bonsai and does not directly generalize to arbitrary botanical morphologies (Wu et al., 2 Apr 2025); DET-GS remains reliant on monocular depth quality in textureless or occluded regions (Huang et al., 6 Aug 2025); GeoTexDensifier depends on the quality of relative monocular depth priors for VDRC validation (Jiang et al., 2024); GS4City requires aligned, semantically rich LoD3 CityGML models and is correspondingly urban-domain specific (Zhang et al., 13 Apr 2026).

Another recurring limitation is efficiency under richer structural supervision. SplatCo notes sensitivity to camera pose quality and viewpoint distribution, together with increased resource demand from joint structure-view optimization (Xiao et al., 23 May 2025). The structure-aware densification framework that uses multiscale frequency analysis raises training VRAM from 9.9 GB in 3DGS to 13.0 GB, even though inference remains in the same rendering regime (Lyu et al., 30 Apr 2026). HiSplat improves sparse-view generalization but still relies on geometric and photometric structure rather than explicit semantics or surface constraints (Tang et al., 2024).

The main research directions are correspondingly consistent across papers. For avatars, the obvious extensions are better adaptiveness for wide clothing, richer interaction such as sketch- or text-guided editing, and broader generalization beyond single-person monocular capture (Su et al., 10 Apr 2026). For structured generation and reconstruction, several papers point toward end-to-end differentiable structural generators, richer learned graph or semantic priors, confidence-aware depth supervision, more views or active view selection, and tighter coupling between structural supervision and Gaussian densification (Wu et al., 2 Apr 2025). A plausible implication is that future SFGS systems will increasingly combine three ingredients: a domain prior that specifies admissible structure, a fine-grained Gaussian field that concentrates capacity where detail is needed, and a supervision mechanism that enforces multi-view or temporal consistency without collapsing the underlying topology.