Geometry-Aware Style Transfer in 3D Gaussian Splatting

Abstract: In this paper, we present a novel geometry-aware style transfer framework for 3D Gaussian splatting (3DGS) that simultaneously transfers appearance attributes and geometric structures. Unlike prior works that primarily focus on color-based stylization and often overlook structural adaptation, our method explicitly incorporates geometry adaptation through a decoupled optimization scheme that alternately updates color and geometry parameters. This strategy alleviates potential interference between color and geometry updates, leading to stable and consistent scene-level geometry transformation. The decoupled optimization is enabled by the proposed geometry-aware contrastive feature matching (GCFM). GCFM integrates RGB, depth, and edge cues into a contrastive objective and is employed in both optimization phases to effectively transfer structural characteristics from style images to Gaussian primitives. Extensive experiments show that our approach achieves superior performance in both qualitative fidelity and quantitative metrics, significantly outperforming existing 3DGS-based stylization methods. Our code is available at \href{https://github.com/oweixx/gast}{https://github.com/oweixx/gast}.

• The paper presents an innovative geometry-aware style transfer technique that jointly adapts color and geometry parameters through a decoupled optimization scheme.
• It leverages multi-modal cues from RGB, depth, and edge features via a Geometry-aware Contrastive Feature Matching loss to enhance structural fidelity.
• The approach achieves superior perceptual quality and multi-view consistency with real-time efficiency, demonstrating significant improvements over traditional methods.

Geometry-Aware Style Transfer in 3D Gaussian Splatting

Motivation and Problem Statement

Recent advances in neural style transfer have enabled the embedding of artistic features into images and volumetric 3D representations. However, the majority of 3D stylization frameworks—especially those based on 3D Gaussian Splatting (3DGS)—focus predominantly on color appearance transfer, enforcing rigid geometric preservation due to the inherent instability of geometry modifications. This conservatism limits expressive capacity and fails to exploit the full potential of 3D stylization, which should include structural adaptation alongside appearance modification.

This paper introduces an explicit geometry-aware style transfer methodology for 3DGS that jointly adapts both color and geometry parameters. The framework leverages a decoupled optimization paradigm and a contrastive feature matching scheme integrating multi-modal cues (RGB, depth, edge) to ensure high fidelity and stable geometric stylization.

Framework Overview and Key Innovations

The proposed method begins by reconstructing a 3DGS scene from input multiview images, followed by color statistics alignment of content and style images. The critical novelty lies in the style transfer phase, where a decoupled optimization strategy alternates between color and geometry updates, rather than performing joint optimization. This alternation mitigates destructive interference between color and geometry adjustments, yielding more stable and consistent stylization. Optimization is guided by a Geometry-aware Contrastive Feature Matching (GCFM) loss, which incorporates joint features extracted from color, estimated depth, and structural edges.

Figure 1: Architecture of the geometry-aware style transfer pipeline, combining color matching, decoupled optimization phases, and multi-modal GCFM loss.

The GCFM loss employs a nearest-neighbor contrast between rendered and style image features, explicitly pulling rendered features toward structurally similar style features, and pushing them away from dissimilar ones. This multi-modal alignment ensures robust geometric transfer and avoids issues arising from noisy or ill-posed single-modality matching.

Decoupled Optimization for Stable Geometry Adaptation

Direct, simultaneous tuning of both color and geometry parameters is shown to cause instability and over-stylization, often corrupting scene structure. The proposed decoupled scheme alternates between optimizing color while fixing geometry, and geometry while fixing color, in multiple outer cycles, each with fixed numbers of inner updates for each phase.

Figure 2: Joint-step versus decoupled optimization. The decoupled scheme stabilizes geometry transformation and prevents color-induced structural collapse.

Empirical ablation shows that only geometry or only color optimization fails to achieve balanced stylization. Joint-step approaches exhibit instability and degradation, while the decoupled scheme achieves both perceptual style fidelity and multi-view consistency.

Geometry-Aware Contrastive Feature Matching (GCFM)

Traditional feature matching losses in 3DGS stylization are limited to color (RGB) cues and suffer from weak geometric awareness. Depth and edge modalities are incorporated to reinforce scene structure and spatial boundaries. The GCFM aggregates VGG-based feature embeddings over all modalities and applies a spatial contrastive objective, with positive anchors as closest style features and negatives as farthest. This enforces discriminative geometric style adaptation and enhances structural detail preservation.

Figure 3: Output examples showing geometry-aware stylization. Both color and geometric structural details are adaptively transferred and aligned with the style reference.

Ablation shows that GCFM’s contrastive loss and multi-modal feature aggregation are both crucial for structural distinctiveness and geometry reproduction.

Evaluation and Results

Experiments spanning eight 3D scenes and nine diverse style references quantify qualitative and quantitative improvements over state-of-the-art 3DGS-based stylization methods. The method achieves:

• Lowest Single Image Fréchet Inception Distance (SIFID) for perceptual style fidelity.
• Superior multi-view consistency scores (LPIPS, masked RMSE) excluding overly conservative baselines.
• Real-time stylization efficiency, completing full stylization within an average of fourteen minutes.
• Positive outcomes in user studies, with higher ranks for style similarity, visual appeal, and content recognizability.

Ablation studies confirm the superiority of the decoupled update scheme and validate hyperparameter robustness for the GCFM weighting.

Practical and Theoretical Implications

The explicit geometry-aware stylization addresses the limitations of conservative color-only 3DGS approaches. By integrating geometric adaptation, the model enables expressive and structurally consistent 3D stylization, critical for applications in immersive content creation, interactive graphics, and artistic rendering paradigms. The decoupled optimization paradigm may generalize to other high-dimensional scene representations where parameter update interference impedes stable learning.

From a theoretical perspective, the multi-modal contrastive loss opens avenues for hierarchical feature aggregation and structural alignment in high-dimensional generative modeling. The methodology can be extended to incorporate semantic cues or higher-level geometric priors for domain-specific stylization tasks.

Conclusion

This paper establishes a geometry-aware style transfer framework for 3DGS, resolving the instability of geometry adaptation through decoupled optimization and robust contrastive feature matching across RGB, depth, and edge modalities. The approach outperforms leading 3DGS stylization baselines in both qualitative and quantitative measures, achieving efficient and expressive stylization with structural fidelity and multi-view coherence. Future work includes extending the framework to dynamic scenes, semantic-aware stylization, and leveraging more advanced multi-modal embeddings for richer structural transfer.