GarmentGS: Point-Cloud Guided Gaussian Splatting for High-Fidelity Non-Watertight 3D Garment Reconstruction (2505.02126v2)

Abstract: Traditional 3D garment creation requires extensive manual operations, resulting in time and labor costs. Recently, 3D Gaussian Splatting has achieved breakthrough progress in 3D scene reconstruction and rendering, attracting widespread attention and opening new pathways for 3D garment reconstruction. However, due to the unstructured and irregular nature of Gaussian primitives, it is difficult to reconstruct high-fidelity, non-watertight 3D garments. In this paper, we present GarmentGS, a dense point cloud-guided method that can reconstruct high-fidelity garment surfaces with high geometric accuracy and generate non-watertight, single-layer meshes. Our method introduces a fast dense point cloud reconstruction module that can complete garment point cloud reconstruction in 10 minutes, compared to traditional methods that require several hours. Furthermore, we use dense point clouds to guide the movement, flattening, and rotation of Gaussian primitives, enabling better distribution on the garment surface to achieve superior rendering effects and geometric accuracy. Through numerical and visual comparisons, our method achieves fast training and real-time rendering while maintaining competitive quality.

• The paper presents a novel 3D garment reconstruction method that leverages dense point clouds and Gaussian splatting techniques.
• The methodology achieves efficient reconstruction in about 10 minutes by optimizing garment surfaces through precise movement, flattening, and rotation of Gaussian primitives.
• Experimental results on DeepFashion3D demonstrate superior rendering quality and geometric accuracy using metrics like SSIM, PSNR, and Chamfer Distance.

GarmentGS: An Overview of Point-Cloud Guided Gaussian Splatting for 3D Garment Reconstruction

The paper "GarmentGS: Point-Cloud Guided Gaussian Splatting for High-Fidelity Non-Watertight 3D Garment Reconstruction" introduces an innovative method for constructing 3D garments that leverages dense point clouds and 3D Gaussian Splatting (3DGS). The method effectively addresses the challenges associated with traditional garment generation workflows, which are typically time-consuming and labor-intensive. This problem becomes increasingly apparent with the rise of virtual applications that necessitate efficient garment creation.

Methodology

The approach begins with dense point cloud reconstruction from multi-view images. This method dramatically reduces the time required for point cloud generation in comparison to older methods, achieving completion in about 10 minutes. The dense point clouds are then used to guide the optimization of Gaussian primitives, focusing on their movement, flattening, and rotation to achieve non-watertight, high-fidelity garment surfaces.

Figure 1: Our method reconstructs 3D garments from multi-view images through fast dense point cloud reconstruction and Gaussian primitive optimization, achieving high-fidelity rendering effect while generating non-watertight meshes that preserve complex topological.

Gaussian Primitive Optimization

This core component involves three primary steps:

1. Movement: The Gaussian primitives are repositioned closer to their nearest corresponding dense point cloud locations using a K-D tree for nearest-neighbor queries.
2. Flattening: The transformation of Gaussian ellipsoids into 2D Gaussian disks is achieved by minimizing one of the scaling axes, enhancing the geometric precision on garment surfaces.
3. Rotation: The disks are aligned with the garment surface by adjusting their normal vectors to align with those of the dense point cloud points.

   Figure 2: Overview of our method. We first (a) pull scattered 3D Gaussian primitives towards their nearest points in the dense point cloud, while (b) flattening 3D Gaussian ellipsoids into 2D Gaussian elliptical disks, then (c) rotate the 2D Gaussian elliptical disks at corresponding positions according to the normal directions of points in the dense point cloud to align the Gaussian primitives precisely with the clothing surface.

Mesh Denoising

Post-reconstruction, the meshes often display unwanted inner layers. The method employs the Local Outlier Factor (LOF) to refine the mesh by removing redundant internal faces, resulting in a single-layer mesh fit for further application in fabric simulation.

Experimental Results

The method was evaluated against several existing state-of-the-art approaches like GSDF and PGSR on the DeepFashion3D dataset. The findings reveal that GarmentGS offers superior rendering quality and geometric accuracy, as evidenced by improvements in metrics such as SSIM, PSNR, and Chamfer Distance. Notably, it achieves non-watertight and wearable outputs, a significant advancement for simulations.

Figure 3: Comparison between ours and other methods on multi-view reconstruction on DeepFashion3D dataset. Top row: rendering effect. Bottom 3 rows: reconstructed meshes.

Implications and Future Work

GarmentGS sets the stage for more efficient industrial 3D garment synthesis applicable in virtual fashion, gaming, and augmented reality. By addressing both rendering fidelity and geometric structure, it opens avenues for integrating computational graphics into practical pipelines. Future research can explore expanding the model's capabilities across different types of garments or enhancing the fabric simulations using advanced physics models.

In conclusion, GarmentGS represents a significant progression in 3D garment reconstruction by leveraging point clouds with Gaussian optimization techniques, demonstrating promising directions for future development in high-fidelity virtual garment production.

Figure 4: Results for some garments. First column: ground truth. Second column: rendering effect. Remaining 5 columns: reconstructed meshes.