LeafFit: Plant Assets Creation from 3D Gaussian Splatting

Abstract: We propose LeafFit, a pipeline that converts 3D Gaussian Splatting (3DGS) of individual plants into editable, instanced mesh assets. While 3DGS faithfully captures complex foliage, its high memory footprint and lack of mesh topology make it incompatible with traditional game production workflows. We address this by leveraging the repetition of leaf shapes; our method segments leaves from the unstructured 3DGS, with optional user interaction included as a fallback. A representative leaf group is selected and converted into a thin, sharp mesh to serve as a template; this template is then fitted to all other leaves via differentiable Moving Least Squares (MLS) deformation. At runtime, the deformation is evaluated efficiently on-the-fly using a vertex shader to minimize storage requirements. Experiments demonstrate that LeafFit achieves higher segmentation quality and deformation accuracy than recent baselines while significantly reducing data size and enabling parameter-level editing.

• The paper proposes LeafFit which uses geodesic-guided segmentation and template-driven MLS deformation to transform 3D Gaussian plant reconstructions into editable mesh assets.
• The method achieves high segmentation accuracy (mean IoU 98.2%) and significant compression (1.13 MB per plant mesh) while preserving thin-leaf details for real-time rendering.
• The pipeline enables efficient parameter-level texture editing and retargeting, bridging high-fidelity captures with scalable virtual environment workflows.

LeafFit: Instance-Aware Plant Asset Creation from 3D Gaussian Splatting

Motivation and Challenges in Plant Asset Generation

Plant asset generation for virtual environments presents unique challenges, especially regarding thin-leaf structures and repetitive foliage within individual specimens. Existing manual modeling and procedural techniques often fail to scale efficiently for real-time rendering or to capture morphological diversity found in nature. 3D Gaussian Splatting (3DGS) has recently emerged as a high-fidelity, efficient scene representation, but lacks explicit mesh topology, suffers from a high memory footprint, and is incompatible with most game engine workflows. Implicit surface conversion approaches, such as Marching Cubes or opacity field extraction, introduce excessive geometry redundancy, lack structural semantics, and fail to preserve thin-leaf sharpness.

LeafFit Pipeline: Instance-Aware Extraction and Registration

LeafFit proposes a modular pipeline for converting unstructured 3DGS plant reconstructions into editable, instanced mesh assets through three main stages: (i) geodesic-guided segmentation; (ii) template-driven non-rigid alignment; and (iii) lightweight surface extraction with seamless texture transfer.

Segmentation uses geodesic distance fields for robust leaf instance identification, leveraging the spatial repetition of leaves within a single plant (Figure 1).

Figure 1: No two leaves belong to a plant are exactly the same, but they are similar. Compression is achieved by instancing deformed template leaves.

Automated segmentation is complemented by manual, interactive tools for handling ambiguous or overlapping leaves, utilizing BVH-accelerated picking for efficient real-time editing (Figure 2).

Figure 2: Manual segmentation tools enable geodesic-driven drag and brush selection for granular leaf instance refinement.

Geodesic-based segmentation constructs a rootward tree by tracing apex-to-root paths in sampled Gaussian primitives, employing deferred-merge rules to mitigate path noise and ambiguous junctions. Leaf apexes are grouped based on the triangle inequality margin over direct and tree-based geodesic paths, forming connected components corresponding to leaf instances (Figure 3 and Figure 4).

Figure 3: Tree-graph construction and apex detection enable robust partitioning of Gaussian primitives.

Figure 4: Triangle inequality-based separation of apex pairs distinguishes overlapping leaf groups.

Leaf bases (petioles) are determined by traversing rootward paths and monitoring local diameter drop-offs within iso-geodesic bands (Figure 5).

Figure 5: Geodesic diameter analysis along rootward paths reveals leaf petiole positions.

Template Leaf Mesh Creation and Deformation via MLS

Once segmentation yields instances, the user selects a template leaf, which is meshed via ball-pivoting to maximize fidelity in thin sheet topology. Deformation employs differentiable Moving Least Squares (MLS) parameterized by per-leaf control points, optimized for depth and Chamfer distance alignment with target leaf point clouds.

MLS deformation is evaluated in real-time using a vertex shader at runtime, minimizing CPU storage by only retaining per-leaf kernel and template mesh data. Mesh extraction qualitative comparisons demonstrate superior preservation of thin structures versus implicit baselines (Figure 6).

Figure 6: Mesh extraction qualitative comparison highlights preservation of thin blades and sharp tips compared to implicit surface conversion methods.

Non-rigid alignment benchmarks against PCA, NR-ICP, and BCPD show consistently lower correspondence errors and Chamfer/Hausdorff distances (Figure 7).

Figure 7: Deformation qualitative comparison across six species: MLS-warped template exhibits minimal distortion and aligns boundaries accurately.

Texture and Geometry Editability

LeafFit’s design, centered on a single template mesh, decouples appearance from geometry, supporting parameter-level edits and retargeting. Uniform UV layouts facilitate texture propagation, where edits or inpainting on the template propagate consistently to every leaf instance (Figure 8).

Figure 8: Editing and retargeting workflows support geometry retargeting and full texture replacement for all leaf instances.

For geometry retargeting across species, orthographic rendering and mean value coordinates define cages for robust control point correspondence, enabling efficient MLS deformation even for large shape gaps (Figure 9).

Figure 9: Correspondence for leaf retargeting across species leverages silhouette-driven MVC cage transfer.

Quantitative Evaluation and Limitations

Strong quantitative results validate segmentation accuracy (mean IoU 98.2%, PQ 98.2%); mesh representations achieve significant compression (plant mesh storage size 1.13 MB, FPS 11,980.11), outperforming implicit baselines in both memory and rendering speed.

LeafFit is training-free and outperforms learned and heuristic baselines for segmentation even on unseen morphologies. Deformation metrics (Corr, CD, HD) show that MLS optimization is crucial for alignment fidelity, with diminishing returns beyond moderate ($K = 32$) control points.

The method’s main limitations concern handling topological ambiguity in dense overlapping foliage, reliance on cleaned-up Gaussian input, and limited dataset diversity predominantly in simple ellipsoidal leaves. Texture instancing across leaves lacks per-leaf variation.

Figure 10: Overlapping leaves introduce segmentation errors via ambiguous geodesic relations.

Implications and Future Directions

Practically, LeafFit enables scalable, editable asset generation for real-time virtual environments by bridging high-fidelity Gaussian captures with mesh-based pipelines. The lightweight framework supports parameter-level manipulation, retargeting, and seamless artist-driven iteration, extending the utility of 3DGS beyond image-space rendering.

Theoretically, the pipeline highlights opportunities for leveraging geometric shape priors, intrinsic symmetries, and cage-based correspondence for improved instance extraction and registration. Further development should explore joint optimization with learning-based approaches for occlusion-robust segmentation, and incorporate richer appearance variations via disentangled template texture refinement.

Conclusion

LeafFit demonstrates the efficacy of geodesic-guided segmentation and template-driven MLS deformation for compact, instance-aware plant asset creation from 3D Gaussian Splatting (2602.11577). The pipeline achieves robust segmentation, high-fidelity mesh extraction, real-time deformation, and seamless editability, validating the principle that morphological repetition in plant structure can be efficiently exploited for scalable virtual asset generation. The work establishes a foundation for future research in leveraging structured geometric priors and learning-driven regularization to further advance procedural plant modeling.