LeafFit: Editable Leaf Mesh Pipeline

• LeafFit is a pipeline that transforms single-plant 3D Gaussian Splatting reconstructions into editable, instanced leaf mesh assets using a shared thin-sheet template.
• It employs differentiable Moving Least Squares deformation and geodesic segmentation to achieve high accuracy and efficient shader-time runtime instancing.
• Empirical evaluations demonstrate superior segmentation metrics and significant memory and rendering efficiency improvements over traditional Gaussian-to-mesh conversions.

Searching arXiv for "LeafFit" and closely related papers to ground the article. LeafFit is a pipeline for converting a single-plant 3D Gaussian Splatting reconstruction into editable, instanced mesh assets. In its 2026 formulation, it segments leaves from an unstructured 3DGS scene, selects a representative leaf group, reconstructs a thin mesh template, and fits that template to the remaining leaves by differentiable Moving Least Squares deformation; at runtime, deformation is evaluated in a vertex shader so that only per-leaf control parameters need to be stored (Luo et al., 12 Feb 2026). The method is motivated by a specific incompatibility: while 3DGS faithfully captures complex foliage, its high memory footprint and lack of mesh topology make it difficult to integrate into conventional DCC and game-engine workflows (Luo et al., 12 Feb 2026).

1. Problem setting and representational assumptions

LeafFit assumes a trained single-plant 3DGS scene that has already been trimmed to remove background. Each Gaussian $g_i$ stores a 3D center $\mu_i \in \mathbb{R}^3$, a 3D covariance $\Sigma_i \in \mathbb{R}^{3\times 3}$, an opacity $\alpha_i \in [0,1]$, and view-dependent color coefficients $\beta_i$; for segmentation and registration, the set of Gaussian centers is treated as a point cloud (Luo et al., 12 Feb 2026).

The underlying problem is not plant reconstruction in the abstract, but the conversion of a foliage-heavy, point-like representation into an asset form compatible with mesh-centric pipelines. The paper identifies three bottlenecks in raw 3DGS for plants: large memory footprint, lack of explicit topology, and poor support for instancing despite the repetition of similar leaf shapes within a plant (Luo et al., 12 Feb 2026). Prior Gaussian-to-mesh conversions based on implicit fields are described as producing thickened leaf blades and dense meshes that are difficult to edit, while also struggling with sharp edges and front/back texturing (Luo et al., 12 Feb 2026). LeafFit addresses these issues by treating repeated leaves as instances of a shared template and moving deformation cost to shader-time evaluation.

The 3DGS rendering model used in the formulation follows standard screen-space Gaussian projection. For a splat projected to screen coordinates $\hat{u}_i$, the pixel-space opacity contribution at pixel $u$ is

$$w_i(u) = \alpha_i \exp\!\left(- \tfrac{1}{2} (u - \hat{u}_i)^T S_i^{-1} (u - \hat{u}_i)\right),$$

and the final color is accumulated by front-to-back alpha blending (Luo et al., 12 Feb 2026). In LeafFit, those rendered depth and color maps are later reused in the fitting losses.

2. Leaf segmentation from unstructured 3DGS

Leaf segmentation is training-free and is based on geodesic reasoning over Gaussian centers. A user first selects a root Gaussian near the stem, and LeafFit computes heat geodesic distances $D_r[i]$ from that root (Luo et al., 12 Feb 2026). The scene is then downsampled by farthest point sampling to a sparse set $\bar{G}$, and apex candidates are defined as strict local maxima of the root-distance field within a $\mu_i \in \mathbb{R}^3$0-NN graph (Luo et al., 12 Feb 2026).

For each apex, the algorithm traces a rootward path by greedy descent on the geodesic field. The paper emphasizes a deferred-merge rule: when a path approaches previously visited nodes, merging is postponed by an additional step so that noise does not cause premature branch fusion (Luo et al., 12 Feb 2026). This is followed by grouping multiple apexes that belong to the same leaf. The grouping criterion compares direct geodesic distance against the path through the lowest common ancestor:

$\mu_i \in \mathbb{R}^3$1

If the inequality holds by margin $\mu_i \in \mathbb{R}^3$2, the apexes are treated as belonging to the same leaf instance (Luo et al., 12 Feb 2026).

Leaf base estimation is performed by scanning rootward along the main apex path and measuring local width in narrow iso-geodesic bands,

$\mu_i \in \mathbb{R}^3$3

with local diameter

$\mu_i \in \mathbb{R}^3$4

Outside an initial protection window of fraction $\mu_i \in \mathbb{R}^3$5, the first point with $\mu_i \in \mathbb{R}^3$6 is declared the petiole or base (Luo et al., 12 Feb 2026). This produces per-leaf instance sets from the otherwise unstructured Gaussian field.

Automatic segmentation is supplemented by a manual fallback. A drag tool selects primitives using a distance field from a picked source point, a brush tool selects by geodesic radius, and GPU BVH acceleration supports real-time ray–Gaussian picking (Luo et al., 12 Feb 2026). This makes the system explicitly hybrid rather than purely automatic.

3. Template selection, thin-sheet reconstruction, and texturing

After segmentation, each leaf point set is denoised by MLS projection onto locally fitted planes, rejecting outliers far from the estimated sheet (Luo et al., 12 Feb 2026). A representative leaf $\mu_i \in \mathbb{R}^3$7 is then chosen manually as the template source. The template surface is reconstructed by downsampling $\mu_i \in \mathbb{R}^3$8 with farthest point sampling and running the Ball Pivoting Algorithm, followed by hole filling to obtain a clean, watertight mesh $\mu_i \in \mathbb{R}^3$9 (Luo et al., 12 Feb 2026).

This stage is central to LeafFit’s distinction from implicit-field baselines. The objective is not merely any mesh, but a thin triangular sheet that preserves sharp tips and supports separate appearance on the two sides of the blade. To do this, the template is PCA-aligned, orthographic depth and color are rendered from front and back, and textures are baked for both sides (Luo et al., 12 Feb 2026). Triangles are duplicated per side, normals are split, and separate UV patches are assigned to front and back. The result is a single-sheet representation with two-sided appearance, rather than a thick shell.

The paper states that thinness and sharpness are not enforced by explicit regularizers. Instead, thinness comes from the template reconstruction itself, and subsequent MLS warps preserve it as locally affine deformations (Luo et al., 12 Feb 2026). This point is important because it clarifies that geometric quality is front-loaded into template construction rather than added later as a penalty term.

4. Differentiable MLS fitting and runtime instancing

LeafFit fits the template to each target leaf using differentiable affine MLS. Each target leaf $\Sigma_i \in \mathbb{R}^{3\times 3}$0 is sparsified to $\Sigma_i \in \mathbb{R}^{3\times 3}$1 control points $\Sigma_i \in \mathbb{R}^{3\times 3}$2, with initial correspondences from the template controls $\Sigma_i \in \mathbb{R}^{3\times 3}$3 computed by a Jonker–Volgenant linear assignment (Luo et al., 12 Feb 2026). The fitting objective combines an orthographic depth loss and a bidirectional 3D Chamfer loss,

$\Sigma_i \in \mathbb{R}^{3\times 3}$4

with $\Sigma_i \in \mathbb{R}^{3\times 3}$5 (Luo et al., 12 Feb 2026).

For any point $\Sigma_i \in \mathbb{R}^{3\times 3}$6, the deformation is defined by Gaussian-kernel MLS weights

$\Sigma_i \in \mathbb{R}^{3\times 3}$7

weighted centroids $\Sigma_i \in \mathbb{R}^{3\times 3}$8 and $\Sigma_i \in \mathbb{R}^{3\times 3}$9, and matrices

$\alpha_i \in [0,1]$0

The affine transform is

$\alpha_i \in [0,1]$1

and the deformation field is

$\alpha_i \in [0,1]$2

All terms are differentiable in the target control points and kernel width, enabling gradient-based optimization (Luo et al., 12 Feb 2026).

At runtime, the method stores a shared template mesh, front/back textures, shared template controls $\alpha_i \in [0,1]$3, kernel width $\alpha_i \in [0,1]$4, and per-leaf target controls $\alpha_i \in [0,1]$5; no per-leaf meshes are stored (Luo et al., 12 Feb 2026). Vertex positions are deformed on the fly in a vertex shader, and normals are transformed by the inverse-transpose of the affine map (Luo et al., 12 Feb 2026). Storage therefore scales as shared $\alpha_i \in [0,1]$6 plus per-leaf $\alpha_i \in [0,1]$7, while per-instance computation is $\alpha_i \in [0,1]$8 and is parallelized on the GPU (Luo et al., 12 Feb 2026). The paper identifies $\alpha_i \in [0,1]$9 as an accuracy–efficiency sweet spot.

5. Empirical evaluation

The experiments use eight single-plant captures obtained with an iPhone 12 mini: Green Pepper (9 leaves), Rubber Tree (14), Golden Pothos (5), Black Pearl Pepper (11), Ruby Leaf (7), White Flag Bush (27), Cotton Rose (12, multi-lobed), and Lucky Bamboo (11, thin) (Luo et al., 12 Feb 2026). Segmentation is evaluated against Point Transformer v3 and a Euclidean-density heuristic.

Task	Method	Result
Segmentation	LeafFit	Acc 98.95 ± 0.58; mIoU 98.20 ± 0.95; mF1 99.08 ± 0.49; PQ 98.20 ± 0.95; Time 4.456 ± 2.724 s
Segmentation	Euclidean Density	Acc 97.26 ± 1.92; mIoU 95.16 ± 2.92; mF1 97.35 ± 1.75; PQ 94.96 ± 3.39; Time 3.786 ± 2.334 s
Segmentation	Point Transformer v3	Acc 94.52 ± 4.37; mIoU 86.88 ± 9.26; mF1 89.54 ± 8.29; PQ 89.36 ± 7.90; Time 16.534 ± 10.062 s

On mesh extraction and rendering, LeafFit reports a plant-level mesh asset at 11,980.11 FPS, 1.13 MB, and 2,432 vertices, with a leaf-level size of 0.334 MB and 2,048 vertices (Luo et al., 12 Feb 2026). The corresponding GOF and 2DGS mesh conversions are much larger, and the paper characterizes them qualitatively as producing thickened blades and denser triangulations (Luo et al., 12 Feb 2026).

For deformation accuracy, the full method outperforms PCA, NR-ICP, BCPD, and its own no-optimization variant. Reported values are Corr 0.0823, CD 0.0022, and HD 0.4669, versus 0.1186, 0.0061, and 0.7554 for BCPD, and 0.1115, 0.0050, and 0.6512 for the unoptimized LeafFit variant (Luo et al., 12 Feb 2026). The ablation over control points shows progressive improvement from $\beta_i$0 to $\beta_i$1, with $\beta_i$2 retaining most of the gain (Luo et al., 12 Feb 2026).

These results support two distinct claims. First, segmentation quality is high despite the method being training-free. Second, the compression mechanism is not merely storage reduction: it is coupled to faster rendering and to editable instancing.

6. Editing, limitations, and adjacent research

Because each leaf instance is represented as an MLS-deformed template, LeafFit supports parameter-level editing. The paper lists scale, bend, and twist modifications via control-point edits; phyllotaxis and arrangement edits via stem and instance transforms; and texture editing through the shared front/back UV textures, with changes propagating to all instances (Luo et al., 12 Feb 2026). It also describes cross-species retargeting using 2D cages and Mean Value Coordinates, followed by 3D back-projection and MLS fitting (Luo et al., 12 Feb 2026).

The system’s limitations are explicit. It requires an isolated single-plant 3DGS as input; stem reconstruction is simplified to cylinders and is not the focus; overlapping leaves can corrupt geodesic segmentation through horizontal merges or vertical shortcuts; dataset diversity is limited and predominantly elliptic; all leaves share the template texture unless edited; and the MLS stage has no explicit self-intersection constraints (Luo et al., 12 Feb 2026). These are architectural rather than incidental limitations.

A plausible implication is that LeafFit occupies a distinct position between plant phenotyping and graphics-oriented leaf modeling. In adjacent work, NeuraLeaf is a neural parametric model that disentangles a 2D base shape from a 3D deformation and learns UV-aligned texture for 3D leaves (Yang et al., 17 Jul 2025). In a different application domain, LeafInst addresses instance segmentation for fine-grained forestry leaf phenotyping in UAV RGB imagery through AFPN, DASP, DARH, and TCFU modules on the Poplar-leaf benchmark (Luo et al., 4 Mar 2026). LeafFit differs from both: it is neither a phenotyping system nor a generic neural leaf prior, but a 3DGS-to-asset conversion pipeline centered on instancing, shader-time deformation, and compatibility with production asset workflows (Luo et al., 12 Feb 2026).