In-Optimization Mesh Extraction

• In-Optimization Mesh Extraction is a method that integrates mesh generation directly into iterative optimization, enabling real-time gradient flow and adaptive mesh updates.
• It employs differentiable techniques such as marching cubes/tetrahedra and set-theoretic panel extraction to achieve efficient, design-driven mesh refinement.
• Applications span multidisciplinary design, inverse graphics, and Gaussian Splatting, enhancing reconstruction accuracy, structural performance, and visual fidelity.

In-Optimization Mesh Extraction refers to the class of algorithms and computational methodologies that integrate surface mesh extraction directly within iterative optimization processes. Rather than operating as a heuristic or post-hoc step, mesh extraction becomes a differentiable or combinatorial transformation embedded in the optimization loop, enabling the concurrent update of both mesh geometry and downstream task objectives. This paradigm spans applications in structural design optimization, photogrammetry, inverse graphics, and geometric learning. Methodologies include set-theoretic panel extraction for design decomposition, gradient-based isosurface extraction via differentiable marching algorithms, template-based mesh deformation, and direct supervision of mesh parameters through render-based or geometric losses.

1. Foundational Concepts and Motivations

In-optimization mesh extraction is motivated by the need to tightly couple mesh geometry with task-specific objectives during optimization, rather than treating mesh generation as a fixed preprocessing or offline post-processing step. The canonical goal is to ensure that the mesh representation—whether a structural panel, a manifold surface, or a volumetric discretization—adapts adaptively and differentiably to changing objectives, constraints, and data. This is critical in:

• Multidisciplinary Design Optimization (MDO), where local panels with evolving boundaries must be extracted repeatedly from a global FE model to localize design, analysis, and update steps (De et al., 2020).
• Visual-objective learning settings (e.g., 3D reconstruction, inverse rendering, neural fields), where the output mesh must support gradient flow from image or field-based losses directly to its geometry, ensuring the surface is optimized for accuracy, regularity, or physical objectives (Shen et al., 2023, Binninger et al., 7 May 2025, Knodt et al., 8 Jan 2026, Cui et al., 26 Apr 2026).
• Differentiable surface representations (e.g., in Gaussian Splatting, SDF rasterization, isosurface learning), where mesh extraction and parameter update are interleaved, enabling end-to-end learning pipelines (Guédon et al., 2023, Zhou et al., 11 Dec 2025).

2. Set-Theoretic Extraction in Structural Optimization

In distributed design optimization for thin-walled structures, a generalized set-theoretic framework for panel extraction from 2D FE meshes was established by De and Kapania. The domain is decomposed into panels via user-defined boundary curves, and each panel is extracted by a connectivity closure process:

• Nodes $\mathcal{N}$ and elements $\mathcal{E}$ are linked via connectivity maps (Conn).
• Given boundary curves $C_i$ (as node lists), boundary nodes $B_{C_i}$ and boundary elements $E_{C_i}$ (with at least 2 nodes on $C_i$) are identified.
• The closure over the adjacency graph iteratively grows the panel node and element sets, ensuring full inclusion of connected elements inside the specified boundary.
• A “mid-element” correction step discriminates inside from outside in internal boundaries.
• This process is purely index-based and agnostic to nodal coordinates (coordinate-free).
• Embedding this panel extraction in a distributed MDO loop allows for local panel-wise FEA and sizing/stiffener optimization per iteration, with negligible computational overhead relative to FE solves (De et al., 2020).

3. Differentiable Mesh Extraction in Learning-Based Optimization

Multiple recent works formulate mesh extraction as a differentiable operator suitable for gradient-based optimization, primarily in 3D applications but extensible to 2D and UV domains.

Marching Tetrahedra/Cubes and Differentiable Isosurfacing

Differentiable isosurface extraction constitutes the mainstream approach:

• Scalar fields (often SDFs) are parameterized on a grid (regular, octree, adaptive, or tetrahedralized point sets), and the zero-level set is meshed via marching cubes/ tetrahedra.
• In FlexiCubes, per-cell learnable parameters—edge weights $\alpha$, face weights $\beta$, quad-splitting weights $\gamma$, and grid deformation vectors $\delta$—modulate the surface extraction to preserve detailed features and adapt to complex losses. Extraction is via dual marching cubes, with all arithmetic steps differentiable (Shen et al., 2023).
• SDFRaster and TetWeave both deploy marching tetrahedra within the optimization loop on Delaunay tetrahedral grids; TetWeave further endows each point with directional SDFs via spherical harmonics, and adaptively resamples the point set where errors are high. Both guarantee manifold, watertight, intersection-free outputs, and support closed-form gradient flow back through the mesh extraction (Cui et al., 26 Apr 2026, Binninger et al., 7 May 2025).
• Grid-based methods with convex barycentric weighting (via per-edge nonnegative weights) facilitate locally injective deformation with explicit barrier energies, enabling inversion-free optimization of both vertex placements and isosurface extraction (Knodt et al., 8 Jan 2026).

Method	Surface Extraction	Grid Structure	Differentiable?	Key Innovations
FlexiCubes	Dual marching cubes	Regular + local deforms	Yes	Per-cell $\mathcal{E}$0, $\mathcal{E}$1
TetWeave	Marching tetrahedra	Delaunay-tet on-the-fly	Yes	Directional SDF, adaptive sampling
SDFRaster	Marching tetrahedra	Delaunay-tet fixed	Yes	SDF rasterization, in-loop extraction
Grid Deformation	Marching tetrahedra	Regular grid, convex comb.	Yes	Kernel-convexity, IPC-style barriers

Performance metrics include Chamfer distance, Hausdorff distance, mesh sliver percentage, and convergence criteria under gradient-based losses (e.g., photometric, geometric, fairness).

4. In-Optimization Mesh Extraction in Gaussian Splatting and Hybrid Mesh-Splat Frameworks

Recent advances in 3D Gaussian Splatting have motivated efficient, consistent mesh extraction during or immediately after optimization.

• SuGaR introduces a surface-aligned regularizer into the Gaussian Splatting loop, regularizing opacity and density alignment, then extracts a mesh by sampling the level set of the learned density and running Poisson reconstruction on the resulting oriented points. This stage is conducted within minutes on GPU hardware, and yields editable meshes that can be further refined by “binding” new mesh-tied Gaussians, enabling backward gradient flow to both splat and mesh geometry (Guédon et al., 2023).
• DeMapGS attaches Gaussians to a deformable mesh template, enabling simultaneous optimization of mesh vertex positions (via template deformation) and surface attributes (via splats). Because vertex updates and attribute mapping are executed together, (V,F) is always valid at any iteration, and mesh extraction—along with UV-mapped attributes—is immediate and differentiable (Zhou et al., 11 Dec 2025).

Method	Mesh Extraction Stage	Mesh Editable?	Downstream Utility
SuGaR	Post regularized-splat opt.	Yes	Sculpting, anim., relighting
DeMapGS	Template updated each iter.	Yes	Texturing, manipulation

Both frameworks demonstrate the feasibility and efficiency of in-optimization mesh extraction in hybrid explicit–implicit scene representations.

5. Adaptive and Memory-Efficient Mesh Representation

A key challenge in mesh-based optimization is the scaling of memory and computational cost with mesh resolution and surface complexity.

• TetWeave introduces an on-the-fly Delaunay tetrahedralization and error-driven point resampling, ensuring near-linear memory scaling ($\mathcal{E}$2) relative to the mesh output size (Binninger et al., 7 May 2025).
• SDFRaster and FlexiCubes employ adaptive grid pruning and densification strategies to focus resources only near the zero-level set, reducing unnecessary parameterization of empty space or low-detail regions (Cui et al., 26 Apr 2026, Shen et al., 2023).
• In DeMapGS, CUDA-based parallelization by mesh face and localized splat neighborhoods ensures that mesh extraction and UV/texture computation are feasible for large templates and high-resolution attributes (Zhou et al., 11 Dec 2025).

These methods address the inefficiencies of classical voxel or uniform-grid-based approaches, which inherently scale with the volume rather than the surface.

6. Integration into Optimization Pipelines

Embedding mesh extraction directly into optimization workflows enables end-to-end differentiation, rapid design iteration, and application-specific tailoring.

• In distributed structural optimization (e.g., aircraft wing with curvilinear ribs/spars), panel extraction is used every iteration: boundaries evolve as design variables are updated, panels are extracted, analyzed, and design variables are locally re-optimized; results are assembled for global FE solves and aeroelastic updates (De et al., 2020).
• In neural surface representations, isosurface extraction, mesh-based rendering, field-based losses, and attribute mapping are intertwined in a loop with mesh geometry, grid/point parameters, and SDF values all updated together (Shen et al., 2023, Binninger et al., 7 May 2025, Cui et al., 26 Apr 2026).
• Gaussian Splatting pipelines transition from splat-only optimization to mesh-bound splat refinement post-alignment, with gradient backpropagation into both representations for fine detail recovery and downstream manipulation (Guédon et al., 2023, Zhou et al., 11 Dec 2025).

Optimization pipelines typically use automatic differentiation frameworks (e.g., PyTorch, Taichi), and are compatible with modern hardware acceleration via CUDA or custom JIT compilation, facilitating thousands of iterations and high-resolution outputs.

7. Quantitative Evaluation and Limitations

The effectiveness of in-optimization mesh extraction is validated through several quantitative and qualitative metrics:

• Chamfer/edge distances, F1-scores, and Hausdorff distances measure reconstruction accuracy (Shen et al., 2023, Knodt et al., 8 Jan 2026, Cui et al., 26 Apr 2026).
• Mesh quality indicators (aspect ratio, sliver triangle percentage, normal deviation) assess intrinsic geometric fidelity.
• Render-based metrics (PSNR, SSIM, LPIPS) evaluate visual fidelity, especially in photogrammetric and view-synthesis applications (Guédon et al., 2023, Zhou et al., 11 Dec 2025).
• Memory and runtime profiling show near-linear or quadratic scaling versus cubic scaling in voxelized approaches (Binninger et al., 7 May 2025, Shen et al., 2023).
• Extraction times: mesh generation in minutes (SuGaR, DeMapGS) versus hours or days for classical SDF pipelines (Guédon et al., 2023, Zhou et al., 11 Dec 2025).

Limitations, as reported, include possible “bridge-walking” across complex 2D panel boundaries (set-theoretic methods), topological discontinuities during isosurface flips (FlexiCubes), and the need for closed-form or auto-differentiable mappings throughout the extraction process.

• De and Kapania, "Algorithms for 2D Mesh Decomposition in Distributed Design Optimization" (De et al., 2020)
• "Flexible Isosurface Extraction for Gradient-Based Mesh Optimization" (Shen et al., 2023)
• "TetWeave: Isosurface Extraction using On-The-Fly Delaunay Tetrahedral Grids for Gradient-Based Mesh Optimization" (Binninger et al., 7 May 2025)
• "Differential Locally Injective Grid Deformation and Optimization" (Knodt et al., 8 Jan 2026)
• "DeMapGS: Simultaneous Mesh Deformation and Surface Attribute Mapping via Gaussian Splatting" (Zhou et al., 11 Dec 2025)
• "Distance Field Rasterization for End-to-End Mesh Reconstruction" (Cui et al., 26 Apr 2026)
• "SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering" (Guédon et al., 2023)