Semantically Layered 3D Mesh Representation

• The paper introduces a novel pipeline that extracts planar primitives and constructs a semantic LOD–Tree to balance geometric fidelity with efficient multi-resolution urban modeling.
• It employs region growing, α–shapes, and multi-stage clustering to differentiate principal and secondary structures, ensuring semantically coherent level-of-detail generation.
• Empirical results on urban datasets demonstrate reduced mesh fragmentation, improved structural fidelity, and faster candidate selection for interactive VR and digital twin applications.

A semantically layered 3D mesh representation is an approach that structures geometric data together with rich, explicitly organized semantic information—such as physical, functional, or relational attributes—enabling both robust geometry processing and high-level scene understanding. This paradigm balances the needs of geometric fidelity, efficiency, and semantic interpretability by partitioning models into hierarchical or multi-resolution structures where semantic groupings (e.g., principal building shells, doors, windows) are encoded as distinct, queryable layers or levels-of-detail. These representations are pivotal for applications in urban modeling, AR/VR, robotics, and digital twins, where both efficiency and semantic meaning are imperative.

1. Pipeline Overview: Primitive Detection to Semantic LOD Construction

The proposed algorithm (Pan et al., 21 May 2025) initiates with the extraction of planar primitives from an oriented mesh or point cloud. These primitives are detected using a tailored region growing algorithm. Subsequently, IO-View analysis via α-shapes is performed on the planar segments to fill holes and identify true boundaries. This enables the discrimination of the core/principal structure (core interior/exterior) against supplementary structures (windows, balconies). After IO-View processing, secondary structures are grouped using a two-stage mean-shift clustering—first by projected area, then by average volume—producing a series of semantically coherent level sets.

The set of planar primitives is then reorganized as scale-sorted sets: S₀ (principal) and S₁, S₂, …, S_N (secondary/finer). These define the sequence for a binary space partitioning (BSP) procedure, which recursively splits the model’s bounding region along planar primitives, followed by merging BSP nodes when primitives are too fine (below a threshold, e.g., K=10) for individually meaningful subdivision. The result is encapsulated in an LOD–Tree—a semantic-aware hierarchical structure with nodes at increasing granularity mapping to increasingly detailed sets of 3D primitives or groups.

Traversal of this LOD–Tree employs a priority queue, where each node is assigned a diff–value:

$$d(n, LN(n)) = | x_k(n) V(n) - \sum_{m \in LN(n)} x_k(m) V(m) |$$

where $x_k(\cdot)$ reflects in/out labeling and $V(\cdot)$ denotes node volume. Nodes with high diff–value are recursively expanded, yielding a family of candidate LOD representations.

2. Semantic Layering of Urban 3D Meshes

Semantic layering is achieved by explicitly partitioning the model into meaningful volumetric and surface groupings based on geometric–semantic cues. Core IO–View analysis distinguishes:

• Principal primitives: Faces separating the “core interior” $V_{\mathrm{in}}$ and “core exterior” $V_{\mathrm{out}}$ (building skeleton).
• Secondary primitives: Volumes that represent features such as windows, balconies, or chimneys, identified as addon or cutout volumes during α–shape processing.

These features are subjected to hierarchical clustering (projected area and then mean volume) to form clusters and multi-level sets ($L_1, L_2, \ldots$), ensuring that elements of similar semantic significance are grouped. The resulting representation becomes semantically layered: principal volumes define the major spatial envelope; secondary clustered features correspond to details relevant to energy simulation, urban analytics, or AR navigation.

This grouping enables selective emphasis of detail relevant to particular applications (simulation of facades vs. navigation around entryways) and aligns with established urban standards (e.g., CityGML’s LOD taxonomy).

3. LOD–Tree Structure: Hierarchical and Semantic Organization

The LOD–Tree improves upon classical BSP trees by using semantically ordered set S (principal to secondary), prioritizing splits on primitives representing salient features. Nodes with trivial splits (few primitives) are merged, producing multi-branch nodes that correspond to architecturally coherent elements (rectilinear walls, grouped windows).

• Upper LOD-tree nodes (root, higher levels): Represent coarse geometries—e.g., entire roofs or block facades.
• Lower LOD-tree nodes: Encode fine details—e.g., window bays, door recesses, fine add-on structures.

Traversal is performed greedily by expanding the priority-queue node with the maximal diff–value at each step. The process continues until the cumulative diff–value is below a designated threshold, or all nodes have been resolved to “anchor” models (maximal detail at the queried level).

This structure enables fine control over the resolution and semantic content exposed in any generated LOD model.

4. Noise Robustness and Topological Integrity

The robustness of the method to noise and artifacts is ensured by adaptive regularization strategies:

• Mild regularization is applied for principal primitives (tighter angular and distance constraints) so their geometry is preserved.
• Stronger regularization is implemented for the more noise-affected secondary primitives (relaxed angular constraint up to 15°), with automatic replacement by templates for poorly captured regions.
• Detection thresholds during region growing are set to small values to minimize false positives from noisy data.
• Merging of BSP nodes compensates for missing or corrupted primitives so that minor irregularities do not produce fragmented or topologically inconsistent LOD nodes.

Experiments demonstrate that, even with substantial added Gaussian noise (σ up to 0.2 m), stable and clean LOD models are produced, and detailed architectural features (overhangs, stairs) are also successfully recovered.

5. Experimental Validation and Quantitative Assessment

Empirical analysis on 21 real-world urban datasets demonstrates that the LOD–Tree achieves:

• Substantially reduced number of space cuts compared to classical BSP trees (e.g., 5,961 vs. 25,883 cuts in one complex case).
• High structural fidelity and low error metrics (RMSE, visual error, Pareto front) for LOD models at varying resolutions, with less geometric error at coarse levels than classical geometric simplification baselines (Lowpoly, QEM, NeuralLOD).
• Time-efficient candidate selection (faster than baselines) and higher subjective quality in user studies, attributed to early semantic grouping and reduced mesh fragmentation.

The approach supports watertight, concise meshes well-suited for interactive rendering, streaming, and further semantic augmentation.

6. Applications: VR, Digital Twins, and Urban Analytics

The semantically layered LOD–Tree supports a wide range of applications:

• Virtual and Augmented Reality: Enables dynamic switching among LODs based on viewpoint or resource constraints, delivering efficiency during interactive navigation in rich urban scenes.
• Urban Simulation/Navigation: Underpins applications such as solar modeling and emergency response, where accurate differential detail by semantic type (structural envelope vs. facade components) is essential.
• Digital Twins and CityGML: Semantic level-sets map directly to formal LOD classes (e.g., walls/roofs, then facade details) as defined in digital twin and urban scene standards.
• Mobile/Embedded Systems: Concise LOD–Tree outputs are efficient in memory and render rapidly.

7. Significance, Limitations, and Future Directions

The approach demonstrates that semantic grouping of planar primitives, enforced via IO–View analysis and multi-stage clustering, is critical for constructing LOD hierarchies that are both efficient and semantically meaningful. The method is robust to real-world data imperfections and scalable to large urban models.

Limitations noted include the dependency of semantic layering completeness on the quality of initial primitive extraction and the challenge of distinguishing semantically similar but geometrically ambiguous structures (e.g., distinguishing decorative facade features from windows in noisy data). Extensions may encompass explicit integration of domain ontologies or further learning-based refinement of semantic groupings.

In summary, semantically layered 3D mesh representation via a hierarchical LOD–Tree founded on geometric–semantic analysis constitutes a highly practical, robust, and scalable solution for multi-resolution urban modeling, supplying both geometric efficiency and rich semantic interpretability for a spectrum of advanced applications (Pan et al., 21 May 2025).