3D Mesh Saliency Ground Truth Methods

• 3D mesh saliency ground truth is a quantitative measure of human visual attention on 3D surfaces using VR and 6DoF eye-tracking experiments.
• It employs advanced gaze event processing, including ray–mesh intersection, adaptive fixation detection, and geodesic smoothing to generate continuous saliency distributions.
• The methodology enhances 3D vision algorithms by providing robust benchmarks through metrics like correlation coefficient, KL divergence, and internal consistency.

3D mesh saliency ground truth (GT) is the quantitative annotation or measurement of human visual attention across the surfaces of 3D meshes, serving as the benchmark for training and evaluating saliency prediction algorithms in immersive and computer vision contexts. Unlike 2D saliency maps, mesh saliency GT must handle the spatial, geometric, and topological complexities of 3D surface data, and is typically derived from large-scale eye-tracking experiments with human subjects interacting with mesh stimuli in virtual reality (VR). Mesh saliency ground truth datasets, protocols, and processing pipelines underpin the development of human-aligned, deployable 3D saliency models.

1. Experimental Design and Eye-Tracking Protocols

Acquisition of mesh saliency GT requires tightly controlled experimental setups that capture both gaze and 6-degree-of-freedom (6DoF) head pose in VR environments. Recent protocols employ head-mounted displays (e.g., HTC Vive Pro Eye, HTC Vive with aGlass tracker) with integrated eye-trackers operating at 120 Hz or higher (Zhang et al., 2 Apr 2025, Ding et al., 2020). Participants, typically naïve to computer graphics, observe 3D meshes rendered on VR turntables or placed freely in VR rooms. Standardized procedures randomize viewing order and conditions (e.g., textured vs. non-textured), mandate centered fixation resets before trials, and allow subjects unrestricted head and body motion.

Raw data streams encompass:

• Eye origin and direction (per-frame)
• Head position and orientation (6DoF)
• Model identity and metadata

Key innovations include immersive free movement, multiple visual conditions (texture, flat shading), and the explicit logging of both gaze vectors and participant 6DoF navigation (Zhang et al., 2 Apr 2025, Ding et al., 2020). The 6DoF mesh saliency database (“6DoFMS”), for example, stores per-frame {position, orientation, gaze offset}, fixation clusters, and per-pose saliency maps (Ding et al., 2020).

2. Gaze Event Processing and Fixation Extraction

Transforming raw gaze streams into actionable GT relies on event segmentation and spatial filtering. The typical process comprises:

• Ray–mesh intersection: Each gaze vector is intersected with the mesh (Möller–Trumbore algorithm with BVH acceleration).
• I-VT fixation detection: Gaze points are labeled as fixations or saccades based on adaptive velocity thresholds, which may be scaled by participant-mesh distance (Zhang et al., 2 Apr 2025, Ding et al., 2020).
• Outlier rejection: Rays missing the mesh or lying outside the display field are discarded.
• Spatial binning: Duplicate fixation events within a specified surface threshold (e.g., <0.5 mm) are merged.
• Clustering: Fixation points can be post-processed via random-walk affinity clustering to identify stable fixation centers on the mesh surface (Ding et al., 2020).

In 6DoF-aware settings, head pose and gaze offset are combined to compute accurate 3D sight rays per frame, and participant revisit patterns are leveraged to aggregate fixations across discrete "head-poses" (Ding et al., 2020).

3. Surface Density Estimation and Smoothing Methods

After fixation extraction, gaze events are diffused across the mesh to yield continuous saliency distributions. Two main paradigms are employed:

A. Gaussian-Based Surface Smoothing

Each fixation point $f_i$ contributes locally via a surface-based Gaussian kernel with a specified angular aperture (e.g., $\theta = 1^\circ$). Surface saliency $S(v)$ at vertex $v$ is accumulated as:

$$S(v) = \sum_{i=1}^N \exp\left(-\frac{d_{\mathrm{geo}}(v, f_i)^2}{2\sigma^2}\right)$$

where $d_{\mathrm{geo}}$ is mesh geodesic distance and $\sigma$ is set according to the desired spread (e.g., kernel falls to 0.1 at $\pm0.5^\circ$, $\sigma \approx 0.9$ mm) (Zhang et al., 2 Apr 2025). Per-face saliency is then computed by averaging over vertices.

B. View Cone Sampling and Hybrid Smoothing

Recent advances address the limitations of single-ray sampling and Euclidean smoothing. The view cone sampling (VCS) strategy replaces each gaze ray with a bundle of rays distributed according to a truncated Gaussian within a 5° cone to mimic human foveal vision. Rays are filtered by incidence angle to avoid grazing artifacts (Zheng et al., 6 Jan 2026). Hits are accumulated per-face, not per-point.

The hybrid manifold–Euclidean constrained diffusion (HCD) algorithm propagates these hits using Gaussian diffusion restricted to geodesic neighborhoods (BFS up to $d_{\max}=3\sigma_2$), followed by Laplacian smoothing across mesh vertices. This methodology prevents topological "short circuits"—the leakage of saliency signals across disconnected or thin mesh regions—by strictly enforcing manifold adjacency (Zheng et al., 6 Jan 2026).

The overall process for robust mesh saliency GT generation can be schematized as:

Step	Description	Key Parameter/Equation
VCS Sampling	Gaze replaced by Gaussian ray bundle	$R_f$ (5°), $\sigma_1 = R_f/6$
Hit Accumulation	Per-face raw hit count	$S_{\text{raw}}(f_i)$
Geodesic Diffusion	Kernel-weighted sum over face graph	$S_{\text{diff}}(f_j)$
Vertex Projection	Averaging diffused face saliency over vertex adjacencies	$S_{\text{vertex}}(v)$
Laplacian Smoothing	Iterative vertex-level blending	$\lambda$ (0.1–0.2), $K$ (5–10)
Gamma Correction	Nonlinear dynamic range enhancement	$\gamma = 0.5$

4. Ground Truth Formatting, Data Organization, and Accessibility

Mesh saliency GT is distributed in formats that enable scalable usage in research and benchmarking. Standard conventions include:

• Mesh files: 3D models (OBJ/OFF), spanning diverse object classes, provided at native face/vertex counts.
• Saliency data: Stored as NumPy arrays with per-face or per-vertex floating-point saliency values. For example, each mesh is paired with 1×F arrays, with F being the number of faces (Zhang et al., 2 Apr 2025).
• Metadata: CSVs encode model identity, category, partition (train/test), and statistics (face/vertex counts), as well as anonymized subject demographics (Zhang et al., 2 Apr 2025).
• Code bases: Repositories include Python/MATLAB scripts for data loading, visualization (e.g., VTK overlays), and metric computation.
• Viewpoint-conditioned maps: In the 6DoFMS dataset, GT saliency maps are indexed by discrete head-poses, allowing precise view-dependent evaluation (Ding et al., 2020).
• Open access: Databases are publicly released under liberal licenses (e.g., MIT), enabling widespread academic and commercial adoption (Zhang et al., 2 Apr 2025).

5. Evaluation Metrics for Mesh Saliency Prediction

Performance metrics for mesh saliency GT emphasize quantitative correspondence between predicted and empirically derived saliency distributions. Standard metrics include (Zhang et al., 2 Apr 2025, Zheng et al., 6 Jan 2026, Ding et al., 2020):

• Correlation Coefficient (CC): Linear agreement between predicted $P$ and GT $G$ normalized so $\sum_i p_i = \sum_i g_i = 1$.

$$CC(P,G) = \frac{\sum_i (p_i - \mu_P)(g_i - \mu_G)}{\sigma_P \sigma_G}$$

• Histogram Intersection/Similarity (SIM): Overlap of predicted and ground truth histograms.

$SIM(P,G) = \sum_i \min(p_i, g_i)$

• Kullback–Leibler Divergence (KL): Information-theoretic divergence from GT to prediction.

$KL(G\|P) = \sum_i g_i \log \frac{g_i}{p_i}$

• Saliency Error (SE): Mean squared (or L1) error across all faces or vertices.

$SE(P,G) = \frac{1}{F}\sum_{i=1}^F (p_i - g_i)^2$

• Shuffled AUC (sAUC): Assesses fixation discrimination under controlled random sampling (Zheng et al., 6 Jan 2026).
• Internal Consistency (IC): Agreement between odd/even subject splits.

The 6DoFMS protocol introduces pose-weighted aggregation: metrics are averaged across head-poses, weighted by the number of visits, to support viewpoint-aware benchmarking.

6. Comparison of Methodological Advances and Contemporary Benchmarks

Single-ray intersection with Euclidean smoothing, the traditional paradigm, has demonstrable limitations: strong aliasing on high-frequency textures, topological leakage across nonadjacent mesh regions, and unstable performance on large or sparse meshes (Zheng et al., 6 Jan 2026). The VCS+HCD pipeline directly addresses these issues by:

• Simulating the spatial extent of human foveal vision, reducing sample sparsity,
• Enforcing geodesic locality during saliency diffusion, and
• Combining mesh-manifold and Euclidean smoothing to retain both topological faithfulness and perceptual smoothness.

In direct comparison, VCS+HCD provides a 2.45× increase in correlation coefficient, a 3× reduction in KL divergence, and a substantial boost in internal consistency (CC rises from 0.1970 to 0.4829 and IC from 0.0557 to 0.8137) (Zheng et al., 6 Jan 2026). VCS+HCD also achieves a 31× improvement in coverage on high-face-count meshes.

The 6DoFMS database introduced pose-conditional GT maps, supporting the evaluation of saliency algorithms under unconstrained subject navigation and head movement. Saliency patterns are empirically found to decorrelate as viewpoint angle diverges, indicating the importance of 6DoF-aware evaluation (Ding et al., 2020).

7. Applications and Future Directions

Mesh saliency GT datasets and methodologies support:

• Training and validation of attention-aware 3D vision systems, including non-textured and textured mesh models (Zhang et al., 2 Apr 2025).
• Development of unified state-space models and manifold-aware architectures (e.g., Mesh Mamba) that integrate geometric and appearance cues.
• Benchmarking performance and analyzing inter-observer bias, uniqueness, center, and depth preferences in 3D attention (Ding et al., 2020).
• Robust model selection and cross-validation across novel object classes, mesh resolutions, and VR interaction paradigms.

Future advances are likely to further integrate perceptual modeling, context-dependent attention, and dynamic interaction, leveraging the increased granularity and fidelity afforded by VR-based, 6DoF mesh saliency GT (Zheng et al., 6 Jan 2026, Zhang et al., 2 Apr 2025, Ding et al., 2020).