Mesh2HRTF Integration Methods

• Mesh2HRTF Integration is a process that combines high-resolution scans, photogrammetry, and ML-enhanced upsampling to synthesize personalized HRTFs using boundary element methods.
• It leverages adaptive mesh grading and rigorous preprocessing to optimize simulation accuracy and reduce mesh complexity by up to 10× without sacrificing spectral fidelity.
• Evaluation utilizes ITD, ILD, and LSD metrics, confirming that enhanced geometric fidelity and tailored simulation parameters significantly improve spatial audio reproduction.

Mesh2HRTF Integration refers to the incorporation of the open-source Mesh2HRTF software toolbox into workflows that generate individual Head-Related Transfer Functions (HRTFs) from 3D head and pinna surface meshes. Mesh2HRTF numerically computes HRTFs from user-supplied geometry using boundary element methods (BEM), adapted finite mesh representations, and tailored preprocessing. Integration workflows now encompass classical high-resolution scan-based pipelines, photogrammetry-reconstructed (PR) mesh pipelines, and machine-learning-enhanced hybrid pipelines that address typical geometric and spectral limitations in low-cost mesh acquisition.

1. Pipeline Structures and Modalities

Mesh2HRTF integration pipelines take varied forms depending on the source and fidelity of the underlying mesh geometry:

• Classical Pipeline: Utilizes high-resolution 3D head and pinna scans (laser or structured-light), followed by mesh preprocessing (clean-up, re-meshing, region labelling, and grading), then BEM-based HRTF synthesis within Mesh2HRTF. Ground-truth HRTF comparison and auditory validation are integral steps (Zolfaghari et al., 2014).
• Photogrammetry Pipeline: Uses dense multi-view photographic capture (typically 72 RGB + depth images at 5° azimuth increments) processed via Apple’s Photogrammetry or Object Capture API, producing PR meshes that are then preprocessed (alignment, “beheading,” artifact removal, grading, labelling). Preprocessed meshes are input into Mesh2HRTF for numerical HRTF computation and evaluation against measured and random HRTFs (Pirard et al., 25 Mar 2026).
• ML-Enhanced Pipeline: Addresses intrinsic limitations of PR meshes by applying a trained graph neural network (GNN) to upsample low-resolution PR meshes into high-resolution proxies before HRTF synthesis. The upsampled (GNN-refined) meshes follow the same Mesh2HRTF preprocessing and simulation sequence. Comparison to measured HRTFs, laser-scan HRTFs, and KEMAR anthropometric HRTFs finalizes validation (Pirard et al., 3 Oct 2025).

2. Mesh Acquisition, Preprocessing, and Grading

Accurate HRTF synthesis critically depends on mesh acquisition, geometric fidelity, and preprocessing methodology:

• Acquisition Modalities: Mesh inputs derive from high-end laser scans (reference), consumer-grade photogrammetry, and hybrid ML reconstructions. Typical PR mesh densities after grading: ~80 000 vertices/160 000 faces; high-res scans: ~250 000 vertices/500 000 faces. Fine details in the pinnae and canal regions are often under-resolved in PR meshes (Pirard et al., 25 Mar 2026).
• Preprocessing Steps: All pipelines require rigid alignment (landmark-based + ICP; standard axes: x forward, y right, z up), “beheading” (removal of torso/neck below C7), removal of non-manifold edges and floating artifacts (MeshLab, Blender), and curvature-adaptive mesh grading.
• Curvature-Adaptive Mesh Grading: Following (Ziegelwanger et al., 2016), edge lengths are adaptively set via power-law or raised-cosine grading functions (e.g., COS2 with ℓ_min ≈ 1–2 mm at the microphone patch or pinna and ℓ_max ≈ 8–20 mm in flatter or distal regions), yielding ∼7–10× reduction in element count at <1 dB spectral error and <1° polar/quad error increase compared to uniform fine meshes.

Pipeline	Mesh Source	Grading Method	Typical Input Faces
Classical	Laser scan	COS2, POW1, POW4, etc.	500 000
Photogrammetry	Apple API (PR)	COS2, ℓ_min=0.5 mm	160 000
ML-Enhanced	GNN-Refined PR	Same as above	160 000+

Mesh region labelling (skin, left/right ear canal) is needed for boundary condition assignment in BEM.

3. Mesh2HRTF Simulation and Configuration

Mesh2HRTF numerically solves the Helmholtz equation over the (graded and labelled) mesh surfaces using the Burton–Miller combined-field integral equation, with FMM acceleration (Zolfaghari et al., 2014).

• Boundary Conditions: Primary: rigid (“sound-hard”) surface except for receiver regions (blocked canal patch where ∂ₙφ is prescribed). Optionally, impedance-matched or custom boundary types.
• Simulation Parameters: Frequency grid typically spans 0–24 kHz, with step sizes of 100–150 Hz. Edge lengths in the mesh are chosen to ensure at least six elements per minimum wavelength at f_max near the microphones, relaxed elsewhere by grading (Ziegelwanger et al., 2016).
• BEM Solver: Multilevel FMM (ml-fmm) is the standard choice; matrix assembly and GMRES iterative solution target error tolerances of 10⁻³ to 10⁻⁴, with preconditioning from near-field blocks. Runtime: 10–20 min for PR meshes, 30–45 min for full-res scans (single GPU/cluster node) (Pirard et al., 25 Mar 2026).
• Command-line Integration Example:

mesh2hrtf \ --mesh subjectP0004_PR.stl \ --output subjectP0004_PR.sofa \ --solver ml-fmm \ --freqMin 0 --freqMax 24000 --freqStep 150 \ --srcPos sonicom_source_positions.txt \ --recPosEarL earL_coords.txt --recPosEarR earR_coords.txt \ --computeNormals --normalizeLevel \ --windowTemplate "KEMAR_Knowl_EarSim_LargeEars_Windowed_NoITD_48kHz.sofa" \ --removeITD --itdPadding 0.8

• Post-processing: Alignment of synthesized HRTFs to measurement grids, temporal windowing for HRIRs, level normalization, ITD removal and metadata tracking (Pirard et al., 25 Mar 2026).

4. Evaluation Metrics and Perceptual Significance

Evaluation follows both numerical and perceptual conventions using established metrics:

• Numerical Metrics:
  • Interaural Time Difference (ITD) Error:

  $$\varepsilon_{\mathrm{ITD}}(d) = |\mathrm{ITD}_{\mathrm{cond}}(d) - \mathrm{ITD}_{\mathrm{meas}}(d)|$$ - Interaural Level Difference (ILD) Error:

  $$\varepsilon_{\mathrm{ILD}}(d) = |\mathrm{ILD}_{\mathrm{cond}}(d) - \mathrm{ILD}_{\mathrm{meas}}(d)|$$ - Log-Spectral Distortion (LSD):

  $$\mathrm{LSD}_e(d) = \sqrt{ \frac{1}{K} \sum_{k=1}^K \Big[ 20\log_{10}|H_{\mathrm{cond}}(e,d,f_k)| - 20\log_{10}|H_{\mathrm{meas}}(e,d,f_k)| \Big]^2 }$$

• Perceptual Metrics: Behavioural experiments (N = 27) using virtual loudspeaker arrays capture absolute, quadrant, and front-back errors, with SRM measured by speech-reception threshold differences (Pirard et al., 3 Oct 2025, Pirard et al., 25 Mar 2026).
• Auditory-Model Predictions: Evaluation with Baumgartner2014 and Barumerli2023 confirms PR-mesh–derived Mesh2HRTF HRTFs yield elevated quadrant and front-back error rates relative to measured or KEMAR HRTFs (Pirard et al., 25 Mar 2026).

Condition	ITD Error (μs)	ILD Error (dB)	LSD (dB)	Quadrant Error (%)	GCE (°)
Measured HRTF	—	—	—	15.4	23.4
PR-synthetic	24.5	2.8	10.2	40.6	39.3
Random HRTF	34.7	2.1	6.6	27.1	30.5

5. Hybrid ML-Enhanced Mesh Upsampling

To address PR-mesh limitations, GNN-based upsampling techniques are employed:

• Neural Subdivision: Following the methodology of Liu et al. (2020), the GNN predicts refined vertex positions for each subdivision step, operating on one- or two-ring mesh neighborhoods. MeshCNN or edge-based MLP modules are used for message passing (Pirard et al., 3 Oct 2025).
• Loss Function: The main geometric loss is the (bidirectional) Hausdorff distance between predicted and ground-truth scan vertex sets; regularization terms on smoothness and edge-length may also appear.
• Upsampling Pipeline:

1. Train GNN on pairs of PR and high-res meshes (using bijection from surface homeomorphism matching).
2. Apply GNN to unseen PR meshes; output is a high-fidelity upsampled mesh.
3. Feed into standard Mesh2HRTF workflow for HRTF synthesis.

This approach aims to recover fine pinna detail not captured by photogrammetry alone, leading to perceptually significant improvements in simulated HRTF cues, especially for elevation and spectral features.

6. Role of Mesh Grading and Computational Efficiency

A-priori mesh grading is essential for both computational tractability and preserving perceptually-relevant features:

• Mathematical Framework: Local target mesh edge lengths are prescribed by a function of distance to the microphone patch (Γ*), selecting from power-law or raised-cosine grading. Example:

$$\hat{\ell}_e = \hat{\ell}_{\min} + (\hat{\ell}_{\max} - \hat{\ell}_{\min})\, \mu(\bar{d}_e)$$

where $\mu(\bar{d}_e)$ implements the chosen grading profile (Ziegelwanger et al., 2016).

• Quantitative Impact: Graded meshes (e.g., COS2, ℓ_min = 1 mm, ℓ_max = 12 mm) reduce element count by 7–10× compared to uniform fine meshes, with <1 dB LSD or 1° polar/quad error increment for whole-head HRTFs.
• Best Practices: Always ensure dense meshing over the ipsilateral pinnae and canal region for accurate mid/high-frequency cues; aggressive coarsening elsewhere can be tolerated (Ziegelwanger et al., 2016).

7. Challenges, Limitations, and Open Directions

• Photogrammetry Limitations: PR meshes currently under-resolve high-frequency pinna features critical for accurate monaural and elevation cues; ILD and LSD metrics degrade substantially for PR-synthetic HRTFs (Pirard et al., 25 Mar 2026).
• ML-Upsampling Gaps: While GNN-upsampled meshes improve geometric fidelity, implementation details (activation types, optimizer settings) remain underreported and may affect reproducibility (Pirard et al., 3 Oct 2025).
• BEM Scope: Mesh2HRTF prescribes rigid and impedance matching boundary conditions, which neglect skin compliance or cavity resonances present in-vivo; this limits ultimate physiological realism (Zolfaghari et al., 2014).
• Future Directions: Prospective refinements include higher-resolution photogrammetry, data-driven mesh refinement, hybrid geometric-ML pipelines, and frequency-adaptive mesh grading. Incorporation of torso geometry and further automation in mesh preprocessing remain priorities for closing the fidelity gap between synthesized and measured HRTFs (Pirard et al., 25 Mar 2026).

• Pirard et al., "Enhancing Photogrammetry Reconstruction For HRTF Synthesis Via A Graph Neural Network" (Pirard et al., 3 Oct 2025)
• Pirard et al., "Photogrammetry-Reconstructed 3D Head Meshes for Accessible Individual Head-Related Transfer Functions" (Pirard et al., 25 Mar 2026)
• Ziegelwanger et al., "A-priori mesh grading for the numerical calculation of the head-related transfer functions" (Ziegelwanger et al., 2016)
• Ginzburg et al., "Large Deformation Diffeomorphic Metric Mapping And Fast-Multipole Boundary Element Method Provide New Insights For Binaural Acoustics" (Zolfaghari et al., 2014)