Graph Neural Field with Spatial-Correlation Augmentation

• The paper introduces a unified graph-based pipeline that integrates HRTF personalization and upsampling modules to accurately reconstruct spatial audio cues.
• It employs multi-head graph attention networks and low-rank adaptation to encode and decode HRTF features, achieving notable LSD and ILD reductions.
• Its modular framework minimizes labor-intensive measurements while enforcing directional consistency, supporting scalable personalization for immersive VR/AR applications.

Graph Neural Field with Spatial-Correlation Augmentation (GraphNF-SCA) is a graph-based machine learning pipeline designed for high-fidelity personalization of Head-Related Transfer Functions (HRTFs) in spatial audio rendering. HRTFs encode how a listener's anatomy and sound source direction shape the arrival of audio at each ear, enabling immersive experiences on VR/AR devices. Traditionally, personalized HRTF acquisition is laborious, requiring dense subject-specific measurements in an anechoic chamber. GraphNF-SCA addresses this challenge by leveraging graph neural networks (GNNs) and explicit spatial-correlation modeling to generalize individual HRTFs for unseen subjects from minimal input, outperforming prior approaches in both sparsity and accuracy.

1. Theoretical Foundations and Motivation

HRTFs are inherently subject-dependent—varying by listener anatomy such as pinna shape, head size, and torso geometry—and position-dependent, varying by source direction in three-dimensional space. Previous HRTF-personalization (HRTF-P) models approximate personalized HRTFs from anthropometric features or a sparse set of measurements, but typically ignore the geometric structure and mutual dependencies across directions. Conversely, HRTF-upsampling (HRTF-U) methods interpolate well when many measurements are available, yet fail to generalize to subjects with limited or no measurements. GraphNF-SCA's key innovation is to unify these paradigms in a graph-centric architecture that utilizes both subject similarity and spatial correlation information efficiently.

2. Architecture and Workflow: Modules of GraphNF-SCA

GraphNF-SCA comprises three sequential components: HRTF-P (personalization), HRTF-U (upsampling), and a fine-tuning stage for spatial-correlation augmentation.

HRTF-P Module

• Neighbor Retrieval: For a new listener, the system selects reference subjects whose interaural level-difference (ILD) or time-difference (ITD) features closely match the target.

$$\mathcal{N}_{\hat s}=\{\,s\mid \|a_s - a_{\hat s}\|_2<\delta_s\}$$

Here, $a_s$ denotes the measured ILD/ITD feature vector for subject $s$.

• Graph Construction: Vertices represent stacked left/right ear HRTF magnitude vectors at direction $d$ for each neighbor, connected by unit-weight edges.
• Encoder: Stacked graph attention network (GAT) layers with multi-head attention encode node features, using learnable weights and ELU activation.

$X^{(l+1)} = \sigma(A X^{(l)} W^{(l)} + b^{(l)})$

Multi-head attention coefficients use a LeakyReLU followed by softmax.

• Clue Fusion: Concatenated vectors of direction and subject features ($c_s^d$) are encoded via an MLP and fused with the GAT output.
• Decoder: The universal representation $\tilde F^d$ passes through fully connected layers with low-rank adaptation (LoRA), and deconvolution layers reconstruct the personalized HRTF.

$\bar F^d \;\leftarrow\;\bar F^d + u\,v^\top$

• Loss: Log-spectral distortion (LSD) is the training objective:

$$\mathcal L_{\mathrm{P}} = \sqrt{\frac{1}{2K}\sum_{k=1}^{2K}\left[20 \log_{10}\frac{\hat H_{\hat s}^d(k)}{H_{\hat s}^d(k)}\right]^2}$$

HRTF-U Module

• Direction Graph Construction: Vertices include measured HRTFs at nearby directions plus a dummy vertex for prediction. Edges exist if the angular separation is within a scaled threshold, and weights are Gaussian:

$$W_{dp} = \exp\left(-\frac{\|d-p\|_2^2}{2\sigma^2}\right)$$

• Processing: The directional graph undergoes $I$ GAT layers and a final MLP, focusing prediction on the dummy vertex.
• Loss: Again, LSD is used, evaluating reconstruction fidelity.

Fine-Tuning (Spatial-Correlation Augmentation)

• Pretraining: HRTF-P is optimized with RAdam (lr=1e-3, 200 epochs); HRTF-U uses Adam (lr=2e-3, 200 epochs), both with patience-based learning-rate decay.
• Spatial Consistency Enforcement: For each new subject, HRTF-P predicts HRTFs over all directions; the output forms the vertex set of the spatial graph for each direction. Only the HRTF-U MLP is fine-tuned (20 epochs, Adam lr=2e-3, exponential decay 0.95/epoch), enforcing smoothness and multi-directional consistency in the final output.

3. Dataset Utilization and Evaluation Metrics

GraphNF-SCA has been evaluated on three prominent datasets:

• SONICOM: 200 subjects, 793 directions, 48kHz sampling; splits 160/19/20 for training/validation/testing.
• CIPIC: 45 subjects, 1250 directions, 44.1kHz; splits 40/4/5.
• HUTUBS: 96 subjects measured (440 directions), plus 1730 simulated directions, splits 77/9/10.

Metrics employed include:

• Log-Spectral Distortion (LSD): Quantifies magnitude-spectrum reconstruction error (see above).
• Interaural Level Difference error (ILD):

$$\text{ILD error} = |20 \log_{10}\frac{\hat H_L}{\hat H_R} - 20 \log_{10}\frac{H_L}{H_R}|$$

4. Quantitative Performance and Comparative Analysis

GraphNF-SCA attains state-of-the-art results under both sparse and dense measurement scenarios:

Dataset	Sparse Setting (3 directions)	LSD (dB)	ILD (dB)	Dense Setting (100 directions)	LSD (dB)	ILD (dB)
SONICOM	GraphNF-SCA	3.60	0.96	GraphNF-SCA	2.72	0.70
SONICOM	GraphNF	4.33	1.22	GraphNF	--	--
CIPIC	GraphNF-SCA	~3.8	~1.1	GraphNF-SCA	--	--
CIPIC	RANF	~4.6	~1.5	RANF	--	--

• GraphNF-SCA yields a 16.9% reduction in LSD (from 4.33 dB to 3.60 dB) and a 21.3% reduction in ILD (from 1.22 dB to 0.96 dB) versus GraphNF-alone under the 3-measurement regime (SONICOM).
• Under dense measurement conditions (100 directions), GraphNF-SCA produces LSD = 2.72 dB and ILD = 0.70 dB, outperforming all tested baselines including nearest neighbor, HRTF selection, NF(CbC), NF(LoRA), and RANF.
• On CIPIC and HUTUBS, GraphNF-SCA preserves a 10–15% advantage even as measurement density increases, suggesting robust generalization.

5. Qualitative Improvements and Error Distribution

GraphNF-SCA achieves notable advances in error distribution and spectral fidelity:

• Direction-wise Error Reduction: Visualizations (Figure 5) show a reduction in LSD errors, particularly for contralateral directions (azimuth 180°–360°), mitigating head shadowing artifacts prevalent in other methods.
• Spectral Tracking: Overlay plots (Figure 7) indicate that GraphNF-SCA’s predicted HRTF magnitudes more faithfully match ground truth peaks and notches, especially at high frequencies, compared with RANF.

This suggests that multi-directional graph-based smoothing effectively controls angular artifacts, resulting in perceptually coherent HRTFs even in challenging contralateral regions.

6. Significance, Limitations, and Potential Extensions

GraphNF-SCA integrates anatomy-conditional encoding, geometry-aware upsampling, and lightweight fine-tuning to deliver highly individualized HRTFs from sparse subject input. Individualization for unseen subjects is feasible with minimal measurement (as few as three directions), which implies dramatic reductions in data collection overhead for practical applications in VR/AR audio rendering.

A plausible implication is that explicit spatial-correlation modeling, as embodied in GraphNF-SCA, addresses both angular smoothness and head-shadowing effects superiorly compared to previous position-by-position schemes. However, the method’s reliance on pretrained graph modules and fine-tuning requires careful dataset management and parameter selection to maintain stability and generalizability.

For future research, pathways include adapting the spatial-correlation augmentation mechanism to additional forms of channel or cross-modal data, and exploring its use in real-time adaptive audio personalization for mobile or embedded deployments.

7. Connections to Related Methodologies

GraphNF-SCA draws on graph attention networks (GATs), low-rank adaptation (LoRA), and spectral-domain loss optimization—an intersection exemplifying trends in graph-based processing for high-dimensional signal personalization. The modular approach of combining independent personalization and spatial upsampling modules followed by fine-tuning suggests applicability in other parameterized spatial signal problems requiring both individualization and geometric consistency, such as room impulse response modeling, spatial filtering, and sensor array interpolation.

The interplay between subject-based and geometric-based graph construction in GraphNF-SCA represents a distinct progression over classical regression or manifold-based interpolation, placing it at the forefront of modern spatial audio modeling methodologies.