Masked Wavelet Scattering Transform Neural Field for Sound Field Reconstruction

Abstract: In this paper, we propose a reconstruction framework that leverages the Wavelet Scattering Transform (WST) as a multi-scale feature extractor to impose statistical priors under sparse observation conditions. The reconstruction problem is formulated as an optimization task and solved using a neural field, with the WST incorporated into the training loss function. As a proof of concept, we validate the proposed method on HRTF upsampling. A masking strategy is applied to the WST coefficients, resulting in a two-phase procedure. The first phase learns a binary mask from a small multi-subject dataset, while the second phase applies the learned mask to the WST coefficients of an individual HRTF to preserve informative statistical structures during reconstruction. Validation against baseline methods, which also serve as an ablation study of the different components of the framework, demonstrates the effectiveness of the proposed approach.

• The paper introduces a new framework that integrates the wavelet scattering transform as a statistical prior to enhance sound field reconstruction.
• It employs a two-phase approach with adaptive masking and an MLP-based neural field, significantly outperforming conventional methods.
• Empirical results on the HUTUBS HRTF dataset show superior metrics (LSD, NMSE, NCC) and improved high-frequency detail recovery.

Masked Wavelet Scattering Transform Neural Field for Sound Field Reconstruction

Motivation and Context

Sound field reconstruction from sparse measurements is a longstanding inverse problem fundamental to domains such as spatial audio, room acoustics, and auditory personalization. Recent interest centers on Head-Related Transfer Function (HRTF) upsampling, in which individual anatomical idiosyncrasies and sparse sampling yield highly challenging data-impoverished settings. Conventional deep learning strategies in this space are frequently hampered by overfitting due to limited training data, a constraint intensified in individualized HRTF contexts. The "Masked Wavelet Scattering Transform Neural Field" (MSNF) framework (2606.04370) addresses this challenge directly by imposing interpretable, multi-scale statistical priors via the Wavelet Scattering Transform (WST), integrated into a neural field optimization scheme to enhance generalization and resolution under sparse acquisition regimes.

Wavelet Scattering Transform as Statistical Prior

The WST generates multi-scale, translation-invariant feature representations using cascaded Morlet wavelet convolutions and nonlinear modulus operations. Order-$i$ scattering coefficients $S^i p[u]$ characterize structural statistics from zeroth (coarse averaging) through higher orders (reintroducing high-frequency detail), truncated at second order—which is sufficient due to negligible energy beyond this level. The WST thus offers a fixed, non-learned feature set with guaranteed interpretability and established mathematical properties, outperforming learned CNN backbones especially in low-data settings ([oyallon2018scattering]). Historically, WST-enabled synthesis and generation frameworks have advanced fields ranging from astrophysics to audio texture synthesis ([cheng2024scattering], [mousset2024generative]), leveraging the transform's capacity to model non-Gaussian, heterogeneous random fields beyond classical second-order Gaussian statistics.

MSNF Framework and Masking Strategy

MSNF is designed as a two-phase procedure:

Phase 1 (Mask Identification): A binary mask $m\in\mathbb{R}^{N_{co}}$ is adaptively learned over WST coefficients from a small multi-subject dataset. The mask highlights coefficients with statistically consistent behavior across subjects, suppressing coefficients dominated by idiosyncratic or noisy features. This joint optimization step utilizes a loss combining mean-squared error in the spatial domain and masked scattering coefficient regularization, encouraging agreement among selected statistical features. Hadamard product masking is applied directly to the WST output.

Figure 1: Schematic of MSNF, delineating Phase 1 mask identification and Phase 2 neural field reconstruction with trainable components highlighted.

Phase 2 (Neural Field Reconstruction): For a target HRTF, an MLP-based neural field backbone receives spatial coordinates embedded via Random Fourier Features for spectral bias compensation ([tancik2020fourier]). Reconstruction proceeds with a composite loss featuring (a) data fidelity on observed measurements and (b) masked WST regularization with respect to a reference individual's WST coefficients. GradNorm is deployed to adaptively balance loss weighting terms ($\beta_1$, $\beta_2$), and a two-stage training protocol first optimizes solely for data fidelity before introducing WST-based prior regularization. Only network parameters $\gamma$ are learned in this phase, with masking coefficients $m$ held fixed from Phase 1.

Empirical Validation

The framework is validated on the HUTUBS HRTF simulation dataset ([brinkmann2019cross]), leveraging its high spatial and frequency resolution. The test scenario evaluates dense spatial reconstruction ($1730$ points) from a minimalist grid ($7\times7$), using three metrics: Log-Spectral Distortion (LSD), Normalized Mean Square Error (NMSE), and Normalized Cross-Correlation (NCC). Baselines include Spherical Harmonic (SH) interpolation, vanilla neural field (NF), and Scattering Neural Field (SNF) (omitting masking).

MSNF robustly outperforms all baselines across metrics (LSD, NMSE, NCC), exhibiting pronounced numerical superiority: mean LSD of 5.34 versus SH at 6.64 and NF at 6.10; NMSE of 0.14 versus SH at 0.23 and NF at 0.20; NCC of 87.79% versus SH at 69.66% and NF at 84.74%.

Qualitative comparison in high-frequency reconstruction ($14,470$ Hz) demonstrates MSNF's ability to recover fine spatial detail and lobe structure, whereas SH interpolation produces artifacts and NF may miss relevant structure. SNF sans masking, in contrast, risks mode collapse by overfitting to reference subject statistics, underscoring the necessity of coefficient masking for generalization.

Figure 2: Reconstruction results for HRTF at $S^i p[u]$0 Hz, right ear. MSNF closely approximates ground truth, outperforms SH and NF, and avoids degenerate overfitting seen in SNF.

Implications and Future Directions

MSNF's adoption of WST as a statistical prior establishes a viable regime for signal reconstruction amid extreme data scarcity, providing both interpretability and controllability over retained features. By adaptively masking WST coefficients, the framework avoids overfitting, maintains structure across scales, and enables flexible adaptation to heterogeneous or nonhomogeneous fields without manual mask specification. The modularity of WST regularization and masking strategy is broadly applicable—not only for HRTF upsampling but for more general sound field, holography, and other physics-informed problems where individualized detail and limited observations prevail ([verburg2025differentiable], [koyama2025physics], [olivieri2024physics]).

MSNF opens the door to future advances in:

• Cross-modal reconstruction and synthesis tasks in signal processing, astrophysics, and medical imaging, exploiting the statistical flexibility of WST;
• Incorporation of complex-valued data (e.g., phase-sensitive HRTFs);
• Joint space-frequency domain learning with richer scattering architectures (e.g., joint time-frequency scattering [anden2019joint]);
• Integration with graph-based or spherical neural fields, further improving spatial modeling;
• Hierarchical or multi-subject transfer protocols for maximal personalization.

Conclusion

The Masked Wavelet Scattering Transform Neural Field establishes an effective and interpretable statistical prior regime for sound field reconstruction under sparse observation constraints. Empirical results demonstrate compelling performance improvements over conventional and deep learning baselines. The framework's scalable adaptability and statistical controllability highlight its potential to advance signal reconstruction across diverse individualized, data-scarce domains.