Neural Acoustic Multipole Splatting

• Neural Acoustic Multipole Splatting is a framework that synthesizes room impulse responses using learnable multipoles with directional patterns defined via spherical harmonic decomposition.
• It employs dual neural branches to predict time-domain signals and frequency-dependent directivity, ensuring compliance with the Helmholtz equation.
• A pruning strategy eliminates low-energy multipoles during training, significantly enhancing computational efficiency and RIR synthesis fidelity.

Neural Acoustic Multipole Splatting (NAMS) is a data-driven framework for room impulse response (RIR) synthesis at arbitrary receiver positions, leveraging representations based on neural multipoles instead of conventional monopole point sources. Each multipole in the NAMS model is both spatially positioned and characterized by a learnable directionality pattern via spherical harmonic decomposition, enabling expressive modeling of sound fields that adhere to the physical constraints of the Helmholtz equation. NAMS integrates a neural network architecture for multipole emission and directivity prediction, and introduces a pruning strategy that progressively eliminates redundant multipoles, leading to efficient and accurate RIR synthesis suitable for spatial audio rendering in real or synthetic environments.

1. Multipole Modeling in Acoustic Wave Equations

The theoretical basis for acoustic multipole modeling originates from kinetic theory and fluid mechanics (Viggen, 2013). By introducing a source term $s(x,\xi,t)$ into the Boltzmann equation, multipole source terms (monopole, dipole, quadrupole) naturally arise in the wave equation:

$$\left(\frac{1}{c_0^2}\right)\frac{\partial^2 p}{\partial t^2} - \nabla^2 p = \frac{\partial S_0}{\partial t} - \frac{\partial S_i}{\partial x_i} + \frac{\mu}{p_0} \frac{\partial^2}{\partial x_i \partial x_j} [S_{ij} - 3\delta_{ij} (p_0/\rho_0) S_0]$$

where $S_0$ is the monopole (mass injection term), $S_i$ the dipole (force term), and $S_{ij}$ the quadrupole source from viscous corrections. These multipoles correspond to physically interpretable acoustic phenomena (isotropic emission, force-driven directivity, complex flow/turbulence effects) and provide a systematic, physically grounded hierarchy for constructing sound fields in computational models.

2. NAMS Framework: Multipole Splatting via Deep Learning

NAMS departs from dense monopole source modeling and instead places neural acoustic multipoles in a computational domain. Each multipole is assigned a learnable position $x_p$ and is defined by two neural branches:

• Signal Branch: For each multipole position, an MLP predicts a time-domain emission signal $s_p(t)$, receiving a sinusoidal positional encoding of $x_p$.
• Directivity Branch: For each receiver position $x_r$, the network processes the relative coordinate $(x_p - x_r)$ using encodings and an MLP, ultimately outputting spherical harmonic coefficients $B_{nm,p}(f)$, which determine the frequency-dependent directional pattern $D_p(f, x_r)$.

The RIR at a receiver, in the frequency domain, is then synthesized as:

$$H(f, x_r) = \sum_{p=1}^P S_p(f) \left[ \frac{e^{-j2\pi f r_p(x_r)/c}} {r_p(x_r)} \right] D_p(f, x_r)$$

with $S_p(f)$ the Fourier transform of $s_p(t)$, $r_p(x_r) = \|\mathbf{x}_r - \mathbf{x}_p\|$, and $c$ the speed of sound. The directivity function is represented as

$$D_p(f, x_r) = \sum_{n=0}^N \sum_{m=-n}^{n} B_{nm,p}(f) Y_n^m(\Omega_p(x_r))$$

where $Y_n^m$ are spherical harmonic basis functions, and $\Omega_p(x_r)$ is the angular coordinate of multipole $p$ as seen from the receiver.

This formulation ensures that each multipole is both an emission site and encodes orientation-dependent characteristics, addressing limitations of isotropic monopole-only schemes and aligning with physical wave propagation solutions.

3. Pruning Strategy for Efficient Multipole Utilization

NAMS introduces a pruning mechanism to mitigate overfitting and computational inefficiency from the initial dense splatting. Multipoles are first distributed liberally (e.g., on spheres around the sound source), and during training, the energy in each $s_p(t)$ signal is evaluated every 20 epochs after the first 100. Multipoles whose energy falls below 50% of the global median are removed. This reduces the number of active multipoles to approximately 20–22% of the original set, retaining only those crucial for accurate sound field representation.

The pruning strategy yields several practical benefits:

Training Step	Multipole Set Size	Pruning Criterion
Initial	Dense (e.g., 1089)	None
Pruning	Sparse (~20%)	$E_p < 0.5 \cdot \text{median}(E)$

where $E_p$ is the energy of multipole $p$, evaluated over its emission signal.

This iterative reduction leads to faster inference (2.1–2.2 ms), improved RIR fidelity, and establishes a physically meaningful configuration of contributing multipoles.

4. Experimental Evaluation and Ablation Studies

Comparative experiments against methods such as Neural Acoustic Fields (NAF) (Luo et al., 2022), AVR, and state-of-the-art hybrids demonstrate that NAMS reliably outperforms competitors across several acoustic evaluation metrics:

• Phase error, amplitude error, envelope error (percentage differences)
• Reverberation time (T60), clarity (C50), early decay time (EDT)

Ablation studies provide further insight:

• Multipole models (spherical harmonics with $N>0$) outperform monopole-only models (spherical harmonics order $N=0$).
• Pruning from dense splatting improves both computational speed and accuracy, indicating that physically motivated, compact multipole placement is preferable to uniform, parameter-heavy distributions.

5. Connections to Geometry-Aware Neural Sound Propagation

Related geometric deep learning methods model acoustic scattering fields by predicting spherical harmonic coefficients from point cloud object representations via permutation-invariant architectures (PointNet (Tang et al., 2020), Laplacian-based encoders (Meng et al., 2021)). These methods demonstrate efficacy in learning wave-based corrections for interactive sound propagation, but NAMS advances the paradigm by representing the whole sound field as a structured sum of directionally-tuned multipoles instead of isolated scattering coefficients.

The multipole architecture employed in NAMS aligns with theoretical approaches that decompose wave equations into contributions from fundamental source types, as derived in kinetic theory and continuum acoustics (Viggen, 2013). This suggests that the NAMS framework combines data-driven neural modeling with physically interpretable basis functions, allowing faithful reproduction and generalization across complex acoustic scenes.

6. Implications, Applications, and Future Directions

NAMS fills a critical need for rapid, accurate, and physically grounded spatial audio synthesis in domains including virtual and augmented reality, gaming, simulation, and architectural acoustics. By enabling efficient prediction of RIRs at unseen receiver locations, NAMS offers dynamic simulation of room acoustics, early reflections, and diffuse reverberation.

Plausible implications include the extension of NAMS to time-varying environments, integration with neural radiance field pipelines (as in NeRAF (Brunetto et al., 28 May 2024)), and further refinement of pruning/object selection via geometric cues or hybrid audio-visual learning. The combination of physical modeling and neural signal representation suggests future exploration in multi-modal simulation, data-efficient scene understanding, and real-time immersive audio rendering.

NAMS demonstrates that acoustic multipole splatting—under neural guidance—provides a compact, interpretable, and high-performance approach to sound field synthesis, bridging classical wave physics and modern deep learning for advanced room acoustics modeling (Baek et al., 22 Sep 2025).