Sensitivity Analysis of Generative Spatial Audio Metrics: A Study on Responsiveness, Smoothness, and Symmetry

Abstract: Evaluating generative spatial audio for First-Order Ambisonics (FOA) remains challenging due to a limited understanding of how metrics respond to changes in spatial parameters such as azimuth and elevation. We propose a framework to analyze metric sensitivity along continuous spatial trajectories, drawing on principles of sensitivity analysis in parametric sound synthesis. Using controlled FOA scenes with increasing scene complexity, we define three desiderata for metric behavior: Responsiveness, Smoothness, and Symmetry. We assess standard distribution-based and sample-based metrics, including Fréchet Audio Distance (FAD), intensity vectors, and acoustic maps. Our findings show that FAD using localization-specific embeddings and acoustic maps yield high Responsiveness and robust Smoothness and Symmetry across conditions, while intensity vectors degrade with increasing scene complexity. This is the first step towards investigating the sensitivity of metrics for generative spatial audio.

• The paper demonstrates that localization-specific metrics, notably F-PSELD and MVDR-AM, excel in responsiveness and noise robustness.
• It introduces a framework based on three key criteria—responsiveness, smoothness, and symmetry—to assess metric performance across varied scene complexities.
• It highlights the limitations of intensity vector methods in multi-source scenarios, emphasizing the need for geometry-aware metric designs.

Sensitivity Analysis of Generative Spatial Audio Metrics: Responsiveness, Smoothness, and Symmetry

Introduction

The paper "Sensitivity Analysis of Generative Spatial Audio Metrics: A Study on Responsiveness, Smoothness, and Symmetry" (2606.11581) systematically examines the behavior of standard evaluation metrics for generative First-Order Ambisonics (FOA) spatial audio under parameterized control trajectories. The work addresses the lack of consensus around reliable evaluation methodologies for spatial generative models, specifically focusing on how different metrics respond to continuous changes in spatial parameters such as azimuth and elevation. Drawing analogies from sensitivity analysis in parametric sound synthesis, the authors propose three desiderata for evaluating metric quality—Responsiveness, Smoothness, and Symmetry—and empirically analyze the behavior and robustness of a diverse set of both distribution-based and sample-based metrics across varying scene complexities and noise conditions.

Methodology

The core framework revolves around the analysis of metric response curves under controlled variations of spatial parameters. For each synthesized FOA scene, the audio source is systematically moved through a circular trajectory in either azimuth or elevation, generating sequences of audio samples that encode specific parametric changes. Metrics are then evaluated based on three principal criteria:

• Responsiveness quantifies the magnitude and monotonicity of metric output in response to parameter variation, measured by the mean absolute slope of the response curve, penalized by $R^2$ goodness-of-fit.
• Smoothness evaluates the local continuity of the metric's response as parameter changes, penalizing abrupt jumps and favoring consistent, jitter-free variation.
• Symmetry assesses consistency of the response curve under forward versus reverse parameter sweeps, measuring the root mean square error between symmetrically placed points around the trajectory center.

The metrics studied can be categorized into distribution-based (e.g., FAD with different embeddings such as VGGish, StereoCRW, GRAM, PSELDNet) and sample-based (e.g., Intensity Vectors, MVDR Acoustic Maps, Interchannel Phase Differences, Log Spectral Distance, GCCPHAT). All metrics were normalized to enable shape-based comparison.

Figure 1: The idealized response curve for a well-behaved metric along a closed spatial trajectory—monotonic, smooth, and symmetric.

Scene complexity is systematically varied, with experimental conditions including single-source, multiple-source (counter-rotating), and multiple-instances-of-the-same-source layouts, each under both clean and additive noise (random SNR 0–15 dB) conditions. The synthesized dataset comprises over 68,000 FOA samples using SoundSpaces 3D RIRs and monophonic events from FSD50K, ensuring parameter diversity and repeatability.

Experimental Results

The comparative analysis covers all three metric desiderata under a range of experimental regimes:

• Responsiveness: Metrics leveraging localization-specific embeddings (F-PSELD, MVDR-AM, IV) display high response sensitivity to controlled spatial changes, reflected in pronounced, appropriately shaped response curves. Distribution-based metrics encoding explicit spatial properties outperform those employing monaural or stereo features.
• Smoothness–Responsiveness Trade-off: There is a clear operational trade-off; metrics such as F-PSELD and MVDR-AM achieve superior balance, simultaneously maintaining high Responsiveness and Smoothness. Metrics like LSD, GCCPHAT, and IPD, while extremely smooth, are unresponsive, failing to register spatial variations adequately. The upper right quadrant in the Responsiveness-vs-Smoothness plane identifies the most desirable metrics.

  Figure 2: Empirical results for Responsiveness, Smoothness, and Symmetry measures across all experimental conditions and metrics.

  

  Figure 3: Responsiveness vs. Smoothness trade-off; ideal metrics reside in the high-high quadrant, dominated by F-PSELD and MVDR-AM.

• Symmetry: Generally high symmetry scores are observed across all metrics, including the under-responsive ones, indicating that symmetry alone is not a differentiating criterion for metric quality.
• Noise Robustness: F-PSELD and MVDR-AM maintain stable sensitivity profiles under additive noise, evidenced by minimal percentage change in scores. Conversely, sample-based metrics like LSD and phase-sensitive measures collapse under noise, masking their already weak spatial responsiveness.

  Figure 4: Percentage change in evaluation metric scores due to additive noise; closer to zero denotes greater robustness.

• Scene Complexity Robustness: MVDR-AM and F-PSELD are invariant to single vs. multi-source arrangements, although IVs collapse (lose responsiveness and smoothness, inflate symmetry) when evaluating symmetric multi-source layouts. This breakdown is attributed to mirrored source cancellations and points to the limitation of raw intensity vector-based methods when used in complex generative FOA scenarios.

  Figure 5: Variation of Responsiveness, Smoothness, and Symmetry across increasing spatial scene complexity under clean conditions.

Theoretical and Practical Implications

This study delivers several implications for the development and evaluation of generative spatial audio models:

• Metric Selection: Metrics informed by explicit localization representations (especially those leveraging SELD-trained embeddings and hybrid features that combine IVs and spectral properties) offer superior parameter sensitivity and stability. The prevalent use of monaural and magnitude-based metrics (LSD, mono-FAD) in the literature is empirically unsupported for nuanced spatial modeling tasks.
• Meta-Evaluation Framework: The codification of Responsiveness, Smoothness, and Symmetry provides a practical framework for the meta-evaluation of FOA metrics, enabling rigorous benchmarking beyond black-box model testing.
• Limitations of IVs: While effective for single and multi-source sweeps, IVs are unreliable for mirrored multi-instance scenarios, highlighting the need for robust, geometry-aware representations in future evaluation metrics.

The findings underscore the importance of not only metric value but also the dynamic quality of metric response, with responsiveness and smoothness essential for capturing the fidelity of generative models under artist or user-driven parameter changes. The framework is extensible to further metrics, higher-order ambisonics, and real-world perceptual validation.

Future Directions

Relevant future directions include:

• Expansion to more diverse and higher-order spatial audio datasets, including real world and artist-annotated scenes.
• Integration with subjective human perceptual evaluations to validate and calibrate proposed sensitivity measures.
• Inclusion of advanced, self-supervised and transformer-based representations for generalized metric design.

Conclusion

This paper delivers a rigorous analysis of the sensitivity properties of evaluation metrics for generative spatial audio, establishing that localization-focused, hybrid-embedding-based metrics maintain superior sensitivity, smoothness, and robustness under realistic conditions of scene complexity and noise. These insights inform future best practices for both metric selection and generative architecture validation in spatial audio research. The proposed meta-evaluation framework represents an essential tool for reliable, interpretable benchmarking of FOA generative models.