Quantifying the Uncertainty of Blindly Estimated Room Embeddings Using a Dispersion-Calibrated Score

Abstract: Room embeddings derived from reverberant speech are often unreliable: speech content and recording degradation can alter the representation even when speaker, room, and source-receiver geometry remain unchanged, degrading downstream task performance. We propose a framework that learns room embeddings robust to speech-content variation and a representation-level uncertainty score from reverberant speech without downstream-task supervision. The embedding is anchored to a structured room impulse response (RIR) latent space and trained using a multi-view data structure with Kullback-Leibler (KL)-based alignment; a multi-positive contrastive term further refines robustness. A lightweight uncertainty head is calibrated using the dispersion of corruption-induced embeddings and optimized with a rank-based objective. Across waveform- and spectrogram-level corruptions, the score is consistent with representation dispersion and enables effective selective prediction while requiring only a single utterance at inference.

• The paper proposes a novel framework that quantifies uncertainty in room embeddings using a dispersion-calibrated score.
• It employs a three-stage pipeline combining RIR-VAE pretraining, contrastive embedding alignment, and uncertainty ranking based on cosine dispersion metrics.
• Experimental results show improved room verification accuracy and reliable selective prediction under diverse corruption protocols.

Quantifying the Uncertainty of Blindly Estimated Room Embeddings Using a Dispersion-Calibrated Score

Introduction

This work addresses the problem of quantifying and leveraging uncertainty in representation learning for blind room embedding from reverberant speech. Unlike traditional approaches that seek task-specific estimates (e.g., $T_{60}$, $C_{50}$), the proposed framework aims to robustly learn task-agnostic room embeddings that are disentangled from speech content, noise, and other nuisance factors. A central contribution is the introduction of a scalar uncertainty score $U$—calibrated against the dispersion of corruption-induced representation shift—that enables reliable identification of untrustworthy embeddings without recourse to downstream annotations or paired utterances during inference.

Methodology

The system employs a three-stage training pipeline, where each stage is dedicated to a distinct modeling objective (Figure 1).

Figure 1: Overview of the three-stage training pipeline. Stage-1 trains an RIR-VAE to obtain posterior statistics $$(\boldsymbol{\mu}_{z_H}, \boldsymbol{\sigma}^2_{z_H})$$. Stage-2 learns a speech embedding $\mathbf{z}_Y$ using multi-positive contrastive learning and KL alignment to the frozen RIR posterior. Stage-3 freezes the speech encoder and trains a scalar uncertainty head supervised by embedding dispersion $\delta$.

Stage-1 (RIR-VAE Pretraining):

A variational autoencoder is trained on RIR log-mel spectrograms, yielding a structured latent space. This space provides a geometry for alignment, with $\boldsymbol{\mu}_{z_H}$ and $\boldsymbol{\sigma}^2_{z_H}$ forming the basis for subsequent speech embedding anchoring.

Stage-2 (Contrastive Embedding with RIR Alignment):

A hybrid CNN-Transformer speech encoder is optimized using:

• KL alignment: The encoder pushes utterance-level embeddings $\mathbf{z}_Y$ to be geometrically consistent with the RIR-VAE posterior (fixing covariance to the identity matrix).
• Multi-positive contrastive training: By leveraging multiple utterances per RIR, the model ensures robust invariance to speech content, addressing the core confounding factor in blind room embedding. The combined loss is a convex combination of both terms, controlled by a scheduling parameter.

Stage-3 (Dispersion-Calibrated Uncertainty Estimation):

The (frozen) encoder is used to accumulate corruption-induced dispersion statistics from the embedding space. A lightweight MLP-based uncertainty head is then trained, with supervision derived from pairwise ranking on dispersion magnitudes—ensuring monotonicity but not regressed scale. Dispersion is computed as the cosine distance between clean and corrupted utterance embeddings, and the uncertainty head is optimized using a margin-based rank loss.

Experimental Setup

Data Construction:

The system is evaluated on a large, multi-source collection of measured RIRs, partitioned to avoid RIR identity overlap across splits. Reverberant speech is synthesized using EARS anechoic utterances, resulting in nearly 100 hours of data.

Corruption Protocols:

Waveform-domain (pink noise at various SNRs) and spectrogram-domain (frequency/time masking) corruptions are systematically applied. This controlled protocol underpins both the calculation of embedding dispersion targets and the testing of uncertainty generalization.

Metrics:

• Room Embedding Quality: Average Precision (AP) for RIR verification, log-mel spectrogram MAE for RIR reconstruction, and MAPE/MAE for blind $T_{60}$ and $C_{50}$0 estimation.
• Uncertainty Calibration: Spearman rank correlation between the predicted $C_{50}$1 and the dispersion $C_{50}$2.
• Selective Prediction: Downstream performance (verification/reconstruction error) when retaining only the lowest-uncertainty samples.

Results and Analysis

Representation Quality:

The multi-view (MV) batch construction significantly elevates verification AP (0.98, MRL-MV vs. 0.95, MRL-SV). Adding the multi-positive contrastive term achieves AP=0.99, setting a new accuracy level for task-agnostic blind room embedding. Notably, performance on RIR log-mel reconstruction and parameter estimation remains nearly unchanged with the contrastive addition, indicating that content invariance is most critical for high-fidelity verification.

Robustness to Speech Content and Corruption:

The introduction of MV data and contrastive learning establishes invariance across speech content. Consistency of the uncertainty $C_{50}$3 with sample-level embedding dispersion is evidenced by global Spearman’s $C_{50}$4 (see Table 1), outperforming previous approaches (MRL-MV) and baseline heuristics (corruption severity controls).

Method	Global $C_{50}$5	Noise $C_{50}$6	F.Mask $C_{50}$7	T.Mask $C_{50}$8
Severity	–	0.28	0.16	0.17
MRL-MV	0.85	0.59	0.66	0.68
Proposed ($C_{50}$9)	0.90	0.83	0.79	0.86

Selective Prediction:

When samples are ranked and selected by $U$0, the best possible downstream performance is achieved even without access to ground-truth corruption metadata. $U$1 is strictly a single-utterance statistic, facilitating practical deployment. In all corruption scenarios, ranking by $U$2 outperforms ranking by control parameters, showing greater monotonicity and steeper performance recovery as noisy/masked utterances are pruned (Figure 2).

Figure 2: Selective prediction. Samples are sorted by uncertainty $U$3 (solid) or by corruption severity controls (dashed).

Implications and Future Directions

The framework advances the state of the art in content-robust, blind room embedding by:

• Demonstrating that multi-view and contrastive paradigms should be standard for disentangling room from speech artifacts in reverberant speech.
• Providing a reliable, task-agnostic uncertainty metric; critical for downstream applications performing inference or domain adaptation in-the-wild.
• Offering single-utterance uncertainty scoring, thus reducing labeling burdens and enabling open-world deployment in variable signal conditions.

Key limitations include dependence on specific corruption models (real world degradations may differ), reliance on paired clean/corrupted samples for uncertainty calibration, and remaining in the single-channel domain. Rigorous evaluation with unseen real-world noises, interfering speakers, and device mismatches is essential for further validation. Extension to room-disjoint and cross-dataset scenarios, as well as adaptation to multi-microphone or spatialized representations, represents a logical next step.

Conclusion

This study demonstrates that robust, task-agnostic room embeddings can be achieved using a pipeline of structured RIR latent representation, multi-view contrastive alignment, and rank-based uncertainty calibration. The dispersion-calibrated uncertainty score enables effective detection and filtering of unreliable embeddings under a wide range of signal corruptions. The findings identify critical ingredients for scalable, high-reliability room acoustics representation and uncertainty modeling, motivating further development toward generic, out-of-distribution robust acoustic scene understanding.