Spatial LibriSpeech Audio Corpus

• Spatial LibriSpeech is a large synthetic audio corpus featuring over 650 hours of multichannel speech paired with dense spatial and room-acoustic annotations.
• It is engineered via a simulation pipeline that combines realistic room impulse responses, noise augmentation, and precise geometric labeling for robust 3D source localization and acoustic parameter estimation.
• The dataset underpins benchmark tasks for spatial audio and multimodal LLM research, demonstrating significant performance improvements in tasks like 3D localization and reverberation time estimation.

Spatial LibriSpeech is a large-scale, synthetically spatialized audio corpus derived from LibriSpeech, specifically constructed for machine learning research in spatial audio, 3D source localization, room-acoustic inference, and multimodal spatial-language modeling. It comprises over 650 hours of multichannel (19-channel and FOA) speech, paired with densely annotated spatial and room-acoustic metadata. Spatial LibriSpeech provides a quantitative foundation for benchmarking spatial audio models, including modern auditory LLMs and classical spatial audio networks, and has established itself as a reference dataset for spatially informed audio modeling (Sarabia et al., 2023, Tang et al., 2024).

1. Dataset Construction and Properties

Spatial LibriSpeech is synthesized by augmenting clean LibriSpeech utterances with simulated three-dimensional reverberant acoustics and spatial annotations. The core pipeline consists of:

• Source Corpus and Segmentation: Uses the train-clean-100, train-clean-360, and test-clean splits of LibriSpeech, containing one speaker per utterance.
  • Training set: 573 h 13 m 12 s, 171,951 utterances
  • Test set: 85 h 29 m 20 s, 49,505 utterances
• Spatialization:
  • Room Generation: 8,952 unique shoebox rooms (train), 4,970 (test), sampled with floor area in [13, 277] m², surface properties from a database of real materials, and $T_{100}$ (target reverberation time) spanning 145–2,846 ms.
  • Room Impulse Response (RIR) Simulation: Uses a full geometrical acoustics solver that models direct, reflected, and late reverberant paths, microphone directivity (including diffraction for the 19-channel spherical array and FOA), and speech-source directivity via boundary-element head models.
  • Microphone Configuration: 19-channel spherical array (Zylia); collapsed to 4-channel full-bandwidth first-order Ambisonics (FOA), with $16$ kHz sampling.
• Noise Augmentation: Each utterance can be mixed with randomly selected distractor noise (from DNS Challenge 2021), spatialized with an independent RIR and mixed at SNRs uniformly in $[10, 40]$ dB.
• Annotation:
  • Source Position and Orientation: Spherical coordinates $(r, \theta, \phi)$, $r\in[0.5, 4.0]$ m (train), $[0.9, 4.0]$ m (test); $\theta$ (azimuth, $[-180^\circ, +180^\circ]$), $\phi$ (elevation, $[-\frac{\pi}{2}, \frac{\pi}{2}]$).
  • Room and Acoustic Metrics: Room geometry, surface types, direct-to-reverberant ratio (DRR), reverberation times $16$0, $16$1, $16$2 (with $16$3, $16$4: decay rate), and clarity $16$5, all in broadband and 1/3-octave bands.
• Processing Workflow: Each speech sample is silence-trimmed, convolved with its unique multi-channel RIR, scaled to random active speech level (85–100 dB SPL), with optional noise injection as above.

2. Baseline Tasks and Model Architectures

Spatial LibriSpeech defines a suite of canonical spatial audio tasks, serving as benchmarks for diverse model classes:

• 3D Source Localization: Regression for azimuth and elevation, evaluated by the great-circle angular distance formula:

$16$6

• Distance Estimation: Regression for $16$7.
• Reverberation Time ($16$8) Estimation: Regression over 20 1/3-octave bands.
• Direct-to-Reverberant Ratio (DRR) Estimation: Regression over 20 1/3-octave bands.

Baseline Model:

• Input: 4-channel FOA, decomposed into active and reactive sound-field branches.
• Network: Two parallel 3D-convolutional towers (each 4 layers: [2,4,8,16] channels), per-branch processing, outputs concatenated and passed to a 3-layer MLP (50% dropout).
• Optimization: Adam optimizer ($16$9), 20 epochs (localization, distance), 50 epochs (DRR, $[10, 40]$0), total updates $[10, 40]$180M and 21M respectively.
• Loss: Regression, evaluated by median absolute error (MAE), interquartile range (IQR), and Pearson correlation.

Task	MAE (Test Set)	IQR
3D Localization	$[10, 40]$2	$[10, 40]$3
Distance Estimation	$[10, 40]$4 m	$[10, 40]$5 m
$[10, 40]$6 Estimate	$[10, 40]$7 ms	$[10, 40]$8 ms
DRR Estimate	$[10, 40]$9 dB	$(r, \theta, \phi)$0 dB

(Sarabia et al., 2023)

3. Use in LLM Spatial Audio Research

Spatial LibriSpeech has been adopted as the standard evaluation corpus for new forms of LLMs capable of processing spatial audio. Representative systems include:

• "Can LLMs Understand Spatial Audio?" leverages FOA spatialization (W channel to Whisper, X/Y/Z to framewise short-time intensity vectors) and fuses spatial cues using a Q-Former before LLM adaptation (LoRA fine-tuned Vicuna-7B).
• Tasks: Sound Source Localization (SSL), Far-field Speech Recognition (FSR), and Localization-Informed Speech Extraction (LSE), with spatial queries posed as text prompts.
• Metrics: SSL reports mean angular error, FSR uses WER. For SSL,

$(r, \theta, \phi)$1

• Results: On the original Spatial LibriSpeech test set, LLM+IV "Before" fusion achieves $(r, \theta, \phi)$2 MAE for SSL (vs. $(r, \theta, \phi)$3 baseline), and FSR WER reduces from 9.0% (no spatial cue) to 7.6%. This establishes a new synthetic-data state-of-the-art for LLM spatial understanding (Tang et al., 2024).

4. Dataset Comparisons and Sector Impact

Spatial LibriSpeech occupies a distinct position among open spatial audio corpora:

• Compared to Meeting-Centric Datasets: Datasets like LibriWASN or the multi-channel LibriSpeech playback set (Sharma et al., 2022, Schmalenstroeer et al., 2023) are based on loudspeaker playback in real rooms with physical microphone arrays (often asynchronous) and serve benchmarks for meeting transcription/separation; these typically emphasize synchronization, diarization, and overlapping speech.
• Synthetic, Well-Controlled Acoustics: Spatial LibriSpeech's synthetic pipeline allows dense, uniform sampling over positions, directions, room types, SNRs, and reverberation times. This facilitates systematic generalization studies, domain adaptation, and ablation of spatial cues.
• Integration into Multimodal and Wearable Paradigms: The OmniTalk dataset—termed "Spatial LibriSpeech" in (Mishra et al., 11 Apr 2025)—further employs LibriSpeech-based spatialization via dense sampled monaural impulse responses (Owlet microstructure) for wearable applications, supporting single- and multi-speaker scenarios with per-utterance $(r, \theta, \phi)$4 DoA labeling.

5. Generalization, Applications, and Limitations

Generalization:

• Baseline models trained exclusively on Spatial LibriSpeech exhibit strong transferability to external evaluation corpora, with 3D localization median error of $(r, \theta, \phi)$5 on TUT Sound Events 2018 (real data), and $(r, \theta, \phi)$6 estimation error of $(r, \theta, \phi)$7 ms on the ACE Challenge after fine-tuning (Sarabia et al., 2023).

Applications:

• Training and benchmarking for:
  • 3D source localization and distance estimation for AR/VR, robotic audition
  • Room-acoustic parameter estimation and dereverberation algorithms
  • Self-supervised representation learning in spatial audio
  • Multimodal LLMs integrating spatial and linguistic understanding

Limitations:

• Entirely synthetic simulation may result in a domain transfer gap for real-room effects (diffraction, microphone characteristics). This is partially mitigated by model fine-tuning.
• The dataset is restricted to single-source (non-overlapping) utterances; overlapping/multi-talker scenarios are addressed in separate efforts (e.g., DSS LibriSpeech, OmniTalk).
• Fixed microphone array geometry (spherical/FOA) and SNR/sample realism are determined by simulation parameters—real world hardware and noise non-idealities may expose additional modeling challenges.

6. Future Directions and Extensions

Potential extensions and open research questions for Spatial LibriSpeech include:

• Incorporation of denoising, self-supervised and multi-task learning objectives;
• Expansion to multi-source overlap and dynamic source-movement scenarios;
• Inclusion of real recorded FOA/array room data to expose models to non-ideal spatial propagation and sensor effects;
• Contrastive spatial-language embedding tasks and more granular room-identification/classification challenges;
• Increasing utility for real-time AR/VR and edge applications, especially when linked with LLM-based contextual processing (Mishra et al., 11 Apr 2025).

Spatial LibriSpeech's combination of scale, acoustic diversity, and detailed annotation underlies its adoption as a benchmark for algorithmic progress in spatially informed acoustic and multi-modal modeling (Sarabia et al., 2023, Tang et al., 2024).