---
title: 'LibriQuote: Expressive Speech Dataset'
url: https://www.emergentmind.com/topics/libriquote-dataset
type: topic
---

# LibriQuote: Expressive Speech Dataset

LibriQuote is a large-scale, English speech dataset specifically structured for advancing expressive, zero-shot text-to-speech (TTS) synthesis. Derived from LibriVox audiobook recordings, LibriQuote isolates character quotations—a source of expressive, emotional prosody—from non-expressive narration, and further enriches these samples with contextual and linguistic annotations. The resource is designed to both fine-tune and benchmark TTS systems on expressive tasks, emphasizing the modeling of emotion, accent, and naturalness in synthesized speech [2509.04072].

## 1. Dataset Organization and Composition

LibriQuote is partitioned into a training set containing 12,723 hours of non-expressive narration and 5,359 hours of expressive speech, totaling over 18,000 hours. Non-expressive narration consists primarily of neutral reading, while expressive utterances are sourced from fictional character dialogue within the audiobooks. Each quotation is meticulously aligned with its original textual and audio context, distinguishing LibriQuote from standard audiobook datasets.

| Subset          | Speech Type           | Hours      |
|-----------------|----------------------|------------|
| Neutral         | Non-expressive        | 12,723     |
| Expressive      | Character quotations  | 5,359      |

This division enables explicit modeling of expressive and non-expressive speech phenomena and allows quantitative analysis of their impact on TTS output quality.

## 2. Expressive Speech Annotation and Contextualization

Expressive utterances in LibriQuote are extensively annotated. Notably, each expressive quotation is accompanied by a contextual window—typically ∼100 words drawn from the original text surrounding the quotation, bounded by paragraph limits. This provides both prosodic and semantic context for downstream modeling.

Pseudo-labels are generated for each expressive utterance using large language models (e.g., Phi-4 and a filtered variant Phi-4₍conf₎). These labels capture speech verbs and adverbs from narrative cues, encoding prosodic intent (for example, “he whispered softly” yields “whispered”—a speech verb—and “softly”—an adverb). Such annotations function as an interpretable expressivity signal, guiding both supervised and self-supervised modeling of expressive prosody.

## 3. Test Set Design and Benchmarking Protocol

The LibriQuote test set includes 7.5 hours of expressive quotations from 15 speakers, chosen to be strictly unseen with respect to standard datasets, such as LibriSpeech. The primary evaluation protocol is a “cross-sentence” challenge: a neutral reference utterance (from the same speaker, but outside the expressive set) is used as input, while the system is tasked with synthesizing a target quotation in an expressive style, preserving the reference timbre.

Qualitative t-SNE analysis of latent emotion and accent representations in the test set demonstrates coverage of diverse emotional states and a broad spectrum of accented speech. This ensures the dataset functions as a robust benchmark for expressive speech synthesis, expanding the evaluation beyond intelligibility into fine-grained expressivity and timbral fidelity.

## 4. Objective and Subjective Evaluation Metrics

LibriQuote supports comprehensive evaluation of TTS models using both objective and subjective metrics. Objective metrics include:

- **Word Error Rate (WER):** Measures speech intelligibility.
- **Speaker Similarity (SIM-O):** Cosine-based metric for timbral preservation.
- **Mel Cepstral Distortion (MCD):** Quantifies spectral accuracy.
- **F0 Pearson Correlation (FPC):** Captures prosodic alignment.
- **Accent and Emotion Similarity:** Assessed via latent embedding comparisons.

Subjective evaluations comprise Mean Opinion Score (MOS) for naturalness and Comparative MOS (CMOS) tests for expressivity. Fine-tuning baseline TTS on LibriQuote results in improved WER, higher FPC, and lower MCD, indicating more intelligible and prosodically accurate synthesis. However, subjective ratings reveal model outputs still lag behind ground-truth quotations in perceived expressivity and naturalness.

## 5. Data Efficiency via Expressive Subset Filtering

Experiments with a filtered expressive subset (denoted Q_f), containing only quotations with non-empty adverb pseudo-labels or speech verbs from a curated list, demonstrate comparable improvements in intelligibility and expressivity using only ~10% of the expressive fine-tuning data. This suggests that targeted, label-rich expressive samples can yield high data efficiency for model development—a notable result for scaling expressive TTS in resource-constrained scenarios.

## 6. Alignment Methodology and Technical Resources

Text-to-audio alignment is critical for leveraging LibriQuote in TTS research. The alignment pipeline first matches chains of word-index pairs between automatic speech recognition (ASR) transcriptions and the original book text, followed by concatenated Levenshtein alignment for increased granularity. These alignment tokens enable precise mapping from text to expressive utterance, facilitating time-indexed prosody modeling.

Researchers are provided with train/dev/test splits and full evaluation code for reproducibility. Audio samples, annotations, and further technical details are available at https://libriquote.github.io/ [2509.04072]. The dataset’s design, particularly its separation of expressive and non-expressive speech, detailed contextual metadata, and word-level timing, supports advanced modeling for expressive, zero-shot, and cross-speaker TTS.

## 7. Implications and Applications

LibriQuote enables both fine-tuning and benchmarking of TTS systems on expressive and emotional speech synthesis tasks. Its scale and annotation depth surpass prior expressive corpora, which have generally been limited to small benchmarking sets. Applications include:

- Improving expressive dialogue in audiobook and virtual agent speech synthesis.
- Benchmarking zero-shot TTS systems on tasks requiring both expressivity and timbral continuity.
- Research in emotion, accent, and style-transfer modeling for speech synthesis.

A plausible implication is that explicit context and prosodic labels, combined with large-scale expressive data, may facilitate the development of models capable of synthesizing speech closer to human expressivity—a gap persisting in current state-of-the-art systems, as revealed by comparative MOS and intelligibility assessments in the LibriQuote benchmark.

In summary, LibriQuote constitutes a critical resource for expressive speech modeling, offering robust data partitioning, annotated context, challenging test benchmarks, and rigorous evaluation paradigms for the advancement of expressive, zero-shot TTS technologies [2509.04072].

Source: https://www.emergentmind.com/topics/libriquote-dataset