---
title: 'SpeechRole-Data: Speech Role-Playing Corpus'
url: https://www.emergentmind.com/topics/speechrole-data
type: topic
---

# SpeechRole-Data: Speech Role-Playing Corpus

SpeechRole-Data denotes a role-conditioned dialogue resource whose most prominent recent usage is the large-scale corpus introduced for **speech-based role-playing** in "SpeechRole: A Large-Scale Dataset and Benchmark for Evaluating Speech Role-Playing Agents." In that formulation, it comprises **98 diverse roles** and **112k speech-based single-turn and multi-turn conversations**, with each example pairing dialogue text, speech, contextual background information, a role profile, and a reference audio clip representing the character’s voice. The dataset is coupled with **SpeechRole-Eval**, a benchmark for **Speech Role-Playing Agents (SRPAs)** that measures fundamental interaction ability, speech expressiveness, and role-playing fidelity, and it has subsequently been reused as a benchmark substrate in later SRPA studies [2508.02013][2606.17669].

## 1. Nomenclature and scope

In the 2025 SRPA literature, **SpeechRole-Data** is a **large-scale speech role-playing corpus** created to address the gap between text-only role-playing evaluation and realistic spoken interaction, where **voice identity, timbre, prosody, emotion, and interaction style** are central to whether an agent sounds like a specific character [2508.02013]. The resource is explicitly bilingual, spans **games, movies, TV series, and animation**, and is designed for both **training** and **evaluation** of SRPAs.

The same label also appears in a distinct 2024 role-playing setting in "Thinking Before Speaking: A Role-playing Model with Mindset," where **SpeechRole-Data** refers to a training dataset built around **Role profile summaries from Wikipedia**, **Historical dialogues**, **Scenario expansions generated from the character’s life story**, **Mindset / thinking traces**, and **Hallucination knowledge** used to train refusal on out-of-scope questions. That dataset is reported to contain **152 roles** and **889,779 sentences** [2409.13752].

These two usages describe different objects. One is a **speech corpus and benchmark for SRPAs** [2508.02013]; the other is a **mindset-augmented role-playing dataset** for TBS-style text generation [2409.13752]. This suggests that the label requires paper-specific disambiguation.

## 2. Corpus construction in the SRPA formulation

In the SRPA formulation, SpeechRole-Data is built through a staged pipeline that starts from **role selection** and ends with **reference-voice identification**. The authors curate **98 roles** from two existing role-playing datasets: **18 roles** from **ChatHaruhi** and **80 roles** from **RoleLLM**. These roles are chosen to balance **language, gender, and personality**, with an explicit preference for characters with **distinctive vocal characteristics** [2508.02013].

For each character, metadata is extracted in three parts. **Role Profile** covers temperament, behavioral tendencies, values, and preferences. **Background** covers social identity, relationships, living environment, and settings. **Character Lines** include narration, monologues, and dialogue excerpts from the original works. This metadata is then used to prompt **gpt-4.1-2025-04-14** to generate roughly **800 single-turn dialogues** and **800 multi-turn dialogues** per role before filtering [2508.02013].

Filtering is multi-stage. **Turn-pattern filtering** keeps only dialogues with a clear alternating structure in which the user asks and the role responds. **Context relevance filtering** computes semantic similarity between generated dialogue and source metadata using **all_MiniLM_L6_v2** for English and **text2vec-bge-large-chinese** for Chinese; dialogues with similarity below **0.4** in English or **0.45** in Chinese are discarded. **Near-duplicate removal** then removes highly similar examples using thresholds of **0.9** for English and **0.85** for Chinese, applied separately for single-turn and multi-turn dialogues for each role [2508.02013].

The speech side is built from collected character audio. The authors report downloading **105 video files** totaling about **181 hours** from **Genshin Impact** and other film and television sources. Audio is extracted with **ffmpeg**, converted to **mono 16 kHz WAV**, and cleaned with the **Emilia** framework, which performs **source separation**, **speaker diarization**, and **voice activity detection**. This yields clean **3–10 second single-speaker clips** [2508.02013].

The clips are transcribed with **whisper-medium-v3** and filtered with **DNSMOS P.835 OVRL**; clips with **DNSMOS < 3** are removed. Because diarization provides only speaker indices, the correct character is identified with **gpt-4.1-2025-04-14** and **DeepSeek-V3-0324**, followed by manual verification. Finally, **CAM++** speaker embeddings are used to choose the clip with the **highest average cosine similarity** to the other clips of the same character as the canonical reference audio for that role [2508.02013].

## 3. Data schema, splits, and role properties

Each example in SpeechRole-Data includes **the dialogue text**, **the corresponding speech**, **contextual background information**, **a role profile**, and **a reference audio clip** representing the character’s voice. The final corpus contains **53,902 single-turn dialogues** and **57,944 multi-turn dialogues**, for **111,846 dialogues** in total, reported as **112k** in the abstract [2508.02013].

The split statistics are as follows.

| Split | Characters and samples | Speech hours |
|---|---|---:|
| Train | **78 characters**, **89,461 samples** | **875.25** |
| Dev | **20 characters**, **21,993 samples** | **203.60** |
| Test | **98 characters**, **392 samples** | **4.20** |

Within these splits, the paper reports **43,082** single-turn and **46,379** multi-turn samples in train, **10,627** single-turn and **11,366** multi-turn in dev, and **196** single-turn plus **196** multi-turn in test. The test set uses **two single-turn and two multi-turn dialogues per role** [2508.02013].

A defining property of the corpus is that roles are differentiated not only by identity but by **voice style**. The paper repeatedly emphasizes **timbre**, **prosody**, **rhythm**, **intonation**, **emotion**, and **expressiveness** as character-defining dimensions. It also notes that role utterances are generally longer than user utterances: role utterances are typically **10–20 seconds**, while user utterances are typically **3–6 seconds**. This suggests that the corpus is structured around asymmetric interaction, with short prompts eliciting longer, more character-specific spoken replies [2508.02013].

## 4. SpeechRole-Eval and benchmark protocol

SpeechRole-Data is paired with **SpeechRole-Eval**, an evaluation benchmark designed to measure whether an SRPA can produce **character-consistent spoken dialogue**. The benchmark organizes assessment into **three broad dimensions** and **eight evaluation metrics** [2508.02013].

| Dimension | Metrics |
|---|---|
| Fundamental interaction ability | **Instruction Adherence (IA)**, **Speech Fluency (SF)**, **Conversational Coherence (CC)** |
| Speech quality and expressiveness | **Speech Naturalness (SN)**, **Prosodic Consistency (PC)**, **Emotion Appropriateness (EA)** |
| Role-playing fidelity | **Personality Consistency (PeC)**, **Knowledge Consistency (KC)** |

Prompt design is also structured by task type. The benchmark defines **Internal Reasoning**, **Experiential Narration**, and **Social Communication**, which are further divided into **8 subtask types**, including opinion/emotion inquiry, reasoning/motivation, action description, historical/background description, and advice/life experience [2508.02013].

Scoring is automated with **Gemini 2.5 Pro preview** and validated against human judgments. The procedure is explicitly **reference-based**: the model first generates a short rationale, and each test response is scored together with a **high-quality reference answer** rather than in isolation. The paper states that **“The final evaluation score is calculated as the ratio of the test speech score to the reference score.”** Ratings are on a **1 to 10 scale** [2508.02013].

Human-correlation verification uses **20 dialogue samples**, **2 model-generated responses per sample**, and **4 human annotators**. The reported agreement between automatic and human scoring is **Overall MSE = 0.1260** and **Overall Pearson = 0.7162**. Per-metric Pearson correlations are especially high for **Emotion Appropriateness: 0.8881**, **Speech Naturalness: 0.8093**, and **Knowledge Consistency: 0.7581**. These numbers indicate that the benchmark is intended not merely as a convenience metric, but as an automatic proxy calibrated against human preference [2508.02013].

## 5. Baselines, empirical findings, and training utility

The benchmark compares two SRPA families. **Cascaded SRPAs** decompose the task into **ASR**, **LLM-based character-consistent text generation**, and **TTS**, using components such as **Paraformer Realtime ASR v2**, **Qwen-Plus-Character**, **CosyVoice-v2**, **whisper-large-v3-turbo**, **LLaMA3.1-8B**, **XTTS**, **CosyVoice2**, **E2-TTS**, **F5-TTS**, **FireRedTTS**, **GPT-SoVITS**, **Spark-TTS**, and **MaskGCT**. **End-to-end SRPAs** include **GPT-4o Audio** and **Qwen2.5-Omni-7B** [2508.02013].

The main comparative result is that **cascaded models generally outperform end-to-end models** across most metrics. Among the cascaded baselines, **E2-TTS**, **F5-TTS**, **Spark-TTS**, and **CosyVoice2** are identified as particularly strong. By contrast, end-to-end systems, especially **GPT-4o Audio**, can be competitive on **expressiveness**, **coherence**, and **speech naturalness**, but still struggle with **style consistency**, **preserving role settings**, and **voice identity consistency over longer dialogue**. **Qwen2.5-Omni** performs substantially worse than the other models in this benchmark [2508.02013].

The paper also reports a systematic language effect: models generally perform better on **English** than on **Chinese**, especially in **content coherence** and **knowledge consistency**. This is attributed to **pretraining imbalance** and **TTS limitations**. For voice-cloning-capable models, speaker similarity is measured using **WavLM-large** embeddings and cosine similarity. In English, **E2-TTS** and **F5-TTS** achieve the highest speaker similarity, while **GPT-SoVITS** performs worst; in Chinese, **MaskGCT** slightly outperforms the others in speaker similarity. The paper emphasizes that speaker similarity alone does not guarantee **role consistency** or **prosodic/personality consistency** [2508.02013].

A central training result is the fine-tuning of **Qwen2.5-Omni-7B** on the SpeechRole training set to produce **SpeechRole-Agent**. The reported overall-score improvements are substantial: **English in-domain** rises from **0.4340** to **0.7098**, **Chinese in-domain** from **0.5296** to **0.8468**, **English out-of-domain** from **0.4786** to **0.6795**, and **Chinese out-of-domain** from **0.5117** to **0.8028**. The improvements are described as covering **instruction adherence**, **emotional appropriateness**, **personality consistency**, and **generalization to unseen roles**. A plausible implication is that SpeechRole-Data functions not only as an evaluation resource but also as a supervision source for improving end-to-end SRPAs [2508.02013].

## 6. Later reuse and position in the benchmark landscape

Later SRPA work uses SpeechRole-Data as a benchmark rather than merely as a training corpus. In **DeSRPA**, SpeechRole is treated as the **primary benchmark** for character fidelity in **plot-based, source-anchored settings**, and the experiments use the **SpeechRole-Data test split** with **72 English characters** from movies and TV series, **around 10 dialogue turns per character on average**, and **372 evaluated responses** total. The benchmark is evaluated along **Instruction adherence**, **Speech fluency**, **Conversation coherence**, **Speech naturalness**, **Prosodic consistency**, **Emotion appropriateness**, **Personality consistency**, and **Knowledge consistency**, together with **TTFA**, **SIM**, **EEA**, and **WER**. On this benchmark, **DeSRPA** reports a **mean automated score** of **0.8379**, compared with **0.7747** for the **SpeechRole baseline**, with especially strong scores in **Prosodic Consistency: 0.7958**, **Emotion Appropriateness: 0.8160**, **Personality Consistency: 0.7615**, and **Knowledge Consistency: 0.8116** [2606.17669].

The benchmark has also become part of a broader ecosystem of speech role-playing evaluation resources. **VoxRole** frames itself as a comprehensive benchmark for speech-based RPCAs with **13,335 multi-turn dialogues**, **65.6 hours** of speech, **1,228 unique characters**, and **261 movies**, focusing on long-term persona consistency and multi-dimensional character profiles [2509.03940]. **RoleChat** is presented as the first voice role-playing evaluation dataset enriched with **chain-of-thought reasoning annotations**, containing **50 distinct characters** and **14,032 samples**, and is used to train the **RoleJudge** evaluator [2604.13804]. **Speech-DRAME-EvalBench** separates **Archetype Evaluation** from **Realism Evaluation**, with **8,280** bilingual archetype samples and **15,000** realism samples for training and testing speech evaluation models [2511.01261].

Within this landscape, SpeechRole-Data occupies a distinct position. It couples **role-grounded dialogue generation**, **reference character audio**, and **human-correlated automated evaluation** inside a single SRPA framework [2508.02013]. Subsequent benchmark work broadens evaluation toward richer persona taxonomies, realism, or evaluator training [2509.03940][2604.13804][2511.01261], but SpeechRole-Data remains notable for tying **training data**, **reference voice selection**, and **multidimensional SRPA scoring** into one unified resource.

Source: https://www.emergentmind.com/topics/speechrole-data