SpeechRole-Data: Speech Role-Playing Corpus

• SpeechRole-Data is a large-scale speech role-playing corpus that integrates dialogue text, speech, context, role profiles, and reference audio to support realistic SRPAs.
• It employs a robust pipeline for dialogue generation, voice extraction, and multi-stage filtering using both automatic metrics and manual verification.
• The resource underpins benchmark evaluations with multidimensional metrics for interaction, speech expressiveness, and role consistency to enhance agent training.

SpeechRole-Data denotes a role-conditioned dialogue resource whose most prominent recent usage is the large-scale corpus introduced for speech-based role-playing in "SpeechRole: A Large-Scale Dataset and Benchmark for Evaluating Speech Role-Playing Agents." In that formulation, it comprises 98 diverse roles and 112k speech-based single-turn and multi-turn conversations, with each example pairing dialogue text, speech, contextual background information, a role profile, and a reference audio clip representing the character’s voice. The dataset is coupled with SpeechRole-Eval, a benchmark for Speech Role-Playing Agents (SRPAs) that measures fundamental interaction ability, speech expressiveness, and role-playing fidelity, and it has subsequently been reused as a benchmark substrate in later SRPA studies (Jiang et al., 4 Aug 2025, Tang et al., 16 Jun 2026).

1. Nomenclature and scope

In the 2025 SRPA literature, SpeechRole-Data is a large-scale speech role-playing corpus created to address the gap between text-only role-playing evaluation and realistic spoken interaction, where voice identity, timbre, prosody, emotion, and interaction style are central to whether an agent sounds like a specific character (Jiang et al., 4 Aug 2025). The resource is explicitly bilingual, spans games, movies, TV series, and animation, and is designed for both training and evaluation of SRPAs.

The same label also appears in a distinct 2024 role-playing setting in "Thinking Before Speaking: A Role-playing Model with Mindset," where SpeechRole-Data refers to a training dataset built around Role profile summaries from Wikipedia, Historical dialogues, Scenario expansions generated from the character’s life story, Mindset / thinking traces, and Hallucination knowledge used to train refusal on out-of-scope questions. That dataset is reported to contain 152 roles and 889,779 sentences (Zhang et al., 2024).

These two usages describe different objects. One is a speech corpus and benchmark for SRPAs (Jiang et al., 4 Aug 2025); the other is a mindset-augmented role-playing dataset for TBS-style text generation (Zhang et al., 2024). This suggests that the label requires paper-specific disambiguation.

2. Corpus construction in the SRPA formulation

In the SRPA formulation, SpeechRole-Data is built through a staged pipeline that starts from role selection and ends with reference-voice identification. The authors curate 98 roles from two existing role-playing datasets: 18 roles from ChatHaruhi and 80 roles from RoleLLM. These roles are chosen to balance language, gender, and personality, with an explicit preference for characters with distinctive vocal characteristics (Jiang et al., 4 Aug 2025).

For each character, metadata is extracted in three parts. Role Profile covers temperament, behavioral tendencies, values, and preferences. Background covers social identity, relationships, living environment, and settings. Character Lines include narration, monologues, and dialogue excerpts from the original works. This metadata is then used to prompt gpt-4.1-2025-04-14 to generate roughly 800 single-turn dialogues and 800 multi-turn dialogues per role before filtering (Jiang et al., 4 Aug 2025).

Filtering is multi-stage. Turn-pattern filtering keeps only dialogues with a clear alternating structure in which the user asks and the role responds. Context relevance filtering computes semantic similarity between generated dialogue and source metadata using all_MiniLM_L6_v2 for English and text2vec-bge-large-chinese for Chinese; dialogues with similarity below 0.4 in English or 0.45 in Chinese are discarded. Near-duplicate removal then removes highly similar examples using thresholds of 0.9 for English and 0.85 for Chinese, applied separately for single-turn and multi-turn dialogues for each role (Jiang et al., 4 Aug 2025).

The speech side is built from collected character audio. The authors report downloading 105 video files totaling about 181 hours from Genshin Impact and other film and television sources. Audio is extracted with ffmpeg, converted to mono 16 kHz WAV, and cleaned with the Emilia framework, which performs source separation, speaker diarization, and voice activity detection. This yields clean 3–10 second single-speaker clips (Jiang et al., 4 Aug 2025).

The clips are transcribed with whisper-medium-v3 and filtered with DNSMOS P.835 OVRL; clips with DNSMOS < 3 are removed. Because diarization provides only speaker indices, the correct character is identified with gpt-4.1-2025-04-14 and DeepSeek-V3-0324, followed by manual verification. Finally, CAM++ speaker embeddings are used to choose the clip with the highest average cosine similarity to the other clips of the same character as the canonical reference audio for that role (Jiang et al., 4 Aug 2025).

3. Data schema, splits, and role properties

Each example in SpeechRole-Data includes the dialogue text, the corresponding speech, contextual background information, a role profile, and a reference audio clip representing the character’s voice. The final corpus contains 53,902 single-turn dialogues and 57,944 multi-turn dialogues, for 111,846 dialogues in total, reported as 112k in the abstract (Jiang et al., 4 Aug 2025).

The split statistics are as follows.

Split	Characters and samples	Speech hours
Train	78 characters, 89,461 samples	875.25
Dev	20 characters, 21,993 samples	203.60
Test	98 characters, 392 samples	4.20

Within these splits, the paper reports 43,082 single-turn and 46,379 multi-turn samples in train, 10,627 single-turn and 11,366 multi-turn in dev, and 196 single-turn plus 196 multi-turn in test. The test set uses two single-turn and two multi-turn dialogues per role (Jiang et al., 4 Aug 2025).

A defining property of the corpus is that roles are differentiated not only by identity but by voice style. The paper repeatedly emphasizes timbre, prosody, rhythm, intonation, emotion, and expressiveness as character-defining dimensions. It also notes that role utterances are generally longer than user utterances: role utterances are typically 10–20 seconds, while user utterances are typically 3–6 seconds. This suggests that the corpus is structured around asymmetric interaction, with short prompts eliciting longer, more character-specific spoken replies (Jiang et al., 4 Aug 2025).

4. SpeechRole-Eval and benchmark protocol

SpeechRole-Data is paired with SpeechRole-Eval, an evaluation benchmark designed to measure whether an SRPA can produce character-consistent spoken dialogue. The benchmark organizes assessment into three broad dimensions and eight evaluation metrics (Jiang et al., 4 Aug 2025).

Dimension	Metrics
Fundamental interaction ability	Instruction Adherence (IA), Speech Fluency (SF), Conversational Coherence (CC)
Speech quality and expressiveness	Speech Naturalness (SN), Prosodic Consistency (PC), Emotion Appropriateness (EA)
Role-playing fidelity	Personality Consistency (PeC), Knowledge Consistency (KC)

Prompt design is also structured by task type. The benchmark defines Internal Reasoning, Experiential Narration, and Social Communication, which are further divided into 8 subtask types, including opinion/emotion inquiry, reasoning/motivation, action description, historical/background description, and advice/life experience (Jiang et al., 4 Aug 2025).

Scoring is automated with Gemini 2.5 Pro preview and validated against human judgments. The procedure is explicitly reference-based: the model first generates a short rationale, and each test response is scored together with a high-quality reference answer rather than in isolation. The paper states that “The final evaluation score is calculated as the ratio of the test speech score to the reference score.” Ratings are on a 1 to 10 scale (Jiang et al., 4 Aug 2025).

Human-correlation verification uses 20 dialogue samples, 2 model-generated responses per sample, and 4 human annotators. The reported agreement between automatic and human scoring is Overall MSE = 0.1260 and Overall Pearson = 0.7162. Per-metric Pearson correlations are especially high for Emotion Appropriateness: 0.8881, Speech Naturalness: 0.8093, and Knowledge Consistency: 0.7581. These numbers indicate that the benchmark is intended not merely as a convenience metric, but as an automatic proxy calibrated against human preference (Jiang et al., 4 Aug 2025).

5. Baselines, empirical findings, and training utility

The benchmark compares two SRPA families. Cascaded SRPAs decompose the task into ASR, LLM-based character-consistent text generation, and TTS, using components such as Paraformer Realtime ASR v2, Qwen-Plus-Character, CosyVoice-v2, whisper-large-v3-turbo, LLaMA3.1-8B, XTTS, CosyVoice2, E2-TTS, F5-TTS, FireRedTTS, GPT-SoVITS, Spark-TTS, and MaskGCT. End-to-end SRPAs include GPT-4o Audio and Qwen2.5-Omni-7B (Jiang et al., 4 Aug 2025).

The main comparative result is that cascaded models generally outperform end-to-end models across most metrics. Among the cascaded baselines, E2-TTS, F5-TTS, Spark-TTS, and CosyVoice2 are identified as particularly strong. By contrast, end-to-end systems, especially GPT-4o Audio, can be competitive on expressiveness, coherence, and speech naturalness, but still struggle with style consistency, preserving role settings, and voice identity consistency over longer dialogue. Qwen2.5-Omni performs substantially worse than the other models in this benchmark (Jiang et al., 4 Aug 2025).

The paper also reports a systematic language effect: models generally perform better on English than on Chinese, especially in content coherence and knowledge consistency. This is attributed to pretraining imbalance and TTS limitations. For voice-cloning-capable models, speaker similarity is measured using WavLM-large embeddings and cosine similarity. In English, E2-TTS and F5-TTS achieve the highest speaker similarity, while GPT-SoVITS performs worst; in Chinese, MaskGCT slightly outperforms the others in speaker similarity. The paper emphasizes that speaker similarity alone does not guarantee role consistency or prosodic/personality consistency (Jiang et al., 4 Aug 2025).

A central training result is the fine-tuning of Qwen2.5-Omni-7B on the SpeechRole training set to produce SpeechRole-Agent. The reported overall-score improvements are substantial: English in-domain rises from 0.4340 to 0.7098, Chinese in-domain from 0.5296 to 0.8468, English out-of-domain from 0.4786 to 0.6795, and Chinese out-of-domain from 0.5117 to 0.8028. The improvements are described as covering instruction adherence, emotional appropriateness, personality consistency, and generalization to unseen roles. A plausible implication is that SpeechRole-Data functions not only as an evaluation resource but also as a supervision source for improving end-to-end SRPAs (Jiang et al., 4 Aug 2025).

6. Later reuse and position in the benchmark landscape

Later SRPA work uses SpeechRole-Data as a benchmark rather than merely as a training corpus. In DeSRPA, SpeechRole is treated as the primary benchmark for character fidelity in plot-based, source-anchored settings, and the experiments use the SpeechRole-Data test split with 72 English characters from movies and TV series, around 10 dialogue turns per character on average, and 372 evaluated responses total. The benchmark is evaluated along Instruction adherence, Speech fluency, Conversation coherence, Speech naturalness, Prosodic consistency, Emotion appropriateness, Personality consistency, and Knowledge consistency, together with TTFA, SIM, EEA, and WER. On this benchmark, DeSRPA reports a mean automated score of 0.8379, compared with 0.7747 for the SpeechRole baseline, with especially strong scores in Prosodic Consistency: 0.7958, Emotion Appropriateness: 0.8160, Personality Consistency: 0.7615, and Knowledge Consistency: 0.8116 (Tang et al., 16 Jun 2026).

The benchmark has also become part of a broader ecosystem of speech role-playing evaluation resources. VoxRole frames itself as a comprehensive benchmark for speech-based RPCAs with 13,335 multi-turn dialogues, 65.6 hours of speech, 1,228 unique characters, and 261 movies, focusing on long-term persona consistency and multi-dimensional character profiles (Wu et al., 4 Sep 2025). RoleChat is presented as the first voice role-playing evaluation dataset enriched with chain-of-thought reasoning annotations, containing 50 distinct characters and 14,032 samples, and is used to train the RoleJudge evaluator (Fu et al., 15 Apr 2026). Speech-DRAME-EvalBench separates Archetype Evaluation from Realism Evaluation, with 8,280 bilingual archetype samples and 15,000 realism samples for training and testing speech evaluation models (Shi et al., 3 Nov 2025).

Within this landscape, SpeechRole-Data occupies a distinct position. It couples role-grounded dialogue generation, reference character audio, and human-correlated automated evaluation inside a single SRPA framework (Jiang et al., 4 Aug 2025). Subsequent benchmark work broadens evaluation toward richer persona taxonomies, realism, or evaluator training (Wu et al., 4 Sep 2025, Fu et al., 15 Apr 2026, Shi et al., 3 Nov 2025), but SpeechRole-Data remains notable for tying training data, reference voice selection, and multidimensional SRPA scoring into one unified resource.