--- title: 'TTS-PRISM: Mandarin TTS Diagnostic Model' url: https://www.emergentmind.com/topics/tts-prism type: topic --- # TTS-PRISM: Mandarin TTS Diagnostic Model TTS-PRISM is an end-to-end perceptual reasoning and diagnostic model for Mandarin text-to-speech that is designed to move beyond single-score evaluation such as MOS and instead score speech along 12 perceptual dimensions, generate natural-language explanations grounded in explicit scoring criteria, and expose fine-grained weaknesses associated with perceptual collapse in modern TTS systems [2604.22225]. In the same work, the term also denotes a multi-dimensional diagnostic schema, a targeted synthesis and annotation pipeline built around adversarial perturbations and expert anchors, a schema-driven instruction-tuned evaluator built on MiMo-Audio, and a profiling tool for comparing six TTS paradigms under a common perceptual framework. ## 1. Motivation and diagnostic scope The central problem addressed by TTS-PRISM is that modern generative TTS models can sound globally good while still exhibiting localized artifacts such as subtle pronunciation errors, inconsistent emotion, or missing emphasis. The paper describes this failure mode as **perceptual collapse**: listeners suddenly find an utterance unnatural or distracting despite high overall MOS [2604.22225]. The framework is explicitly positioned against three limitations of conventional evaluation. First, a single scalar MOS is black-box and does not localize which perceptual dimension is failing. Second, objective acoustic metrics such as MCD, STOI, and PESQ do not directly capture perceptual nuances such as stress, emotion consistency, or prosodic appropriateness. Third, generic audio-LLMs used as judges may produce high-level or coherent-sounding rationales without language-specific fine-grained diagnoses, especially for Mandarin phenomena such as tone sandhi and polyphones [2604.22225]. TTS-PRISM therefore formalizes evaluation as a dimension-wise diagnostic task rather than as global quality regression. For each sample, the framework tracks a 12-dimensional score vector rather than collapsing quality to a single number. This design is intended to reveal which aspects of speech are correct, stable, expressive, or deficient, and to provide explanations in terms of acoustic phenomena instead of opaque overall judgments [2604.22225]. ## 2. Twelve-dimensional schema The schema is hierarchical: an **8-dimension Basic Capability Layer** scored on a **1–5** scale, and a **4-dimension Advanced Expressiveness Layer** scored as **0–2 bonus**, where **0 is neutral, not a penalty** [2604.22225]. | Layer | Dimension | Scale / focus | |---|---|---| | Basic Capability | Audio Clarity | 1–5; noise, distortion, artifacts | | Basic Capability | Pronunciation Accuracy | 1–5; articulation, tones, polyphones | | Basic Capability | Intonation | 1–5; pitch contour and phrasing | | Basic Capability | Pauses | 1–5; placement and duration | | Basic Capability | Speech Rate | 1–5; pacing and rhythmic fluency | | Basic Capability | Speaker Consistency | 1–5; speaker identity stability | | Basic Capability | Style Consistency | 1–5; coherence of speaking style | | Basic Capability | Emotion Consistency | 1–5; stability of emotional category | | Advanced Expressiveness | Stress | 0–2; prosodic focus | | Advanced Expressiveness | Lengthening | 0–2; natural syllabic lengthening | | Advanced Expressiveness | Paralinguistics | 0–2; laughter, sighs, breaths, coughs, filled pauses | | Advanced Expressiveness | Emotion Expression | 0–2; intensity and fullness of intended sentiment | Within the Basic Capability Layer, **Audio Clarity** covers physical signal quality, including background noise, distortion, residual non-target voices, and artifacts. The score guidelines are explicitly anchored: **Score 4** corresponds to stationary, low-level noise with uniform distribution and constant energy, such as slight Gaussian noise or mild electrical hum, while **Score 2** corresponds to destructive distortions such as frequent popping, clipping, or metallic artifacts that directly hinder intelligibility [2604.22225]. **Pronunciation Accuracy** is defined more narrowly than ASR correctness. It targets incomplete articulation or slurring, Mandarin-specific confusions such as nasal versus lateral \(n/l\), tone sandhi errors, polyphone disambiguation in context, and sub-phoneme errors that yield a robotic or foreign-accent impression. The three prosody dimensions—**Intonation**, **Pauses**, and **Speech Rate**—decompose prosody accuracy into contour, segmentation, and pacing. The three consistency dimensions—**Speaker Consistency**, **Style Consistency**, and **Emotion Consistency**—track whether an utterance remains stable in identity, style, and emotional category throughout [2604.22225]. The Advanced Expressiveness Layer measures cues that go beyond correctness. **Stress** evaluates emphasis on keywords via pitch and/or loudness; **Lengthening** evaluates natural syllabic elongation at phrase boundaries or emphasis points; **Paralinguistics** captures non-verbal vocal cues; and **Emotion Expression** measures how fully and intensely the speech realizes the intended sentiment of the text. In this layer, a score of **0** indicates absence of the cue rather than degradation, whereas **1** and **2** indicate increasing presence or intensity [2604.22225]. For evaluation, the paper defines ground-truth and predicted per-dimension scores \(S_{gt,i}\) and \(S_{pred,i}\), and measures alignment using **Linear Correlation Coefficient (LCC)**, **Spearman Rank Correlation (SRCC)**, and **Normalized MSE**. No global scalar quality is defined; instead, profiling and diagnostic flags are derived from the 12-dimensional vector [2604.22225]. ## 3. Dataset construction and annotation protocol TTS-PRISM is trained on a purpose-built Mandarin diagnostic dataset generated by a targeted synthesis pipeline with adversarial perturbations and expert anchors [2604.22225]. Rather than passively collecting TTS outputs, the pipeline actively injects perturbations that break specific dimensions, including prosody and rhythm perturbations such as wrong pauses and unnatural tempo, pronunciation errors involving Mandarin homophones and tone sandhi, audio degradation such as noise and distortion, and consistency breaches involving speaker, style, or emotion shifts. The dataset is shaped by both positive and negative anchors. Very high-quality human recordings and top TTS systems define the upper bound for each dimension, while perturbations and low-quality conditions define the lower bound. For **Stress** and **Lengthening**, the authors use **custom professional human recordings** as gold anchors because existing TTS models do not provide sufficiently high-quality examples. For **Paralinguistics** and **Emotion Expression**, they use **NVSpeech** and **FireRedTTS-2** as upper-bound ceiling references. The result is intended to provide balanced positive and negative samples and full coverage across the 1–5 and 0–2 scales [2604.22225]. The instruction-tuning dataset contains approximately **200k Mandarin samples**. Evaluation is conducted on a **1,600-sample Gold Test Set** that is stratified, strictly disjoint from the training data, and contains **20% OOD** material, including unseen TTS systems and real recordings. The text content spans **literary, conversational, and web corpora**. For pronunciation, the authors additionally construct an **11k-sample expert-annotated “Pronunciation Gold Subset”** to inject Mandarin-specific linguistic knowledge and correct LLM weaknesses in tone sandhi and polyphones [2604.22225]. Annotation is hybrid human plus LLM. **Gemini-2.5-Pro** is first used to decompose evaluation into **12 independent per-dimension tasks**, a design intended to mitigate instruction drift and hallucination when judging multiple dimensions simultaneously. Human refinement is then applied where Gemini is weaker, especially for **Stress** and **Lengthening**, using human-instructed rationale refinement as in InstructScore. Gold-set labels are validated by **expert consensus**. The language scope is strictly Mandarin Chinese, with deliberate emphasis on phonetic and tonal details that are often under-served by English-centric benchmarks [2604.22225]. ## 4. Model architecture and schema-driven instruction tuning The evaluator model uses **MiMo-Audio** as backbone, specifically the **7B-parameter** version in this work, described as a large audio-language model pretrained on approximately **100M hours of audio** in an unsupervised fashion [2604.22225]. Input is audio as the primary modality. At inference time, the model emits, for each of the 12 dimensions, both a rationale \(R_i\) and a score \(S_i\), organized as a single interleaved target sequence: $$ Y = [R_1, S_1, R_2, S_2,\dots, R_{12}, S_{12}]. $$ This interleaved design is a core architectural choice. The model is required to **reason before scoring** for each dimension, and the paper treats this as a logical regularizer: generation of rationales is intended to force attention to acoustic criteria rather than permit direct shortcut mapping from audio to score tokens [2604.22225]. Training is **full-parameter supervised fine-tuning** on the 200k aligned samples. The reported optimizer is **AdamW**, with **batch size 1** and **learning rate \(1\times10^{-6}\)**. The loss is standard token-level cross-entropy over the target sequence: $$ \mathcal{L} = -\sum_{t=1}^{T} \log p_\theta(y_t \mid y_{