TTS-PRISM: Mandarin TTS Diagnostic Model

• The paper presents TTS-PRISM as a diagnostic framework that evaluates Mandarin TTS on 12 perceptual dimensions, moving beyond a single MOS score to uncover subtle artifacts.
• The model leverages a 7B-parameter MiMo-Audio backbone with an interleaved rationale and score output to enhance interpretability and pinpoint fine-grained weaknesses such as tone sandhi errors.
• Profiling experiments and ablation studies demonstrate its capacity to diagnose issues like perceptual collapse and to compare TTS paradigms under a unified, multidimensional scoring schema.

TTS-PRISM is an end-to-end perceptual reasoning and diagnostic model for Mandarin text-to-speech that is designed to move beyond single-score evaluation such as MOS and instead score speech along 12 perceptual dimensions, generate natural-language explanations grounded in explicit scoring criteria, and expose fine-grained weaknesses associated with perceptual collapse in modern TTS systems (Wang et al., 24 Apr 2026). In the same work, the term also denotes a multi-dimensional diagnostic schema, a targeted synthesis and annotation pipeline built around adversarial perturbations and expert anchors, a schema-driven instruction-tuned evaluator built on MiMo-Audio, and a profiling tool for comparing six TTS paradigms under a common perceptual framework.

1. Motivation and diagnostic scope

The central problem addressed by TTS-PRISM is that modern generative TTS models can sound globally good while still exhibiting localized artifacts such as subtle pronunciation errors, inconsistent emotion, or missing emphasis. The paper describes this failure mode as perceptual collapse: listeners suddenly find an utterance unnatural or distracting despite high overall MOS (Wang et al., 24 Apr 2026).

The framework is explicitly positioned against three limitations of conventional evaluation. First, a single scalar MOS is black-box and does not localize which perceptual dimension is failing. Second, objective acoustic metrics such as MCD, STOI, and PESQ do not directly capture perceptual nuances such as stress, emotion consistency, or prosodic appropriateness. Third, generic audio-LLMs used as judges may produce high-level or coherent-sounding rationales without language-specific fine-grained diagnoses, especially for Mandarin phenomena such as tone sandhi and polyphones (Wang et al., 24 Apr 2026).

TTS-PRISM therefore formalizes evaluation as a dimension-wise diagnostic task rather than as global quality regression. For each sample, the framework tracks a 12-dimensional score vector rather than collapsing quality to a single number. This design is intended to reveal which aspects of speech are correct, stable, expressive, or deficient, and to provide explanations in terms of acoustic phenomena instead of opaque overall judgments (Wang et al., 24 Apr 2026).

2. Twelve-dimensional schema

The schema is hierarchical: an 8-dimension Basic Capability Layer scored on a 1–5 scale, and a 4-dimension Advanced Expressiveness Layer scored as 0–2 bonus, where 0 is neutral, not a penalty (Wang et al., 24 Apr 2026).

Layer	Dimension	Scale / focus
Basic Capability	Audio Clarity	1–5; noise, distortion, artifacts
Basic Capability	Pronunciation Accuracy	1–5; articulation, tones, polyphones
Basic Capability	Intonation	1–5; pitch contour and phrasing
Basic Capability	Pauses	1–5; placement and duration
Basic Capability	Speech Rate	1–5; pacing and rhythmic fluency
Basic Capability	Speaker Consistency	1–5; speaker identity stability
Basic Capability	Style Consistency	1–5; coherence of speaking style
Basic Capability	Emotion Consistency	1–5; stability of emotional category
Advanced Expressiveness	Stress	0–2; prosodic focus
Advanced Expressiveness	Lengthening	0–2; natural syllabic lengthening
Advanced Expressiveness	Paralinguistics	0–2; laughter, sighs, breaths, coughs, filled pauses
Advanced Expressiveness	Emotion Expression	0–2; intensity and fullness of intended sentiment

Within the Basic Capability Layer, Audio Clarity covers physical signal quality, including background noise, distortion, residual non-target voices, and artifacts. The score guidelines are explicitly anchored: Score 4 corresponds to stationary, low-level noise with uniform distribution and constant energy, such as slight Gaussian noise or mild electrical hum, while Score 2 corresponds to destructive distortions such as frequent popping, clipping, or metallic artifacts that directly hinder intelligibility (Wang et al., 24 Apr 2026).

Pronunciation Accuracy is defined more narrowly than ASR correctness. It targets incomplete articulation or slurring, Mandarin-specific confusions such as nasal versus lateral $n/l$, tone sandhi errors, polyphone disambiguation in context, and sub-phoneme errors that yield a robotic or foreign-accent impression. The three prosody dimensions—Intonation, Pauses, and Speech Rate—decompose prosody accuracy into contour, segmentation, and pacing. The three consistency dimensions—Speaker Consistency, Style Consistency, and Emotion Consistency—track whether an utterance remains stable in identity, style, and emotional category throughout (Wang et al., 24 Apr 2026).

The Advanced Expressiveness Layer measures cues that go beyond correctness. Stress evaluates emphasis on keywords via pitch and/or loudness; Lengthening evaluates natural syllabic elongation at phrase boundaries or emphasis points; Paralinguistics captures non-verbal vocal cues; and Emotion Expression measures how fully and intensely the speech realizes the intended sentiment of the text. In this layer, a score of 0 indicates absence of the cue rather than degradation, whereas 1 and 2 indicate increasing presence or intensity (Wang et al., 24 Apr 2026).

For evaluation, the paper defines ground-truth and predicted per-dimension scores $S_{gt,i}$ and $S_{pred,i}$, and measures alignment using Linear Correlation Coefficient (LCC), Spearman Rank Correlation (SRCC), and Normalized MSE. No global scalar quality is defined; instead, profiling and diagnostic flags are derived from the 12-dimensional vector (Wang et al., 24 Apr 2026).

3. Dataset construction and annotation protocol

TTS-PRISM is trained on a purpose-built Mandarin diagnostic dataset generated by a targeted synthesis pipeline with adversarial perturbations and expert anchors (Wang et al., 24 Apr 2026). Rather than passively collecting TTS outputs, the pipeline actively injects perturbations that break specific dimensions, including prosody and rhythm perturbations such as wrong pauses and unnatural tempo, pronunciation errors involving Mandarin homophones and tone sandhi, audio degradation such as noise and distortion, and consistency breaches involving speaker, style, or emotion shifts.

The dataset is shaped by both positive and negative anchors. Very high-quality human recordings and top TTS systems define the upper bound for each dimension, while perturbations and low-quality conditions define the lower bound. For Stress and Lengthening, the authors use custom professional human recordings as gold anchors because existing TTS models do not provide sufficiently high-quality examples. For Paralinguistics and Emotion Expression, they use NVSpeech and FireRedTTS-2 as upper-bound ceiling references. The result is intended to provide balanced positive and negative samples and full coverage across the 1–5 and 0–2 scales (Wang et al., 24 Apr 2026).

The instruction-tuning dataset contains approximately 200k Mandarin samples. Evaluation is conducted on a 1,600-sample Gold Test Set that is stratified, strictly disjoint from the training data, and contains 20% OOD material, including unseen TTS systems and real recordings. The text content spans literary, conversational, and web corpora. For pronunciation, the authors additionally construct an 11k-sample expert-annotated “Pronunciation Gold Subset” to inject Mandarin-specific linguistic knowledge and correct LLM weaknesses in tone sandhi and polyphones (Wang et al., 24 Apr 2026).

Annotation is hybrid human plus LLM. Gemini-2.5-Pro is first used to decompose evaluation into 12 independent per-dimension tasks, a design intended to mitigate instruction drift and hallucination when judging multiple dimensions simultaneously. Human refinement is then applied where Gemini is weaker, especially for Stress and Lengthening, using human-instructed rationale refinement as in InstructScore. Gold-set labels are validated by expert consensus. The language scope is strictly Mandarin Chinese, with deliberate emphasis on phonetic and tonal details that are often under-served by English-centric benchmarks (Wang et al., 24 Apr 2026).

4. Model architecture and schema-driven instruction tuning

The evaluator model uses MiMo-Audio as backbone, specifically the 7B-parameter version in this work, described as a large audio-LLM pretrained on approximately 100M hours of audio in an unsupervised fashion (Wang et al., 24 Apr 2026). Input is audio as the primary modality. At inference time, the model emits, for each of the 12 dimensions, both a rationale $R_i$ and a score $S_i$, organized as a single interleaved target sequence:

$Y = [R_1, S_1, R_2, S_2,\dots, R_{12}, S_{12}].$

This interleaved design is a core architectural choice. The model is required to reason before scoring for each dimension, and the paper treats this as a logical regularizer: generation of rationales is intended to force attention to acoustic criteria rather than permit direct shortcut mapping from audio to score tokens (Wang et al., 24 Apr 2026).

Training is full-parameter supervised fine-tuning on the 200k aligned samples. The reported optimizer is AdamW, with batch size 1 and learning rate $1\times10^{-6}$. The loss is standard token-level cross-entropy over the target sequence:

$$\mathcal{L} = -\sum_{t=1}^{T} \log p_\theta(y_t \mid y_{<t}, x).$$

Here, $x$ is the input audio and $y_1,\dots,y_T$ are the tokens in $S_{gt,i}$0 (Wang et al., 24 Apr 2026).

Interpretability is therefore expressed in natural language rather than as frame-level attribution. Rationales can reference phenomena such as background hum, overly fast segments, or absent stress, and can explicitly tie those observations to score criteria. To assess whether rationales genuinely support the assigned labels, the paper introduces Rationale Support Consistency (RSC) using Gemini-2.5-Pro as a meta-judge. Given a rationale $S_{gt,i}$1 and the ground-truth score $S_{gt,i}$2, Gemini judges whether the rationale logically supports the score. TTS-PRISM reports RSC = 0.98 (Wang et al., 24 Apr 2026).

5. Human alignment, robustness, and ablations

On the 1,600-sample Gold Test Set, TTS-PRISM is compared with Step-Audio-R1 (33B), Qwen3-Omni (30B), and Gemini-2.5-Pro. Baselines are given the strongest prompting setup available in the paper: 12 separate inferences, one per dimension, to avoid instruction interference. TTS-PRISM still performs single-pass inference (Wang et al., 24 Apr 2026).

The reported results show that the 7B model is competitive with or better than Gemini-2.5-Pro on most dimensions in correlation with human ratings. Representative LCC values include Audio Clarity 0.815 for TTS-PRISM versus 0.756 for Gemini, and Emotion Expression 0.841 versus 0.808. For Paralinguistics, TTS-PRISM reports 0.723 versus Gemini’s 0.751, described as slightly lower but close. The consistency dimensions—speaker, style, and emotion—are reported at approximately 0.76–0.81 LCC (Wang et al., 24 Apr 2026).

A notable weakness is Pronunciation Accuracy. TTS-PRISM reports LCC 0.511, while Gemini reports 0.613. The paper attributes this to MiMo-Audio’s ASR-pretrained bias toward error-tolerant many-to-one mappings, which is useful for robust recognition but unfavorable for strict defect detection. This limitation is described as difficult to fully override through the present training setup (Wang et al., 24 Apr 2026).

Layer-wise robustness is reported separately for in-distribution and out-of-distribution data. On ID samples, the averages are Basic Capability: LCC 0.729, SRCC 0.733, MSE$S_{gt,i}$3 0.041 and Advanced Expressiveness: LCC 0.716, SRCC 0.720, MSE$S_{gt,i}$4 0.045. On the 20% OOD subset, the averages are Basic Capability: LCC 0.690, MSE$S_{gt,i}$5 0.051 and Advanced Expressiveness: LCC 0.675, MSE$S_{gt,i}$6 0.060. The paper interprets this as good generalization to unseen TTS systems and real speech (Wang et al., 24 Apr 2026).

Ablations show that each component materially contributes to performance. Training w/o Negatives causes LCC 0.150, which the paper highlights as worse than the untuned backbone and as evidence that hard negative examples are essential. w/o Instruction Tuning yields LCC 0.320, SRCC 0.302, showing that the pretrained backbone does not inherently provide the required diagnostic behavior. w/o CoT, where the model predicts direct scores without rationales, yields LCC 0.662 compared with 0.717 for the full model, supporting the claim that reasoning acts as an auxiliary regularizer (Wang et al., 24 Apr 2026).

The comparison between rationale quality and alignment is also central. Reported RSC scores are 0.88 for Qwen3-Omni, 0.91 for Step-Audio-R1, and 0.98 for TTS-PRISM. The paper notes that some baselines achieve high RSC yet low LCC, meaning they can produce plausible explanations that do not match actual human scores. TTS-PRISM is presented as coupling high rationale support with high human alignment (Wang et al., 24 Apr 2026).

6. Profiling TTS paradigms and operational use

Beyond per-sample evaluation, TTS-PRISM is used to profile six high-end Mandarin TTS systems: F5-TTS, CosyVoice 3, MaskGCT, Qwen3-TTS, FireRedTTS-2, and IndexTTS2 (Wang et al., 24 Apr 2026). For each system, 500 utterances are synthesized. Basic capabilities are evaluated under default settings using plain text, while advanced expressiveness is evaluated with fully activated expressive controls such as audio prompts and style tags, after which the best achievable average is used to avoid configuration bias.

The profiling results reveal a pronounced ceiling effect in the Basic Capability Layer, with most systems scoring above 4.5 on core dimensions. Qwen3-TTS attains the highest reported Pronunciation Accuracy at 4.860. CosyVoice 3 reports the highest Audio Clarity at 4.803, while IndexTTS2 leads Intonation at 4.787. For Pauses, CosyVoice 3 is highest at 4.829, with Qwen3-TTS close at 4.783. Speaker Consistency is near perfect across systems, at approximately 4.99 (Wang et al., 24 Apr 2026).

The Advanced Expressiveness Layer exposes clearer architectural differences. IndexTTS2 attains the highest Emotion Expression at 1.043 and Lengthening at 1.033, and is labeled “Highly Expressive.” CosyVoice 3 reaches the highest Stress at 1.390 and Paralinguistics at 0.735, and is labeled “Paralinguistic-Enhanced.” MaskGCT shows a severe weakness in Lengthening at 0.067, yielding the flag “Prosody-Limited.” F5-TTS combines strong Basic Capability with Paralinguistics 0.114, yielding “Stable but Flat.” FireRedTTS-2 is described as “Balanced.” Qwen3-TTS combines strong pronunciation with expressive scores such as Lengthening 0.890 and Paralinguistics 0.297, yielding “Pronunciation-Accurate.” These flags are intended as intuitive summaries of the 12-dimensional profile (Wang et al., 24 Apr 2026).

For practitioners, the operational workflow is straightforward. One generates speech from a target TTS system, feeds each audio sample to TTS-PRISM, and receives 12 scores and 12 rationales per utterance. The outputs can then be used to identify systematic weaknesses, track progress during model iteration, and benchmark against public systems under the same schema. The authors release code and model checkpoints at https://github.com/xiaomi-research/tts-prism, together with the explicit 12-dimensional scoring criteria (Wang et al., 24 Apr 2026).

7. Limitations, future directions, and disambiguation

The limitations identified in the paper are specific and substantial (Wang et al., 24 Apr 2026). The framework is Mandarin-only, and its scoring criteria are tailored to Mandarin phonetics, tones, and polyphones; cross-lingual transfer is therefore not guaranteed. Pronunciation remains weaker than in Gemini because the backbone is ASR-pretrained and optimized for robust recognition rather than error detection. The annotation pipeline includes LLM-assisted labeling, which may introduce bias despite the use of expert anchors and expert consensus. Inference cost is non-trivial because the evaluator is a 7B-parameter MiMo-Audio model, and interpretability remains limited to text-level rationales rather than explicit frame-level attention or token-level attribution (Wang et al., 24 Apr 2026).

The future directions named in the paper include reinforcement learning to better align scores with human preferences, multi-language extension, addition of more perceptual dimensions such as discourse-level coherence or cross-sentence prosody, use of TTS-PRISM as a reward model for TTS training, and richer rationales potentially grounded in time-frequency references (Wang et al., 24 Apr 2026). A plausible implication is that the framework is intended not only as an evaluator but also as a candidate supervisory signal for future model optimization, provided the alignment and bias issues are handled carefully.

The name also requires disambiguation. TTS-PRISM in (Wang et al., 24 Apr 2026) is a text-to-speech diagnostic model; it should not be conflated with “Prism” as a test-time scaling framework for discrete diffusion LLMs, where “TTS” means test-time scaling rather than text-to-speech (Bai et al., 2 Feb 2026). It is also distinct from PRISM for empathetic spoken dialogue, which uses a multi-agent architecture with a StyleTTS2-based Vocalizer but does not define a separate TTS-PRISM system (Zhang et al., 11 Jun 2026). A further unrelated use of PRISM appears in distributed Transformer inference for edge deployment, including ViT, BERT, and GPT-2, again without referring to the Mandarin perceptual-diagnosis framework (Qazi et al., 16 Jul 2025).