Discretized Speech Weighted Edit Distance (DS-WED)

• DS-WED is a metric that quantifies prosodic differences by applying weighted edit distance to discretized speech token sequences.
• It uses VAD, self-supervised models, and k-means clustering to extract and compare key prosodic features like pitch, rhythm, and stress.
• Empirical evaluations show DS-WED correlates strongly with human ratings, offering a scalable, GPU-friendly tool for TTS and prosody research.

Discretized Speech Weighted Edit Distance (DS-WED) is an objective metric designed to quantify prosodic variation in speech, particularly in the context of zero-shot text-to-speech (TTS) synthesis. By applying weighted edit distance to discretized token sequences derived from self-supervised speech representations, DS-WED captures perceptual distinctions in prosody such as pitch, rhythm, and stress, offering higher alignment with human judgments compared to traditional acoustic metrics (Yang et al., 24 Sep 2025).

1. Foundations and Definition

DS-WED operates by mapping variable-length speech waveforms to discrete semantic token sequences via self-supervised models (e.g., HuBERT, WavLM) followed by clustering, and then assessing pairwise differences using a weighted variant of Levenshtein edit distance. For two speech utterances $X_1$ and $X_2$:

• Apply a voice activity detector (VAD) to produce silence-trimmed segments $\widetilde{X}_1$, $\widetilde{X}_2$.
• Encode $\widetilde{X}_1$, $\widetilde{X}_2$ into token sequences $c_1, c_2$ via a pretrained speech encoder and k-means clustering.
• Compute the minimum weighted edit distance between $c_1$ and $c_2$:

$$\mathrm{DS\text{-}WED}(c_1, c_2) = \min_{\pi \in \mathcal{A}(c_1, c_2)} \sum_{(i, j, o) \in \pi} w_o \cdot c_o(c_{1,i}, c_{2,j})$$

Where $o$ indexes the edit operation (substitution, insertion, deletion), $w_o$ are empirically set weights, and $c_o(\cdot)$ is the operation-specific cost (Yang et al., 24 Sep 2025).

Prosodic variation between two utterances—crucial to perceived naturalness and expressiveness—is thus quantified as the minimum weighted number of perceptual modifications needed to convert one token sequence into the other.

2. Methodological Pipeline

The DS-WED computation is articulated as follows:

1. Preprocessing and Silence Removal: Waveforms are trimmed to active speech segments using pretrained VADs, ensuring analysis focuses on meaningful content.
2. Speech Discretization: Discrete tokenization is achieved by:
   • Passing audio through a self-supervised model to extract hidden features.
   • Applying k-means clustering to convert continuous embeddings to sequences of integer tokens, forming semantic representations resilient to low-level acoustic variability.
3. Weighted Edit Distance Calculation: The token sequences are compared using a weighted edit distance metric, emphasizing operations to which listeners are most perceptually sensitive (e.g., substitutions weighted more than insertions/deletions). Weights are empirically calibrated, with typical settings such as $w_\mathrm{sub}=1.2$ and $w_\mathrm{ins}=w_\mathrm{del}=1.0$ (Yang et al., 24 Sep 2025).
4. Prosody Diversity Quantification: The resultant DS-WED value reflects the overall prosodic diversity between utterances, supporting interpretation in benchmarking, model development, and perceptual studies.

3. Theoretical Underpinnings and Related Work

DS-WED builds conceptually on weighted edit distance algorithms (see (Das et al., 2023)), where the cost function $w$ can depend on the type and context of edits. In the broader theoretical framework, efficient subquadratic algorithms for weighted edit distance exist (e.g., $O(n+\mathrm{poly}(k))$ time for inputs of length $n$ and distance $k$), and the use of token sequences from pretrained models aligns DS-WED with approaches that compare strings or trees under weighted cost regimes (Das et al., 2023). The integration of tokenization via self-supervised models positions DS-WED within the recent paradigm shift towards representation learning-based metrics in speech analysis.

DS-WED is distinct from exp-edit distance (Baek, 23 Aug 2024), which operates on strings augmented with continuous exponents (e.g., for prosodic duration) and allows partial edit operations; DS-WED instead measures token sequence variation after discretization and is thus robust to continuous but perceptually subtle temporal variations.

4. Empirical Characteristics and Comparative Analysis

Empirical evaluations on the ProsodyEval dataset (Yang et al., 24 Sep 2025), comprising 1,000 synthetic speech samples and 2,000 human ratings, indicate that DS-WED achieves a Pearson correlation of 0.77 with human Prosody Mean Opinion Scores (PMOS)—substantially higher than log F₀ Root Mean Square Error (0.30) or Mel Cepstral Distortion (0.66). DS-WED's robustness extends across variations in:

• Choice of self-supervised encoder (e.g., HuBERT vs. WavLM),
• Tokenization parameters (e.g., number of clusters, SSL layer selection),
• Speech domain (tested on multiple TTS datasets).

The metric is further computationally efficient, reporting a real-time factor (RTF) of 0.110 and suitability for large-scale GPU-based evaluations, unlike CPU-heavy alignment metrics such as DTW (Yang et al., 24 Sep 2025).

5. Factors Influencing Prosody Diversity

Application of DS-WED as an analytic tool on state-of-the-art TTS systems reveals:

• Generative Paradigm: Autoregressive systems exhibit higher DS-WED scores (greater diversity) compared to flow-matching non-autoregressive models, with masked non-autoregressive models acting more competitively.
• Duration Control: Explicit manipulation of duration parameters significantly increases DS-WED, highlighting duration as a key prosodic determinant.
• Reinforcement Learning: Direct Preference Optimization (DPO) and similar RL-driven improvements for intelligibility are observed to reduce prosody diversity as measured by DS-WED.

Such findings provide insight into how training and model design choices can shape prosodic variability in synthetic speech.

6. Limitations and Scope for Extension

The utility of DS-WED currently rests on several assumptions and design decisions:

• Language Restriction: The metric has been validated only for English speech data. Its applicability to other languages remains an open research question.
• Token Notation and Weighting: While current tokenization protocols and operation weights yield high correlation with PMOS, there is scope for further refinement, potentially including perceptually grounded weighting schemes or alternative clustering protocols.
• Granularity: DS-WED, by design, is less sensitive to low-level acoustic artifacts, focusing on higher-level prosodic variation—a trade-off advantageous for prosody assessment but less so when measuring very fine acoustic distinctions.

Future work will likely address these by extending DS-WED cross-linguistically and by refining the representation-learning and weighting components to better align with language-specific perceptual phenomena (Yang et al., 24 Sep 2025).

7. Practical Implications and Impact

DS-WED provides a scalable, perceptually aligned, and computationally efficient means to benchmark and develop TTS systems for expressiveness and naturalness in prosody. It enables:

• Reliable objective evaluation of prosody diversity aligned with human subjective experience.
• Diagnostic insight into the effects of model architecture and training regimens.
• Large-scale, GPU-friendly assessment pipelines for research and industrial deployment.

The metric offers a principled alternative or complement to classical acoustic metrics and underpins a standardized approach for future TTS system development targeting prosodic expressiveness.

---

Summary Table: Methodological and Empirical Features of DS-WED

Aspect	Description	Reference
Signal Processing	VAD silence trimming and tokenization via SSL	(Yang et al., 24 Sep 2025)
Core Metric	Weighted Levenshtein distance over token sequences	(Yang et al., 24 Sep 2025)
Weight Calibration	Empirically tuned, substitutions > insert/del	(Yang et al., 24 Sep 2025)
Correlation w/ Human	Pearson: DS-WED 0.77 vs. log F₀ RMSE 0.30	(Yang et al., 24 Sep 2025)
Computational Profile	RTF = 0.110, GPU-friendly, scalable	(Yang et al., 24 Sep 2025)
Empirical Robustness	Stable across SSL models, clustering params, data	(Yang et al., 24 Sep 2025)

DS-WED is establishing itself as a leading metric for quantifying prosody diversity in zero-shot TTS and beyond, offering both rigor aligned with modern representation learning and practical effectiveness rooted in human-perceptual correlation.