DSDR-Net: Dialect-Specialized Dynamic Routing

• DSDR-Net is a dialect-aware module that replaces the standard Transformer FFN with a composite of shared and dialect-specific private FFNs to model Tibetan dialect nuances.
• It leverages early dialect and speaker conditioning to preserve prosodic timing and phonotactic variation in multilingual text-to-speech synthesis.
• Empirical results demonstrate significant improvements in dialect classification accuracy and naturalness over baseline TTS models.

Dialect-Specialized Dynamic Routing Network (DSDR-Net) is a dialect-aware conditional computation module introduced for Tibetan multi-dialect text-to-speech synthesis, particularly in FMSD-TTS and TMD-TTS, to model fine-grained acoustic and linguistic variation across Ü-Tsang, Amdo, and Kham. In both formulations, DSDR-Net replaces the standard Transformer feed-forward network (FFN) with a composite block consisting of one shared public FFN and three dialect-specific private FFNs, with the active private branch selected deterministically by the dialect ID. The module is paired with early dialect conditioning, and in the few-shot multi-speaker setting also with speaker conditioning, to address the tendency of shared-parameter acoustic models to average across divergent prosody, phonotactics, segmental realizations, and speaker-dependent cues (Liu et al., 20 May 2025, Liu et al., 22 Sep 2025).

1. Research context and motivation

DSDR-Net emerged from a specific Tibetan speech modeling problem. Tibetan has three major dialects—Ü-Tsang, Amdo, and Kham—with substantial phonological, lexical, and syntactic divergence, limited mutual intelligibility, and limited parallel multi-dialect corpora. In the TMD-TTS formulation, existing parallel corpora are described as small, for example approximately 800 samples, which constrains applications such as speech-to-speech dialect conversion (S2SDC). In the FMSD-TTS formulation, the motivation is sharpened further by dialect-specific differences in tonogenesis and pitch, segmental inventory and allophony, and rhythm and timing. Ü-Tsang and Kham are described as tonal, with dialect-specific tone inventories and pitch-contour realizations, whereas Amdo is often treated as non-tonal or weakly tonal; consonant clusters, aspiration and voicing contrasts, vowel quality and length, coarticulation, and duration distributions also vary across dialects (Liu et al., 20 May 2025, Liu et al., 22 Sep 2025).

The immediate modeling problem is that a single shared TTS model often blurs dialectal idiosyncrasies because shared parameters must average across divergent acoustic and linguistic regularities. Earlier multi-dialect strategies that inject dialect information late, for example by generating a single mel-spectrogram followed by dialect-specific vocoders, are characterized as insufficient because dialect-specific vocoders can learn surface coloration but not dialect-specific prosody and timing embedded in the acoustic model. In few-shot multi-speaker synthesis, a further complication is entanglement between speaker identity and dialect identity. DSDR-Net addresses these issues by allocating dialect-specialized parameters inside the acoustic model while preserving a shared pathway for common structure (Liu et al., 20 May 2025, Liu et al., 22 Sep 2025).

The term “dynamic routing” in DSDR-Net should be understood in this narrower sense of conditional computation. This differs from the 2019 Dynamic Routing Networks framework, which performs instance-aware branch selection using RouterNets and Gumbel-Softmax for efficiency-oriented routing across candidate transformations (Cai et al., 2019).

2. Placement within TMD-TTS and FMSD-TTS

In TMD-TTS, Tibetan text is tokenized into character sequences with a 216-character vocabulary. The text encoder is a Transformer with multi-head self-attention, and the standard FFN in each encoder block is replaced by DSDR-Net. A discrete dialect label $\mathrm{did} \in \{0,1,2\}$ is embedded and fused into the encoder and the flow-prediction stages. Duration prediction is used for prosodic timing, and the acoustic model is built upon Matcha-TTS, predicting mel-spectrograms via a conditional flow-matching network. Waveform synthesis is handled by a pretrained BigVGAN vocoder; the paper reports BigVGAN at 16 kHz and 22 kHz in S2SDC validation. The dialect embedding has 128 dimensions, and the DSDR-Net hidden size is reported as 192 dimensions (Liu et al., 22 Sep 2025).

In FMSD-TTS, DSDR-Net occupies the same structural role inside a Transformer-based encoder, but the conditioning stack is augmented for few-shot multi-speaker transfer. Tibetan characters are again tokenized to a 216-symbol vocabulary. A short reference utterance is randomly cropped to 3 s, encoded by ECAPA-TDNN, and L2-normalized to form a speaker embedding. The dialect label is mapped to $\mathrm{did} \in \{0,1,2\}$ and embedded, then fused with the speaker embedding and injected into the text encoder and flow-prediction network. A duration predictor controls non-autoregressive upsampling, the acoustic model follows the Matcha-TTS style of duration-controlled generation, and a single shared BigVGAN vocoder converts mel-spectrograms to waveforms. The reported speaker embedding dimension is 192, the dialect embedding dimension is 128, the fusion output dimension is 128, and the full model size is 41.5M parameters (Liu et al., 20 May 2025).

Both systems therefore place DSDR-Net inside the acoustic model rather than in the vocoder. That architectural choice concentrates dialect modeling in the component that determines timing, prosodic structure, and mel-level acoustic realization, rather than postponing dialect specialization to waveform generation (Liu et al., 22 Sep 2025, Liu et al., 20 May 2025).

3. Formal routing mechanism

DSDR-Net operates on the hidden states produced by self-attention. Given text-side hidden states $h_{\mathrm{text}}$, multi-head self-attention computes

$$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$

where $Q$, $K$, and $V$ are learned projections of $h_{\mathrm{text}}$. The standard Transformer FFN is then replaced by a public-plus-private composition. In TMD-TTS, the private expert set is written as

$$\mathrm{FFN}_{\mathrm{private}} = \{\mathrm{FFN}_0,\mathrm{FFN}_1,\mathrm{FFN}_2\},$$

with the dialect-specific branch selected by $\mathrm{did}$. The final output is

$\mathrm{did} \in \{0,1,2\}$0

In FMSD-TTS, the same mechanism is written with an explicit one-hot gate $\mathrm{did} \in \{0,1,2\}$1, where $\mathrm{did} \in \{0,1,2\}$2 and the remaining entries are zero:

$\mathrm{did} \in \{0,1,2\}$3

Each expert FFN is a two-layer MLP with intermediate ReLU or GELU and reported dimension 192. The input and output shapes remain $\mathrm{did} \in \{0,1,2\}$4, so DSDR-Net is a drop-in replacement for the encoder FFN (Liu et al., 20 May 2025, Liu et al., 22 Sep 2025).

The routing rule is deterministic and label-conditioned. In TMD-TTS, the router is summarized as

$\mathrm{did} \in \{0,1,2\}$5

with binary coupling coefficients $\mathrm{did} \in \{0,1,2\}$6, $\mathrm{did} \in \{0,1,2\}$7, and no iterative refinement. This directly distinguishes DSDR-Net from capsule routing-by-agreement, which uses iterative updates of logits and coupling coefficients, and from soft mixture-of-experts systems, which use learned gating distributions over all experts. It also differs from the original Dynamic Routing Networks of 2019, which generate branch importance weights from RouterNets and recalibrate them with Gumbel-Softmax before top-$\mathrm{did} \in \{0,1,2\}$8 or thresholded selection at inference time (Cai et al., 2019, Liu et al., 20 May 2025, Liu et al., 22 Sep 2025).

A common misconception is therefore that DSDR-Net implements the same routing procedure as capsule networks, MoE gating, or DRNets. The Tibetan TTS formulation does not employ iterative routing updates, learned soft coupling coefficients, Gumbel-Softmax, RouterNets, or load-balancing losses. Its routing is hard, dialect-ID driven, and structurally simple. The model is still described as dynamic routing because the computation path depends on the input dialect (Liu et al., 20 May 2025, Liu et al., 22 Sep 2025).

4. Conditioning, specialization, and disentanglement

DSDR-Net is not an isolated expert switch; it is paired with explicit conditioning modules that make dialect information available before and after attention. In TMD-TTS, the Dialect Fusion Module computes a normalized dialect embedding and injects it into hidden states by residual addition:

$\mathrm{did} \in \{0,1,2\}$9

$h_{\mathrm{text}}$0

The same fusion is applied again in the flow-prediction network. The paper notes that embeddings are concatenated and projected, but the explicit formula is additive after a linear projection of the dialect embedding. The stated effect is that dialect information becomes accessible both to attention and to generation; the text describes this operationally as a residual adapter that shifts hidden states along dialect-specific directions (Liu et al., 22 Sep 2025).

In FMSD-TTS, this idea is generalized to joint speaker-dialect conditioning. A 3 s crop of reference audio is encoded by ECAPA-TDNN and normalized:

$h_{\mathrm{text}}$1

The dialect embedding is likewise normalized:

$h_{\mathrm{text}}$2

The two embeddings are concatenated, linearly projected, and added to the linguistic hidden states:

$h_{\mathrm{text}}$3

$h_{\mathrm{text}}$4

The additive residual form is described as preserving base linguistic content while superimposing style. L2 normalization is used to stabilize scales and discourage trivial magnitude-based dominance. FMSD-TTS also introduces a reference loss,

$h_{\mathrm{text}}$5

which penalizes positive cosine similarity and is intended to regularize speaker and dialect representations toward disentanglement. No adversarial discriminator is used for this purpose, and the system does not introduce explicit tone labels; prosody is learned implicitly in the acoustic model (Liu et al., 20 May 2025).

The public-plus-private decomposition in DSDR-Net is the mechanism by which this conditioning is translated into parameter specialization. The public FFN captures shared structure or language-common phonetics, while the private branch absorbs dialect-specific regularities. In the few-shot setting, this arrangement is explicitly motivated as a way to preserve speaker identity while switching dialectal manifolds (Liu et al., 20 May 2025, Liu et al., 22 Sep 2025).

5. Training protocol and empirical evidence

The two DSDR-Net instantiations are trained under related but not identical protocols. TMD-TTS reports a 179-hour multi-dialect Tibetan corpus consisting of 44 h Ü-Tsang, 45 h Kham, and 90 h Amdo from more than 1,500 speakers, with 40k training samples per dialect and 300 validation and 300 test samples per dialect. It is trained for 500k steps with Adam on two RTX 4090 GPUs, with dialect fusion and DSDR-Net trained jointly end-to-end. TMD-TTS does not enumerate the acoustic-model loss formulas, but states that the architecture is built upon Matcha-TTS and describes typical mel reconstruction, duration, and flow-matching objectives without reporting exact weights, schedules, or additional regularizers (Liu et al., 22 Sep 2025).

FMSD-TTS reports a real multi-dialect Tibetan corpus of approximately 210 h total and also lists Ü-Tsang 44 h, Kham 45 h, and Amdo 90 h, with 40,000 recordings per dialect for training and 300 per dialect for validation and test. It specifies the acoustic-model objective as

$h_{\mathrm{text}}$6

with

$h_{\mathrm{text}}$7

over 80-bin mel-spectrograms and

$h_{\mathrm{text}}$8

Training uses Adam with learning rate $h_{\mathrm{text}}$9, weight decay $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$0, batch size 16 per GPU, and 500,000 steps on an AMD Ryzen 9 5900X and two NVIDIA RTX 4090 GPUs. The BigVGAN vocoder is trained at 16 kHz for 100,000 steps with AdamW, $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$1, learning rate $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$2, decay 0.9999996, segment length 32 mel frames, and batch size 12 per GPU (Liu et al., 20 May 2025).

The empirical contribution of DSDR-Net is clearest in ablation. In TMD-TTS, removing DSDR-Net drops DCA from 80.25% to 60.12% and DECS from 78.3% to 58.6%; removing Dialect Fusion yields 74.15% DCA and 72.8% DECS; removing both gives 33.42% DCA and 32.2% DECS. In FMSD-TTS, removing DSDR-Net drops DCA from 80.25 to 60.12 and DECS from 0.80 to 0.58; removing dialect ID gives 74.15 DCA and 0.72 DECS; removing both gives 33.42 DCA and 0.32 DECS (Liu et al., 22 Sep 2025, Liu et al., 20 May 2025).

The following table aligns the reported ablations:

Variant	TMD-TTS	FMSD-TTS
Full model	DCA 80.25%, DECS 78.3%	DCA 80.25, DECS 0.80
w/o DSDR-Net	DCA 60.12%, DECS 58.6%	DCA 60.12, DECS 0.58
w/o fusion / dialect ID	DCA 74.15%, DECS 72.8%	DCA 74.15, DECS 0.72
w/o both	DCA 33.42%, DECS 32.2%	DCA 33.42, DECS 0.32

In the full TMD-TTS system, the reported per-dialect results are: Ü-Tsang, STOI 94.52%, PESQ 3.03, SI-SDR 17.91 dB, DNSMOS $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$3, DCA 67.41%, DECS 88.09%, RTF 0.032, nMOS 3.83, dMOC 76.64%; Amdo, STOI 94.92%, PESQ 3.13, SI-SDR 21.32 dB, DNSMOS $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$4, DCA 87.78%, DECS 79.17%, RTF 0.032, nMOS 3.84, dMOC 77.01%; Kham, STOI 93.17%, PESQ 3.05, SI-SDR 21.43 dB, DNSMOS $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$5, DCA 61.11%, DECS 67.65%, RTF 0.031, nMOS 3.86, dMOC 75.80%. The paper states that TMD-TTS outperforms SC-CNN, VITS2, and Matcha-TTS baselines across all three dialects, especially on PESQ and SI-SDR, and that dialect similarity improves strongly, with DECS up to 88.09% and DCA up to 87.78% (Liu et al., 22 Sep 2025).

In FMSD-TTS, the main comparisons are reported as follows. SC-CNN + multi-vocoder (64.1M) achieves DCA 40.74%, DECS 0.31, SECS 0.30, nMOS 2.82, sMOS 2.97, dMOC 65.14, and RTF $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$6; VITS2 + multi-vocoder (72.2M) achieves DCA 44.44%, DECS 0.37, SECS 0.41, nMOS 3.18, sMOS 3.52, dMOC 69.15, and RTF $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$7; Matcha-TTS + multi-vocoder (54.6M) achieves DCA 65.80%, DECS 0.65, SECS 0.37, nMOS 3.73, sMOS 3.03, dMOC 73.33, and RTF $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$8; FMSD-TTS (41.5M) achieves DCA 80.25%, DECS 0.80, SECS 0.56, nMOS 3.83, sMOS 3.57, dMOC 76.67, and RTF $$h_{\mathrm{attn}} = \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V,$$9 (Liu et al., 20 May 2025).

6. Downstream utility, efficiency, and open questions

Both DSDR-Net-based systems were validated through downstream speech-to-speech dialect conversion. In TMD-TTS, the synthesized TMDD dataset was used to train DurFlex-EVC with BigVGAN vocoding under four configurations. The reported MOS values are 3.07 for Baseline + BigVGAN 16k, 3.54 for Baseline + BigVGAN 22k, 3.23 for TMDD + BigVGAN 16k, and 3.63 for TMDD + BigVGAN 22k. In FMSD-TTS, a synthetic parallel multi-dialect corpus generated by the model was used with DurFlex-EVC and BigVGAN, yielding MOS 3.23 at 16 kHz and 3.63 at 22 kHz. In both studies, the conclusion is that dialect-faithful synthetic data improves downstream S2SDC naturalness (Liu et al., 22 Sep 2025, Liu et al., 20 May 2025).

From a computational perspective, TMD-TTS states that DSDR-Net adds one extra FFN evaluation per encoder block because the public FFN and exactly one private FFN are evaluated. Relative to a single FFN, this roughly doubles FFN compute, but it avoids evaluating all experts as in soft MoE. Reported inference RTF is approximately 0.031–0.032, slightly slower than VITS2 at approximately 0.020–0.021 yet still near real time. Memory increases because three private FFNs are added alongside one public FFN, but only one private branch is active per forward pass, so activation growth remains modest. Hard routing via $Q$0 is explicitly noted as preventing gate-collapse issues typical of soft MoE, while summing public and private FFNs regularizes specialization by maintaining a shared pathway. No load-balancing loss, expert dropout, or similar routing tricks are reported (Liu et al., 22 Sep 2025).

Reproducibility is partial rather than complete. TMD-TTS reports the 216-character vocabulary, 128-dimensional dialect embeddings, 192-dimensional DSDR-Net, 500k optimization steps with Adam, hardware consisting of two RTX 4090 GPUs, and release of TMDD with 98,142 utterances and 102 h total together with a comprehensive evaluation toolkit. It does not report detailed architecture depths, FFN expansion ratios and activations, learning-rate schedule, batch size, warmup, seeds, text normalization specifics, mel-spectrogram settings, or the TMD-TTS sampling rate, and it does not explicitly state that code is open-sourced. FMSD-TTS reports mel-spectrogram settings of 80 bins with STFT window 1024 and hop 256 at 16 kHz, public release of a large-scale synthetic Tibetan speech corpus and an open-source evaluation toolkit, but likewise does not provide a code repository in the text; licensing and links are not specified (Liu et al., 22 Sep 2025, Liu et al., 20 May 2025).

The main limitations are architectural scope and routing generalization. TMD-TTS notes that hard routing is limited to three known dialect IDs and does not learn soft dialect mixtures or generalize to unseen dialects without retraining a new expert. It also reports no explicit pitch or energy predictors, emphasizing duration as the sole reported prosodic control. FMSD-TTS adds that its evaluation is confined to non-autoregressive models, that the ECAPA-TDNN speaker encoder is not fine-tuned on Tibetan speakers because of limited labels, and that dialect conversion evaluation remains preliminary. Both lines of work identify future directions such as soft routing, capsule-style agreement, MoE with load balancing, explicit pitch and energy modeling, broader back-end alternatives including NaturalSpeech 2 and VITS-family systems, improved vocoders such as BigVSAN and Vocos, unsupervised dialect discovery, multilingual transfer, collection of Tibetan speaker-labeled data, and broader conversion baselines. FMSD-TTS also frames its released resources in the context of minority-language equity and cultural preservation (Liu et al., 22 Sep 2025, Liu et al., 20 May 2025).

DSDR-Net therefore occupies a specific place in contemporary speech synthesis research: it is a label-conditioned expert-routing mechanism embedded inside the acoustic model, designed not for generic compute-adaptive inference but for dialect specialization under low-resource constraints. Its empirical signature is consistent across the Tibetan TTS studies in which it appears: large improvements in dialect classification accuracy and dialect embedding consistency, with accompanying gains in naturalness, speaker similarity in the few-shot setting, and utility of the resulting synthetic corpora for downstream dialect conversion (Liu et al., 20 May 2025, Liu et al., 22 Sep 2025).