Dolphin-CN-Dialect: Where Chinese Dialects Matter

Abstract: We present Dolphin-CN-Dialect, a streaming-capable ASR model with a focus on Chinese and dialect-rich scenarios. Compared to the previous version, Dolphin-CN-Dialect introduces substantial improvements in data processing, tokenization, training stability, and data sampling strategies. To address the challenges of highly imbalanced dialect data, we propose a temperature-based sampling strategy that effectively balances standard Mandarin and low-resource dialects, leading to significant gains in dialect recognition performance. In addition, we redesign the tokenizer to better align with linguistic characteristics, adopting character-level modeling for Chinese and subword modeling for English, while introducing extensible dialect tokens. Experimental results show that Dolphin-CN-Dialect achieves improvement in dialect recognition accuracy and CER reduction compared to Dolphin. Furthermore, Dolphin-CN-Dialect reaches competitive performance with recent SOTA open-source ASR models, while maintaining a significantly smaller model size. Dolphin-CN-Dialect supports both streaming and non-streaming inference, enabling a practical balance between latency and accuracy. It also provides flexible customization through hotword support and efficient deployment optimized for specialized hardware. These improvements make Dolphin-CN-Dialect a strong and practical solution for real-world multi-dialect ASR applications.

• The paper presents a novel ASR model using temperature-based sampling that achieves a 38% improvement in dialect recognition accuracy and a 16% overall CER reduction.
• It employs a hybrid tokenizer with a reduced vocabulary and 80 dialect-specific tokens, optimizing training efficiency and accommodating underrepresented dialects.
• The model integrates dual hotword biasing strategies—encoder-level and prompt-based—for a significant reduction in BWER, enhancing practical deployment.

Dolphin-CN-Dialect: A Data- and Deployment-Centric ASR Model for Chinese Dialect Robustness

Introduction

"Dolphin-CN-Dialect: Where Chinese Dialects Matter" (2605.08961) introduces Dolphin-CN-Dialect, an automatic speech recognition (ASR) model designed to address dialectal diversity and practical deployment requirements for Mandarin Chinese and its numerous regional dialects. The model builds upon the Dolphin-V1 framework, targeting major challenges in multi-dialect ASR: extreme long-tail data distribution, deployment constraints (including streaming inference and customization such as hotword biasing), and the need for tokenizer architectures aligned with linguistic characteristics.

Model Innovations

Temperature-Based Data Sampling

A pivotal contribution is the temperature-based sampling strategy, which modulates the training data distribution to counteract the dominance of standard Mandarin and boost dialect coverage. This approach introduces a temperature parameter $\alpha \in (0,1)$ to interpolate between natural (corpus-proportional) and uniform per-dialect sampling. With carefully tuned $\alpha$, the risk of overfitting low-resource dialects is mitigated while substantially increasing the model's exposure to underrepresented dialects. This results in a 38% improvement in dialect recognition accuracy and a 16% overall CER reduction compared to the previous Dolphin baseline, with minimal impact on Mandarin accuracy.

Hybrid Tokenizer and Extensible Dialect Tokens

The tokenizer is redesigned with a hybrid approach: character-level units for Chinese (to maintain CTC alignment and capture ideogrammatic structure) and BPE-based subword units for English, supplemented by special tokens representing tasks, dialects, timestamps, and regions. The vocabulary is reduced from 40K to 18,173 tokens, which improves training efficiency and stability. Furthermore, the tokenizer reserves slots for 80 dialect-specific tokens, facilitating seamless expansion to additional dialects and granular region modeling, a capability that is critical for practical ASR systems aimed at large linguistic landscapes.

Hotword-Biased Decoding

Dolphin-CN-Dialect integrates two hotword biasing strategies:

• Encoder-level contextual biasing: Hotword lists are embedded and fused with acoustic encoder outputs via a multi-head attention mechanism, making the encoder's representations context- (hotword-) aware. Only parameters of the biasing module are updated in fine-tuning, ensuring data efficiency.
• Prompt-based biasing: For attention-based decoders, hotwords are injected as prompts between special tokens during both training and inference. Robust hotword recognition is achieved via dynamic construction of hotword lists and a two-stage filtering mechanism to control noise and computational cost.

Both strategies show strong relative reductions in biased word error rate (BWER), particularly on long-tail and named-entity-heavy test sets.

Training Data and Pipeline

The training corpus comprises:

• Dataocean AI Chinese dataset: Extensive proprietary coverage of 22 dialects, multiple speaking styles, and challenging acoustic conditions.
• Open-source resources: KeSpeech (multi-dialect, annotated), WenetSpeech (large-scale Mandarin), and Common Voice (accented Mandarin/Taiwanese).

All data are standardized with metadata for dialect/language tags, timestamps, and tasks, supporting multitask objectives (e.g., punctuation prediction, timestamped and non-timestamped recognition).

Advanced data engineering was critical: the team implemented sharding, multi-process loading, and dataset bucketing to achieve high I/O throughput and mitigate training instability due to sequence-length outliers.

Empirical Results

Dialect Robustness and CER

Dolphin-CN-Dialect consistently achieves state-of-the-art or highly competitive CER among models with parameter counts below 1B—often outperforming substantially larger models on challenging dialects such as Wu (9.49% CER), Minnan, and heavily accented Mandarin subsets. On KeSpeech and Common Voice (tw) benchmarks, Dolphin-CN-Dialect-0.4B achieves 5.04% and 5.62% CER, respectively, surpassing all sub-1B ASR models.

The temperature-based data balancing yields strong dialect accuracy improvements while incurring only marginal degradation (~0.2% CER) on standard Mandarin.

Hotword Biasing

Both encoder-level and prompt-based hotword customization strategies yield significant reductions in BWER and overall WER under various deployment scenarios. Notably, in oracle settings where only ground-truth hotwords are provided as prompts, Dolphin-CN-Dialect achieves up to 96% relative reduction in BWER on AISHELL and 84.6% reduction on Common Voice, nearly eliminating hotword errors. This demonstrates the model's intrinsic ability to integrate external contextual knowledge efficiently.

Furthermore, side effects on non-hotword recognition (UWER degradation) are minor—often negligible or even slightly improved—demonstrating controlled biasing with robust general performance.

Deployment and System Adaptability

Dolphin-CN-Dialect supports both streaming and non-streaming inference, allowing practitioners to tune the latency/accuracy trade-off for their use case. The model integrates seamlessly with hotword customization pipelines and supports efficient deployment on specialized hardware. Engineering modifications (e.g., sequence length filtering, data pipeline parallelization) underpinned stable large-scale training and practical system integration.

Theoretical and Practical Implications

This work underscores key insights for robust, scalable, and deployable ASR in linguistically diverse contexts:

• Data-centric strategies, especially adaptive sampling, are at least as important as architectural change for handling extreme long-tail distribution in multi-dialect ASR.
• Explicit dialect modeling (dialect tokens, metadata-augmented training, extensible vocabulary design) is effective for both generalization and future system expansion to novel dialects or speech varieties, reducing brittleness to dialectal variation and unseen regional accents.
• Hybrid tokenization aligned to language typology (e.g., character-level for logographic languages, BPE for Indo-European) improves model robustness and compatibility with CTC-AED joint architectures.
• Controlled, prompt-based contextual adaptation (hotwords) is feasible at industrial scale, provided careful batch design and phrase filtering are used to mitigate noise and interference.

Overall, Dolphin-CN-Dialect operationalizes data-centric ASR system development and sets a new benchmark in maximizing accuracy/efficiency trade-offs in sub-1B parameter regimes.

Limitations and Future Directions

The model currently relies on manually tuned temperature parameters for data sampling, and evaluation is largely focused on Chinese and its dialects; broader Aboriginal or mixed-language coverage remains to be systematically validated. The interplay between streaming optimization, latency requirements, and further improvements in dialect/rare word performance is an open area. Adaptive and self-tuning sampling strategies, scalable to broader multilingual benchmarks, and even finer-grained region-specific dialectal adaptation are natural next steps.

Conclusion

Dolphin-CN-Dialect (2605.08961) demonstrates that principled, data-centric design and deployment-aware engineering can produce ASR systems that robustly and efficiently handle dialectal variation on par with, or superior to, much larger models. The effective balancing of language variety, adaptability to hotword biasing, and support for practical deployment solidify its position as an important reference for future ASR research and industrial adoption, particularly in linguistically diverse environments.