--- title: 'Qwen3-ASR-1.7B: Open-Source ASR Model' url: https://www.emergentmind.com/topics/qwen3-asr-1-7b type: topic --- # Qwen3-ASR-1.7B: Open-Source ASR Model Qwen3-ASR-1.7B is an open-source, large-scale automatic speech recognition (ASR) model developed as part of the Qwen3-ASR family, designed to perform end-to-end ASR and language identification (LID) across 52 languages and dialects. Leveraging the audio and language modeling (LALM) capabilities of its foundation model, Qwen3-Omni, Qwen3-ASR-1.7B realizes state-of-the-art results among open-source systems, offers robust streaming and offline inference, and provides unified architecture for speech understanding and transcription tasks [2601.21337]. ## 1. Model Architecture Qwen3-ASR-1.7B comprises a dual submodule structure: an AuT audio encoder (~300M parameters) and a Qwen3-1.7B decoder (~1.7B parameters). - **AuT audio encoder:** 12-layer Transformer encoder, $d_{model}=1024$, with 16 attention heads ($d_k=d_v=64$). The encoder processes 128-dimensional Fbank features, sampled at 12.5 Hz, and employs an 8× down-sampling scheme. It incorporates FlashAttention with dynamic window length (1–8 s) to support both streaming and offline inference. The feed-forward inner layer dimension is $d_{ff} \approx 4096$ with GELU activation. - **Qwen3-1.7B decoder:** 24-layer Transformer decoder (pretrained as part of Qwen3-Omni), $d_{model} \approx 1024$, 16 attention heads, and standard causal self-attention plus cross-attention to encoded representations. A learned projector aligns encoder and decoder feature spaces. All key computations in the architecture follow standard Transformer formulations: - Multi-head self-attention: $$ Q = X W^Q,\quad K = X W^K,\quad V = X W^V $$ $$ \mathrm{Attention}(Q, K, V) = \mathrm{softmax} \left(\frac{QK^T}{\sqrt{d_k}}\right) V $$ - Layer normalization: $$ \mathrm{LayerNorm}(x) = \gamma \frac{x - \mu(x)}{\sigma(x)} + \beta $$ - Feed-forward sublayer: $$ \mathrm{FFN}(x) = \mathrm{GELU}(x W_1 + b_1) W_2 + b_2 $$ - Cross-entropy loss for SFT: $$ \mathcal{L}_{\mathrm{CE}} = -\sum_{t=1}^T \sum_{v=1}^V \mathbf{1}(y_t = v) \log p_\theta(y_t = v \mid y_{" output schema. Data augmentation includes non-speech detection, streaming enhancement, and context biasing techniques. 4. **ASR Reinforcement Learning (RL):** Applies Group Sequence Policy Optimization (GSPO) on 50K utterances, stratified as 35% Chinese/English, 35% other languages, and 30% functional data. RL improves noise robustness, long-form transcription stability, and challenging scenario generalization. This multi-stage pipeline allows for both extensive language coverage and robustness across realistic ASR use cases. ## 3. Inference and Decoding Qwen3-ASR-1.7B utilizes autoregressive greedy decoding for next-token prediction without beam search. Language identification is integrated by prompting the model with the target language identifier, e.g., `<|im_start|>assistant language …`. For streaming inference, input is chunked into 2-second windows with a 5-token fallback, retaining state over the last 4 chunks. Efficiency metrics (test conditions: single A100 GPU, bfloat16, CUDA Graph, vLLM v0.14.0) are as follows: | Concurrency | Offline RTF | Offline Throughput (s/s) | Online TTFT_avg (ms) | Online RTF | Online Throughput (s/s) | |-------------|-------------|--------------------------|----------------------|------------|-------------------------| | 1 | 0.01482 | 67.48 | 102 | 0.01483 | 67.43 | | 8 | 0.02072 | 386.10 | 224 | 0.02000 | 400.00 | | 64 | 0.07360 | 869.57 | 1597 | 0.06208 | 1030.93 | | 128 | 0.13056 | 980.39 | 3392 | 0.10496 | 1219.51 | *RTF: Real-Time Factor. TTFT: Time To First Token.* This suggests the model scales efficiently with increased parallelization, though RTF and TTFT rise at high concurrency. ## 4. Evaluation and Benchmarking Qwen3-ASR-1.7B demonstrates competitive performance against commercial APIs (GPT-4o, Gemini-2.5) and open-source models (Whisper-lv3) across standard and internal benchmarks. **Open-source benchmarks (WER/CER):** | Dataset | GPT-4o | Gemini-2.5 | Whisper-lv3 | Qwen3-ASR-1.7B | |--------------------------|--------|------------|-------------|----------------| | LibriSpeech (clean) | 1.39 | 2.89 | 1.51 | 1.63 | | LibriSpeech (other) | 3.75 | 3.56 | 3.97 | 3.38 | | GigaSpeech (en) | 25.50 | 9.37 | 9.76 | 8.45 | | Fleurs-en | 2.40 | 2.94 | 4.08 | 3.35 | | WenetSpeech (net) | 15.30 | 14.43 | 9.86 | 4.97 | | AISHELL-2 | 4.24 | 11.62 | 5.06 | 2.71 | | Keystone (Cantonese) | 26.87 | 24.71 | 28.79 | 5.10 | | Wenet-Chuan (hard) | 43.79 | 67.30 | 26.80 | 21.63 | **Internal Robustness Suite (WER):** | Test | GPT-4o | Gemini2.5 | Whisper | Qwen3-0.6B | Qwen3-1.7B | |---------------------|--------|-----------|---------|------------|-------------| | Accented English | 28.56 | 23.85 | 21.30 | 16.62 | 16.07 | | Mandarin (elder) | 14.27 | 36.93 | 10.61 | 4.48 | 3.81 | | Mandarin (noise) | 36.11 | 29.06 | 63.17 | 17.88 | 16.17 | | Cantonese dialog | 16.05 | 14.98 | 31.04 | 4.80 | 4.12 | | 22-dialect mix | 45.37 | 47.70 | 44.55 | 18.24 | 15.94 | **Multilingual and LID performance:** - Average WER on Fleurs (12 languages): 4.90% - MLS (8 languages): 8.55% - CommonVoice (13 languages): 9.18% - Language identification accuracy (30 languages): 97.9% (Whisper-lv3: 94.1%). **Singing and Songs (WER):** | Benchmark | Doubao | FunASR | WhisperX | Qwen3-1.7B | |----------------|--------|--------|----------|------------| | M4Singer | 7.88 | 7.29 | 13.58 | 5.98 | | Opencpop | 3.80 | 2.98 | 9.52 | 3.08 | | PopCS | 8.97 | 9.42 | 13.77 | 8.52 | | EntireSongs-en | – | – | N/A | 14.60 | | EntireSongs-zh | – | – | N/A | 13.91 | **Streaming vs. Offline:** | Mode | Libri clean/other | Fleurs-en | Fleurs-zh | |----------|-------------------|-----------|-----------| | Offline | 1.63 / 3.38 | 3.35 | 2.41 | | Streaming| 1.95 / 4.51 | 4.02 | 2.84 | The model exhibits SOTA open-source ASR accuracy across Mandarin, Chinese dialects, and singing benchmarks and achieves competitive results with proprietary APIs on real-world, robustness, and low-resource tasks. ## 5. Strengths and Limitations **Strengths:** - State-of-the-art open-source ASR performance across English, Mandarin, regional dialects, and singing. - Competitive LID and transcription accuracy compared to leading proprietary APIs. - Unified streaming and offline inference implemented via dynamic FlashAttention. - Integrated prompt-based LID. - Open sourcing under Apache-2.0 facilitates broad adoption and further research. **Limitations:** - Inference RTF (0.015–0.13) at 1.7B scale may hinder use in extremely low-latency, on-device scenarios. - Multilingual performance degrades on long-tail languages in larger benchmarks (e.g., 30-language Fleurs). - Decoding does not currently expose beam search, which may limit performance on noisy or ambiguous inputs where n-best rescoring is critical. - Reproducing the training pipeline—particularly RL and large-scale pretraining—requires substantial computational resources and expertise. ## 6. Context and Impact Qwen3-ASR-1.7B places open-source ASR systems in direct competition with the most capable commercial APIs in both accuracy and real-world robustness. Its architecture exemplifies the trend toward LALM-based ASR, broad multilingualism, and strong integration with language identification. The model’s release—including code for inference and finetuning—under a permissive license substantially reduces barriers to deployment in production and research environments. A plausible implication is increased uptake in commercial, academic, and multilingually diverse ASR pipelines, especially where transparent licensing and custom model adaptation are priorities [2601.21337]. Source: https://www.emergentmind.com/topics/qwen3-asr-1-7b