---
title: 'FireRedASR2: Scalable Multilingual ASR Module'
url: https://www.emergentmind.com/topics/fireredasr2
type: topic
---

# FireRedASR2: Scalable Multilingual ASR Module

FireRedASR2 is the automatic speech recognition module within the FireRedASR2S all-in-one speech system, which also includes Voice Activity Detection, Spoken Language Identification, and punctuation prediction. It is presented as a second-generation upgrade of FireRedASR, intended to improve recognition accuracy and broaden coverage across Mandarin, Chinese dialects and accents, English, code-switching, and singing transcription, while adding timestamp support in the AED variant. FireRedASR2 contains two ASR variants—FireRedASR2-LLM and FireRedASR2-AED—and the paper emphasizes that its core model designs remain largely intact relative to FireRedASR; the principal changes are scaling supervised training data, updating the AED vocabulary, and introducing a timestamping pipeline [2603.10420].

## 1. System role and model variants

FireRedASR2 is the ASR component of FireRedASR2S, a modular end-to-end pipeline that integrates ASR, VAD, LID, and punctuation prediction. Within that pipeline, FireRedASR2 transcribes speech segments and returns an ASR confidence score; FireRedASR2-AED additionally supports token-level and word-level timestamps [2603.10420].

The module is organized into two variants with distinct scale and deployment trade-offs. FireRedASR2-LLM is an Encoder-Adapter-LLM ASR model that uses a pretrained text LLM for autoregressive transcript generation. FireRedASR2-AED is a conventional Attention-based Encoder-Decoder model. The reported parameter scales are **8B+ parameters** for FireRedASR2-LLM and **1B+ parameters** for FireRedASR2-AED. The paper characterizes the LLM variant as the stronger accuracy model and the AED variant as the more compact and practical deployment option [2603.10420].

This bifurcated design continues the structure of the earlier FireRedASR family, which also separated a larger Encoder-Adapter-LLM model from a smaller AED model intended to balance performance and efficiency [2501.14350]. A plausible implication is that FireRedASR2 should be understood less as a new model family with a radically different decomposition than as a scaled and functionally extended continuation of that earlier line.

## 2. Architectural organization

FireRedASR2-AED follows the same general structure as FireRedASR-AED. Its encoder is Conformer-based, with convolutional subsampling followed by stacked Conformer blocks; its decoder is a Transformer decoder attending to encoder states; its training objective is standard token-level cross-entropy; and its input representation is **80-dimensional log Mel filterbank features**, extracted with **25 ms windows** and **10 ms frame shifts**, followed by CMVN. The paper states that its architecture and training recipe mostly follow **FireRedASR-AED model size L** from the earlier report, unless otherwise noted [2603.10420].

FireRedASR2-LLM uses the same acoustic front end and encoder family, but replaces the decoder with an LLM-based text generator. Its stated components are a Conformer audio encoder, a lightweight adapter that maps encoder outputs into the embedding space of a pretrained LLM, and an autoregressive next-token predictor for transcription. The encoder of FireRedASR2-LLM is initialized from the pretrained encoder of FireRedASR2-AED, and its architecture and training strategy otherwise remain the same as FireRedASR-LLM [2603.10420].

The architectural continuity with FireRedASR is explicit. In the earlier FireRedASR report, the LLM variant is described as an Encoder-Adapter-LLM system and the AED variant as a Conformer-plus-Transformer seq2seq model [2501.14350]. For FireRedASR2, the principal novelty is therefore not a change in backbone family. The paper instead frames the advance as data scaling, broader multilingual and dialectal coverage, and timestamp support for AED.

## 3. Training scale, tokenization, and supported transcription settings

The most important update from FireRedASR to FireRedASR2 is supervised data scaling. FireRedASR used about **70k hours** of supervised data, whereas FireRedASR2 uses about **200k hours**. The expanded corpus covers Mandarin, English, Chinese dialects and accents, code-switching, speech and singing, and non-speech audio. The paper explicitly attributes improved generalization and recognition accuracy mainly to this larger and more diverse supervised dataset [2603.10420].

For FireRedASR2-AED, tokenization is mixed: Chinese is represented by Chinese characters and English by BPE tokens. The AED vocabulary is updated from FireRedASR’s **7,832** to **8,667** in FireRedASR2-AED, with the stated purpose of better covering multilingual and dialect scenarios [2603.10420].

FireRedASR2 is evaluated on **24 public test sets** and supports the following transcription settings: Mandarin, Chinese dialects and accents, English, code-switching, and singing lyrics transcription. The benchmark suite includes the Mandarin sets **AISHELL-1**, **AISHELL-2 iOS**, **WenetSpeech Internet**, and **WenetSpeech Meeting**; dialect sets including **KeSpeech**, **WenetSpeech-Yue**, **WenetSpeech-Chuan**, and **MagicData dialect sets**; and the singing benchmark **opencpop**. The paper further notes that the training data does **not** include MagicData dialect/accent speech, making those evaluation sets a clean generalization test [2603.10420].

This distribution of training and evaluation material suggests that FireRedASR2 is intended not merely as a Mandarin benchmark model but as a broad Chinese speech recognition system with multilingual and paralinguistic breadth. That interpretation is consistent with the earlier FireRedASR report, which already emphasized dialect, English, and singing performance beyond core Mandarin ASR [2501.14350].

## 4. Output interfaces: confidence scoring and timestamp generation

FireRedASR2-AED produces an utterance-level confidence score from decoder probabilities. The paper describes the procedure as follows: take the per-token posterior probabilities along the 1-best beam-search hypothesis, exclude special tokens, and aggregate token probabilities, typically using the **geometric mean** of the valid tokens. It further notes that, in practice, the raw score may be post-processed with heuristics such as outlier filtering or clipping. The intended uses are downstream ranking, filtering, or UI display [2603.10420].

A key new capability in FireRedASR2 is timestamp support for the AED variant. The implementation adds a **post-hoc CTC branch** with a four-step training procedure: train the base AED model normally; freeze encoder and decoder; attach a lightweight linear CTC projection head on top of encoder outputs; and train only the CTC branch using the standard CTC objective. The paper states that this design preserves recognition accuracy while enabling alignment-based timestamps [2603.10420].

At inference time, timestamp generation proceeds in a separate alignment stage. The system first decodes the token sequence with AED beam search, then computes frame-level CTC logits, performs CTC forced alignment between the CTC logits and the AED-decoded token sequence, and converts frame alignment into token start and end times using the encoder subsampling rate. Word-level timestamps are derived by post-processing token timestamps: English subwords are merged into words using min start and max end, whereas Chinese characters are treated as word units [2603.10420].

The timestamping mechanism is notable because the paper presents it as an additive capability rather than a redesign of the recognizer. A common misunderstanding would be to treat FireRedASR2 as chiefly a new alignment architecture; the paper instead describes timestamp support as a post-hoc extension layered onto an otherwise largely preserved AED backbone.

## 5. Evaluation protocol and benchmark results

For ASR evaluation, the paper uses **CER (\%)** for Chinese and reports macro-averages across test sets with equal weight per set. The aggregate metrics are **Avg-All-24**, defined as average CER over all 24 test sets; **Avg-Mandarin-4**, average CER over 4 Mandarin test sets; **Avg-Dialect-19**, average CER over 19 Chinese dialect test sets; and **Sing-1**, CER on **opencpop**. Lower CER is better [2603.10420].

The reported aggregate results are as follows:

| Model | Avg-All-24 | Avg-Mandarin-4 | Avg-Dialect-19 |
|---|---:|---:|---:|
| FireRedASR2-LLM | **9.67** | **2.89** | **11.55** |
| FireRedASR2-AED | 9.80 | 3.05 | 11.67 |
| Doubao-ASR | 12.98 | 3.69 | 15.39 |
| Qwen3-ASR | 10.12 | 3.76 | 11.85 |
| Fun-ASR | 10.92 | 4.16 | 12.76 |

For the singing benchmark, the paper reports **1.12% CER** for FireRedASR2-LLM, **1.17% CER** for FireRedASR2-AED, **4.36%** for Doubao-ASR, **2.57%** for Qwen3-ASR, and **3.05%** for Fun-ASR [2603.10420].

Selected per-set results illustrate the relation between the two FireRedASR2 variants. On **aishell1**, the paper reports **0.64** for FireRedASR2-LLM and **0.57** for FireRedASR2-AED; on **aishell2**, **2.15** and **2.51**; on **ws-net**, **4.44** and **4.57**; and on **ws-meeting**, **4.32** and **4.53**. In the appendix, notable dialect results include **Kespeech** at **3.08** and **3.60**, **ws-yue-short** at **5.14** and **5.15**, **ws-yue-long** at **8.71** and **8.54**, **ws-chuan-easy** at **10.90** and **10.60**, **ws-chuan-hard** at **20.71** and **21.35**, **md-shanghai-conv** at **28.70** and **27.02**, **md-wuhan** at **9.60** and **10.81**, and **opencpop** at **1.12** and **1.17**, for FireRedASR2-LLM and FireRedASR2-AED respectively [2603.10420].

The paper’s stated interpretation is that FireRedASR2-LLM is the best overall performer across the aggregated metrics, while FireRedASR2-AED offers a strong smaller-footprint alternative. Both outperform the compared commercial and open baselines on the main aggregate scores, especially on Mandarin and singing transcription [2603.10420].

## 6. Lineage, distinctions from FireRedASR, and related robustness questions

The relation between FireRedASR2 and FireRedASR is described succinctly in the paper. The main updates are: training data grows from **~70k hours** to **~200k hours**; AED vocabulary grows from **7,832** to **8,667**; timestamps were not supported in FireRedASR-AED but are supported in FireRedASR2-AED; and core architectures are largely preserved. The paper therefore presents FireRedASR2 not as a redesign from scratch but as a larger-data, broader-coverage, timestamp-enabled continuation of FireRedASR [2603.10420].

That characterization is important for situating the model historically. FireRedASR, published in 2025, introduced the broader family structure with FireRedASR-LLM and FireRedASR-AED and reported strong public Mandarin benchmark results, including **3.05% CER** Average-4 for FireRedASR-LLM and **3.18% CER** for FireRedASR-AED [2501.14350]. FireRedASR2 extends that line into a broader multilingual and dialectal setting, while preserving the family’s two-variant organization.

The appendix also introduces a qualification regarding dialect comparisons. It notes that Fun-ASR may benefit from proprietary training data on some dialect subsets, which could explain its advantage on certain MagicData dialects like Shanghai and Wuhan in some cases [2603.10420]. This is not a contradiction of the aggregate findings; rather, it indicates that per-subset superiority is not uniform across all dialect conditions.

In a wider research context, FireRedASR2’s emphasis on dialect and accent coverage connects it to work on domain and accent robustness in end-to-end ASR. For example, REDAT studies accent-invariant representation learning through domain adversarial training and reports up to **13% relative WER reduction** on unseen accents in an RNN-T setting [2012.07353], while work on replaceable internal language models and residual softmax addresses inference-time domain adaptation without ASR retraining [2302.08579]. FireRedASR2 does not present those methods as part of its own recipe, but its evaluation emphasis on dialects, accents, and generalization places it within the same broader problem area.

## 7. Technical significance

FireRedASR2 is technically distinguished by the conjunction of three stated properties: substantially increased supervised scale, expanded task and language coverage, and timestamp support in the AED branch. The paper’s summary isolates these points directly: **more training data**, **broader coverage**, and **better functionality** through post-hoc CTC alignment [2603.10420].

Within that framing, FireRedASR2-LLM serves as the accuracy-oriented model and FireRedASR2-AED as the deployment-oriented model. The two are not presented as competing conceptions of ASR, but as complementary realizations of the same system family. Because the paper explicitly attributes the main gains to the enlarged and more diverse supervised dataset rather than to a fundamental architectural revision, FireRedASR2 also functions as evidence for the continued effectiveness of scaling curated supervised speech data in industrial ASR [2603.10420].

A plausible implication is that FireRedASR2’s contribution lies as much in system engineering and corpus construction as in model design. The article’s own formulation supports that reading: the core architectures are largely preserved, while performance and coverage advance through larger supervised training, multilingual and dialectal expansion, and added timestamp functionality.

Source: https://www.emergentmind.com/topics/fireredasr2