FireRedASR2: Scalable Multilingual ASR Module

• FireRedASR2 is a next-generation ASR system featuring two variants—LLM for high accuracy and AED for efficient deployment.
• It scales supervised training from 70k to 200k hours, broadening coverage to Mandarin, dialects, English, code-switching, and singing transcription.
• The AED variant introduces a post-hoc CTC timestamping extension, enabling precise word- and token-level alignment without altering core architecture.

FireRedASR2 is the automatic speech recognition module within the FireRedASR2S all-in-one speech system, which also includes Voice Activity Detection, Spoken Language Identification, and punctuation prediction. It is presented as a second-generation upgrade of FireRedASR, intended to improve recognition accuracy and broaden coverage across Mandarin, Chinese dialects and accents, English, code-switching, and singing transcription, while adding timestamp support in the AED variant. FireRedASR2 contains two ASR variants—FireRedASR2-LLM and FireRedASR2-AED—and the paper emphasizes that its core model designs remain largely intact relative to FireRedASR; the principal changes are scaling supervised training data, updating the AED vocabulary, and introducing a timestamping pipeline (Xu et al., 11 Mar 2026).

1. System role and model variants

FireRedASR2 is the ASR component of FireRedASR2S, a modular end-to-end pipeline that integrates ASR, VAD, LID, and punctuation prediction. Within that pipeline, FireRedASR2 transcribes speech segments and returns an ASR confidence score; FireRedASR2-AED additionally supports token-level and word-level timestamps (Xu et al., 11 Mar 2026).

The module is organized into two variants with distinct scale and deployment trade-offs. FireRedASR2-LLM is an Encoder-Adapter-LLM ASR model that uses a pretrained text LLM for autoregressive transcript generation. FireRedASR2-AED is a conventional Attention-based Encoder-Decoder model. The reported parameter scales are 8B+ parameters for FireRedASR2-LLM and 1B+ parameters for FireRedASR2-AED. The paper characterizes the LLM variant as the stronger accuracy model and the AED variant as the more compact and practical deployment option (Xu et al., 11 Mar 2026).

This bifurcated design continues the structure of the earlier FireRedASR family, which also separated a larger Encoder-Adapter-LLM model from a smaller AED model intended to balance performance and efficiency (Xu et al., 24 Jan 2025). A plausible implication is that FireRedASR2 should be understood less as a new model family with a radically different decomposition than as a scaled and functionally extended continuation of that earlier line.

2. Architectural organization

FireRedASR2-AED follows the same general structure as FireRedASR-AED. Its encoder is Conformer-based, with convolutional subsampling followed by stacked Conformer blocks; its decoder is a Transformer decoder attending to encoder states; its training objective is standard token-level cross-entropy; and its input representation is 80-dimensional log Mel filterbank features, extracted with 25 ms windows and 10 ms frame shifts, followed by CMVN. The paper states that its architecture and training recipe mostly follow FireRedASR-AED model size L from the earlier report, unless otherwise noted (Xu et al., 11 Mar 2026).

FireRedASR2-LLM uses the same acoustic front end and encoder family, but replaces the decoder with an LLM-based text generator. Its stated components are a Conformer audio encoder, a lightweight adapter that maps encoder outputs into the embedding space of a pretrained LLM, and an autoregressive next-token predictor for transcription. The encoder of FireRedASR2-LLM is initialized from the pretrained encoder of FireRedASR2-AED, and its architecture and training strategy otherwise remain the same as FireRedASR-LLM (Xu et al., 11 Mar 2026).

The architectural continuity with FireRedASR is explicit. In the earlier FireRedASR report, the LLM variant is described as an Encoder-Adapter-LLM system and the AED variant as a Conformer-plus-Transformer seq2seq model (Xu et al., 24 Jan 2025). For FireRedASR2, the principal novelty is therefore not a change in backbone family. The paper instead frames the advance as data scaling, broader multilingual and dialectal coverage, and timestamp support for AED.

3. Training scale, tokenization, and supported transcription settings

The most important update from FireRedASR to FireRedASR2 is supervised data scaling. FireRedASR used about 70k hours of supervised data, whereas FireRedASR2 uses about 200k hours. The expanded corpus covers Mandarin, English, Chinese dialects and accents, code-switching, speech and singing, and non-speech audio. The paper explicitly attributes improved generalization and recognition accuracy mainly to this larger and more diverse supervised dataset (Xu et al., 11 Mar 2026).

For FireRedASR2-AED, tokenization is mixed: Chinese is represented by Chinese characters and English by BPE tokens. The AED vocabulary is updated from FireRedASR’s 7,832 to 8,667 in FireRedASR2-AED, with the stated purpose of better covering multilingual and dialect scenarios (Xu et al., 11 Mar 2026).

FireRedASR2 is evaluated on 24 public test sets and supports the following transcription settings: Mandarin, Chinese dialects and accents, English, code-switching, and singing lyrics transcription. The benchmark suite includes the Mandarin sets AISHELL-1, AISHELL-2 iOS, WenetSpeech Internet, and WenetSpeech Meeting; dialect sets including KeSpeech, WenetSpeech-Yue, WenetSpeech-Chuan, and MagicData dialect sets; and the singing benchmark opencpop. The paper further notes that the training data does not include MagicData dialect/accent speech, making those evaluation sets a clean generalization test (Xu et al., 11 Mar 2026).

This distribution of training and evaluation material suggests that FireRedASR2 is intended not merely as a Mandarin benchmark model but as a broad Chinese speech recognition system with multilingual and paralinguistic breadth. That interpretation is consistent with the earlier FireRedASR report, which already emphasized dialect, English, and singing performance beyond core Mandarin ASR (Xu et al., 24 Jan 2025).

4. Output interfaces: confidence scoring and timestamp generation

FireRedASR2-AED produces an utterance-level confidence score from decoder probabilities. The paper describes the procedure as follows: take the per-token posterior probabilities along the 1-best beam-search hypothesis, exclude special tokens, and aggregate token probabilities, typically using the geometric mean of the valid tokens. It further notes that, in practice, the raw score may be post-processed with heuristics such as outlier filtering or clipping. The intended uses are downstream ranking, filtering, or UI display (Xu et al., 11 Mar 2026).

A key new capability in FireRedASR2 is timestamp support for the AED variant. The implementation adds a post-hoc CTC branch with a four-step training procedure: train the base AED model normally; freeze encoder and decoder; attach a lightweight linear CTC projection head on top of encoder outputs; and train only the CTC branch using the standard CTC objective. The paper states that this design preserves recognition accuracy while enabling alignment-based timestamps (Xu et al., 11 Mar 2026).

At inference time, timestamp generation proceeds in a separate alignment stage. The system first decodes the token sequence with AED beam search, then computes frame-level CTC logits, performs CTC forced alignment between the CTC logits and the AED-decoded token sequence, and converts frame alignment into token start and end times using the encoder subsampling rate. Word-level timestamps are derived by post-processing token timestamps: English subwords are merged into words using min start and max end, whereas Chinese characters are treated as word units (Xu et al., 11 Mar 2026).

The timestamping mechanism is notable because the paper presents it as an additive capability rather than a redesign of the recognizer. A common misunderstanding would be to treat FireRedASR2 as chiefly a new alignment architecture; the paper instead describes timestamp support as a post-hoc extension layered onto an otherwise largely preserved AED backbone.

5. Evaluation protocol and benchmark results

For ASR evaluation, the paper uses CER (\%) for Chinese and reports macro-averages across test sets with equal weight per set. The aggregate metrics are Avg-All-24, defined as average CER over all 24 test sets; Avg-Mandarin-4, average CER over 4 Mandarin test sets; Avg-Dialect-19, average CER over 19 Chinese dialect test sets; and Sing-1, CER on opencpop. Lower CER is better (Xu et al., 11 Mar 2026).

The reported aggregate results are as follows:

Model	Avg-All-24	Avg-Mandarin-4	Avg-Dialect-19
FireRedASR2-LLM	9.67	2.89	11.55
FireRedASR2-AED	9.80	3.05	11.67
Doubao-ASR	12.98	3.69	15.39
Qwen3-ASR	10.12	3.76	11.85
Fun-ASR	10.92	4.16	12.76

For the singing benchmark, the paper reports 1.12% CER for FireRedASR2-LLM, 1.17% CER for FireRedASR2-AED, 4.36% for Doubao-ASR, 2.57% for Qwen3-ASR, and 3.05% for Fun-ASR (Xu et al., 11 Mar 2026).

Selected per-set results illustrate the relation between the two FireRedASR2 variants. On aishell1, the paper reports 0.64 for FireRedASR2-LLM and 0.57 for FireRedASR2-AED; on aishell2, 2.15 and 2.51; on ws-net, 4.44 and 4.57; and on ws-meeting, 4.32 and 4.53. In the appendix, notable dialect results include Kespeech at 3.08 and 3.60, ws-yue-short at 5.14 and 5.15, ws-yue-long at 8.71 and 8.54, ws-chuan-easy at 10.90 and 10.60, ws-chuan-hard at 20.71 and 21.35, md-shanghai-conv at 28.70 and 27.02, md-wuhan at 9.60 and 10.81, and opencpop at 1.12 and 1.17, for FireRedASR2-LLM and FireRedASR2-AED respectively (Xu et al., 11 Mar 2026).

The paper’s stated interpretation is that FireRedASR2-LLM is the best overall performer across the aggregated metrics, while FireRedASR2-AED offers a strong smaller-footprint alternative. Both outperform the compared commercial and open baselines on the main aggregate scores, especially on Mandarin and singing transcription (Xu et al., 11 Mar 2026).

6. Lineage, distinctions from FireRedASR, and related robustness questions

The relation between FireRedASR2 and FireRedASR is described succinctly in the paper. The main updates are: training data grows from ~70k hours to ~200k hours; AED vocabulary grows from 7,832 to 8,667; timestamps were not supported in FireRedASR-AED but are supported in FireRedASR2-AED; and core architectures are largely preserved. The paper therefore presents FireRedASR2 not as a redesign from scratch but as a larger-data, broader-coverage, timestamp-enabled continuation of FireRedASR (Xu et al., 11 Mar 2026).

That characterization is important for situating the model historically. FireRedASR, published in 2025, introduced the broader family structure with FireRedASR-LLM and FireRedASR-AED and reported strong public Mandarin benchmark results, including 3.05% CER Average-4 for FireRedASR-LLM and 3.18% CER for FireRedASR-AED (Xu et al., 24 Jan 2025). FireRedASR2 extends that line into a broader multilingual and dialectal setting, while preserving the family’s two-variant organization.

The appendix also introduces a qualification regarding dialect comparisons. It notes that Fun-ASR may benefit from proprietary training data on some dialect subsets, which could explain its advantage on certain MagicData dialects like Shanghai and Wuhan in some cases (Xu et al., 11 Mar 2026). This is not a contradiction of the aggregate findings; rather, it indicates that per-subset superiority is not uniform across all dialect conditions.

In a wider research context, FireRedASR2’s emphasis on dialect and accent coverage connects it to work on domain and accent robustness in end-to-end ASR. For example, REDAT studies accent-invariant representation learning through domain adversarial training and reports up to 13% relative WER reduction on unseen accents in an RNN-T setting (Hu et al., 2020), while work on replaceable internal LLMs and residual softmax addresses inference-time domain adaptation without ASR retraining (Deng et al., 2023). FireRedASR2 does not present those methods as part of its own recipe, but its evaluation emphasis on dialects, accents, and generalization places it within the same broader problem area.

7. Technical significance

FireRedASR2 is technically distinguished by the conjunction of three stated properties: substantially increased supervised scale, expanded task and language coverage, and timestamp support in the AED branch. The paper’s summary isolates these points directly: more training data, broader coverage, and better functionality through post-hoc CTC alignment (Xu et al., 11 Mar 2026).

Within that framing, FireRedASR2-LLM serves as the accuracy-oriented model and FireRedASR2-AED as the deployment-oriented model. The two are not presented as competing conceptions of ASR, but as complementary realizations of the same system family. Because the paper explicitly attributes the main gains to the enlarged and more diverse supervised dataset rather than to a fundamental architectural revision, FireRedASR2 also functions as evidence for the continued effectiveness of scaling curated supervised speech data in industrial ASR (Xu et al., 11 Mar 2026).

A plausible implication is that FireRedASR2’s contribution lies as much in system engineering and corpus construction as in model design. The article’s own formulation supports that reading: the core architectures are largely preserved, while performance and coverage advance through larger supervised training, multilingual and dialectal expansion, and added timestamp functionality.