Inverse Real-Time Factor (RTFx) in ASR

• Inverse RTFx is a dimensionless metric that quantifies ASR throughput by comparing audio duration to inference wall-clock time.
• Standardized reporting mandates consistent hardware and batch sizes to ensure fair and reproducible RTFx comparisons across diverse models.
• Experimental findings reveal that optimizing decoder complexity can significantly boost RTFx, enhancing throughput with modest trade-offs in accuracy.

Inverse real-time factor (RTFx) is a dimensionless metric quantifying inference efficiency in automatic speech recognition (ASR) and related sequence-to-sequence modeling workflows. RTFx expresses the number of seconds of audio that a model can transcribe per one second of wall-clock time under specified hardware conditions. RTFx has emerged as a standard throughput measure on recent reproducible ASR leaderboards and benchmarking efforts, supporting rigorous comparisons of accuracy-efficiency trade-offs across a diverse range of open-source and proprietary systems (Srivastav et al., 8 Oct 2025, Żelasko et al., 7 Mar 2025).

1. Formal Definition and Interpretation

RTFx is defined as: $$\mathrm{RTFx} = \frac{T_{\mathrm{audio}}}{T_{\mathrm{compute}}}$$ where $T_{\mathrm{audio}}$ denotes the duration (in seconds) of the input audio, and $T_{\mathrm{compute}}$ is the wall-clock time (in seconds) required for model inference. For batches of $N$ utterances, the aggregate form is: $$\mathrm{RTFx} = \frac{\sum_{i=1}^N T_{\text{audio},\,i}}{\sum_{i=1}^N T_{\text{transcribe},\,i}}$$ Larger RTFx values indicate that more audio can be processed per unit time, corresponding to lower latency and higher throughput. RTFx is the reciprocal of the traditional real-time factor (RTF), which reports the ratio $T_{\mathrm{compute}}/T_{\mathrm{audio}}$ (Żelasko et al., 7 Mar 2025).

2. Methodological Reporting and Standardization

Reproducibility and fairness in RTFx reporting require normalization of several experimental variables:

• Hardware Consistency: All models must be evaluated on identical hardware configurations, including GPU model, CUDA drivers, and associated software libraries. The Open ASR Leaderboard mandates the use of an NVIDIA A100-SXM4-80 GB GPU (driver 560.28.03, CUDA 12.6) (Srivastav et al., 8 Oct 2025).
• Batch Size Adaptation: To consistently saturate compute resources, a default batch size (often 64) is chosen unless memory constraints necessitate automatic reduction (e.g., 48, 32, 16). This adaptive approach maximizes throughput while ensuring the validity of time measurements for all model scales.
• Isolation of Inference Timing: Only the inference compute time, from audio input to final transcript output, is measured. Overheads such as dataset loading, text normalization, and post-processing are excluded to purely capture model speed.

Comparability of RTFx thus hinges on rigorous control of these factors. Discrepancies in device, driver, decoding strategy, or even minor implementation differences can cause substantial variation in RTFx statistics (Żelasko et al., 7 Mar 2025).

3. Practical Implications and Trade-Offs

RTFx is central in quantifying the trade-off landscape in ASR. High-accuracy models employing deep Conformer encoders and LLM decoders achieve state-of-the-art word error rates (WER), with average WER ≈ 5.6% (short-form English), but typically exhibit RTFx values in the low hundreds (e.g., 145–418). In contrast, models with CTC-based or Token-and-Duration Transducer (TDT) decoders—such as NVIDIA Parakeet TDT 0.6B (RTFx ≈ 3386) and Parakeet CTC 1.1B (RTFx ≈ 2728)—provide over an order of magnitude higher throughput while incurring a modest cost in WER (≈ 6.0–7.4%) (Srivastav et al., 8 Oct 2025).

In long-form transcription, the speed discrepancy widens. For example, OpenAI Whisper Large v3 achieves an RTFx ~68.6 (WER 6.43%), whereas Parakeet CTC 1.1B delivers RTFx ~2793.8 (WER 6.68%). This delineates the suitability of lightweight decoders for applications prioritizing aggregate throughput, such as batch or offline transcription.

4. Experimental Findings in Model Architecture

Detailed profiling reveals that inference bottlenecks are typically associated with decoder complexity. In the encoder-decoder Canary-1B ASR model family, reducing decoder depth from 24 to 4 layers (while maintaining a 24-layer encoder) produces a 3.2× RTFx increase (from 345 to 1,097) on a single NVIDIA RTX 6000 Ada GPU. When these “freed” parameters are reassigned to deepen the encoder (Canary-1B-Flash: 32 encoder + 4 decoder layers), the model achieves an RTFx of 992—still 2.9× faster than the baseline while restoring parameter count and, by extension, model capacity (Żelasko et al., 7 Mar 2025).

This efficiency gain is attributed to the fact that decoder steps, especially in autoregressive architectures, dominate compute time (each token requires a full forward pass). Shrinking the decoder while maintaining or augmenting the encoder sharply boosts RTFx without significant loss in recognition accuracy or convergence properties.

Model Variant	Params (M)	RTFx (s_audio/s_wall)
Canary-1B (24 enc + 24 dec)	1,018	345
Canary-1B, small decoder (4 d)	680	1,097
Canary-1B-Flash (32 enc + 4 d)	882	992

5. Measurement Caveats and Limitations

Although RTFx provides a direct measure of raw model throughput, it is fundamentally hardware- and batch-size-dependent. Results are not inherently portable across different devices, driver versions, or implementations. Furthermore, RTFx does not capture inference memory footprint, streaming latency, or accuracy. For example, batch processing can inflate RTFx by amortizing fixed overhead, but may be inapplicable for latency-critical streaming use cases.

RTFx is also sensitive to decoding strategies; greedy decoding (single hypothesis) is faster than beam search, but may yield inferior accuracy. Reported RTFx values should always be interpreted within the context of the experimental protocol and hardware (Żelasko et al., 7 Mar 2025).

6. Reporting RTFx in Benchmarks and Leaderboards

The Open ASR Leaderboard prominently reports both WER and RTFx for each submitted model across multiple datasets, with sortable columns supporting practitioner-specific prioritization. By contrasting models using both accuracy and throughput metrics, the leaderboard enables transparent selection of architectures that best match operational requirements. Dedicated short-form and long-form tracks highlight the scaling behavior of RTFx with utterance length, decoder architecture, and batching strategy (Srivastav et al., 8 Oct 2025).

The standardization of RTFx reporting advances reproducibility and comparability in ASR research, addressing long-standing gaps in the evaluation of speech models for efficiency as well as accuracy.