CTC-based Blank Filtering

• CTC-based blank filtering is a technique that exploits predominant blank labels in CTC outputs to minimize redundant computation in ASR and sequence modeling.
• The algorithms implement methods like blank collapse, frame skipping, and dynamic layer-skipping, which yield significant speed-ups and memory savings.
• Empirical results demonstrate up to 90% frame reduction with minimal impact on word error rates, validating its efficiency in practical applications.

Connectionist Temporal Classification (CTC)–based blank filtering refers to a family of algorithms and system-level interventions that exploit the characteristic “blank” structure in CTC output posteriors for computational and algorithmic efficiency. Central to these methods is the observation that most frames processed by CTC-trained models are assigned high-probability blank labels, corresponding to intervals between information-bearing (non-blank) acoustic events. By filtering or collapsing long runs of consecutive blanks before downstream processing (WFST decoding, neural transduction, retrieval augmentation, or knowledge distillation), redundant computation is minimized without compromising or even improving final accuracy. This approach underpins techniques such as blank-based frame skipping, dynamic layer-skipping, CTC-guided inference truncation, and blank-regularized knowledge transfer, which have empirically demonstrated multiplicative speed-ups and memory savings in state-of-the-art speech recognition and sequence modeling systems.

1. Mathematical Foundations of Blank Filtering in CTC

Let $X = (x_1,\ldots,x_T)$ be a sequence of $T$ input acoustic frames. A CTC model outputs a posterior matrix $P_{t,v}$ encoding, at step $t$, the probabilities over the symbol vocabulary $V$ including a dedicated blank label (typically $v=0$). The maximum a posteriori label at each frame is $C_t = \arg\max_v P_{t,v}$, partitioning the sequence into blank frames $\{\,t\mid C_t=0\,\}$ and non-blank frames $\{\,t\mid C_t\neq 0\,\}$ (Zhuang et al., 1 Jan 2026). Empirically, a large majority (often 70–90%) of frames in modern CTC-based ASR systems are blank-labeled, exhibiting long consecutive blank runs (Jung et al., 2022, Yang et al., 2023, Tian et al., 2021).

Operationally, CTC blank filtering works by discarding, collapsing, or bypassing these blank frames prior to subsequent computational stages. Specific thresholds $\theta$ on $T$0, run detection protocols, or blockwise aggregation are used to ensure semantic coverage is preserved while optimizing for minimal redundancy.

2. Algorithmic Variants: Filtering, Collapsing, and Skipping

Blank filtering methods are implemented in several algorithmic forms:

Blank Collapse (Frame Removal or Compression):

Frames with blank-posterior above a threshold $T$1 are collapsed, keeping only one representative per run, thus reducing the sequence length for WFST or beam search decoding. The blank-collapse function $T$2 returns only those frames not interior to a blank run exceeding $T$3. This reduces computational complexity from $T$4 to $T$5, where $T$6 is the fraction of collapsed blanks—commonly resulting in up to 78% faster decoding for high-quality models (Jung et al., 2022).

IOO/KOO Filtering for WFST-Decoding:

The “Keep-Only-One” (KOO) algorithm retains a single frame per consecutive block of identical non-blank labels, while “Insert-Only-One” (IOO) deterministically inserts one blank between non-blank segments, yielding a token/blank alternation that cuts decoding steps to $T$7 ($T$8) (Zhuang et al., 1 Jan 2026).

CTC-Guided Frame Skipping in Transducer Systems:

By co-training a CTC branch, encoder frames predicted as blanks can be skipped during RNN-T inference (or even within the encoder). Filtering is done at a confidence threshold $T$9, as $P_{t,v}$0 implies $P_{t,v}$1 can be discarded (Wang et al., 2022, Tian et al., 2021, Yang et al., 2023). Frame-skipping mechanisms may be applied before decoding (“decoder FR”), or within the encoder (“encoder FR”), directly reducing input length and computational load.

Dynamic Layer-Skipping:

For frames with high blank probability in intermediate encoder layers, subsequent layers may be skipped. A spike extension policy is typically used: only if $P_{t,v}$2 for $P_{t,v}$3 (ensuring non-blank boundary robustness) are later layers bypassed, with resulting representations immediately used in output calculation (Hou et al., 2024).

Skip-Blank in Retrieval-Augmented Inference:

During the construction and querying of frame-level retrieval datastores (e.g., kNN-CTC), blank frames are omitted. This avoids storing and searching large numbers of uninformative blank-labeled vectors, reducing memory and improving retrieval speed by $P_{t,v}$4–$P_{t,v}$5 with minimal effect on substitution error rates (Zhou et al., 2023).

3. Theoretical Limits, Complexity, and Trade-offs

The skip or collapse ratio is bounded by the transcript-to-frame length ratio: the theoretical maximum skip fraction is $P_{t,v}$6 where $P_{t,v}$7 is output token length (Yang et al., 2023). For example, on LibriSpeech with $P_{t,v}$8, $P_{t,v}$9. In practice, regularization schemes (penalizing non-blank self-loops via a “soft” loss, or imposing a hard cap $t$0 on repeated non-blanks) can closely approach this ceiling, with minimal word-error degradation up to $t$1 frame reduction (Yang et al., 2023).

Computational cost is dominated by the number of active frames. Standard beam search is $t$2 (with $t$3 the beam size and $t$4 active states), while blank filtering reduces runtime to $t$5 (where $t$6 is the filtered ratio) (Jung et al., 2022, Zhuang et al., 1 Jan 2026). For neural transducer models, encoder and joiner runtime can be reduced by $t$7–$t$8 with carefully-tuned blank filtering (Wang et al., 2022, Tian et al., 2021, Yang et al., 2023).

Trade-offs include:

• Accuracy loss: Aggressive blank filtering (low thresholds, tight non-blank selection) can miss semantic spikes or discard essential context, increasing WER. Moderate thresholds (e.g., $t$9) typically preserve accuracy (Zhuang et al., 1 Jan 2026, Yang et al., 2023).
• Error repair: Skipping blank frames impedes correction for errors involving deletion/insertion of blanks, especially in retrieval scenarios (Zhou et al., 2023).
• Blank boundary robustness: Policy extensions, such as spike-extending the skip decision window (Hou et al., 2024), or the IOO algorithm (Zhuang et al., 1 Jan 2026), mitigate under-segmentation near acoustic transitions.

4. Empirical Performance Across Modalities

Blank filtering and collapse methods are consistently effective:

System/Method	Task	Skip Ratio	Speedup	WER/CER Change	Reference
IOO+KOO WFST Decoding	LibriSpeech	>80%	2–3×	≈0 or slight gain	(Zhuang et al., 1 Jan 2026)
Blank Collapse	LibriSpeech	43–44%	1.75×	<0.01% abs loss	(Jung et al., 2022)
CTC-based Frame Skipping	RNN-T (Libri)	75%	2–4×	Unchanged	(Wang et al., 2022, Yang et al., 2023)
kNN-CTC Skip-Blank	ASR/dict. retr.	85–89%	6–7×	Minimal	(Zhou et al., 2023)
CTC Blank Dynamic Layer-Skip	LibriSpeech	27–53%	29–38%	+0.01–0.15% abs	(Hou et al., 2024)
FSR for Transformer-Transducer	AISHELL-1	≈90%	3.5–4×	≤1% abs	(Tian et al., 2021)

In all experiments, carefully chosen blank thresholds and transition penalties yield strong trade-offs between latency and recognition fidelity, with model-specific tuning recommended.

5. Applications Beyond Speech Recognition

CTC-based blank filtering is not limited to decoding acceleration. Blank-aware alignment and selection play a role in:

• Voice Activity Detection (VAD): Consecutive long blank runs in CTC outputs reliably demarcate non-speech regions, supporting streaming segmentation with explicit control over silence duration thresholds (Yoshimura et al., 2020).
• Knowledge Distillation: When distilling large CTC (“teacher”) models to smaller “student” models, naïve blank elimination can destabilize training, but symmetric blank selection (retaining boundary blank frames around non-blank spikes) enables label-free knowledge transfer with no WER degradation, even when the CTC loss is omitted (i.e., $V$0) (Hilmes et al., 2 Jun 2025).
• Retrieval-Augmented ASR: In kNN-CTC, storing only non-blank frame embeddings minimizes retrieval complexity and memory, at the expense of being unable to correct errors involving deletion/insertion of blanks (Zhou et al., 2023).

6. Hyperparameterization and Practical Guidelines

Key operational hyperparameters and recipes include:

• Blank probability threshold ($V$1 or $V$2): Empirically, $V$3 provides accurate and robust blank detection (Jung et al., 2022, Wang et al., 2022, Yang et al., 2023).
• Non-blank self-loop penalty ($V$4): Soft penalties in the CTC graph typically $V$5 allow up to 75% frame skipping with no adverse WER impact (Yang et al., 2023).
• Consecutive repeated token cap ($V$6): Hard restriction $V$7 is effective for strong speedups with minimal loss, $V$8 approaches theoretical skip ratio at some cost to WER (Yang et al., 2023).
• Spike extension window: For dynamic layer-skipping, a window of 3 frames before the skip decision helps maintain segmentation accuracy (Hou et al., 2024).
• Symmetric blank selection ($V$9): Retain $v=0$0 frames on each side of every non-blank spike for stable knowledge distillation (Hilmes et al., 2 Jun 2025).

A high-level recipe for blank-collapse (as an illustrative example):

$v=0$1 (Jung et al., 2022)

7. Limitations and Open Directions

CTC-based blank filtering, while widely validated, introduces system-specific tuning requirements and practical constraints:

• Over-skipping: Excessive blank filtering or aggressive thresholding degrades WER by missing boundary and weak token spikes (Zhuang et al., 1 Jan 2026, Yang et al., 2023).
• Error recovery: In retrieval and transduction tasks, blank skipping limits correction of insertion/deletion errors; only substitution errors are addressed efficiently (Zhou et al., 2023).
• Language/domain transfer: Most empirical work is on well-resourced, clean Mandarin and English datasets; the robustness of blank filtering techniques to low-resource or highly noisy domains is less well-characterized (Zhuang et al., 1 Jan 2026).
• Dynamic adaptation: Static thresholds may not account for utterance-level difficulty; research on adaptive skip policies (e.g., per-layer or per-utterance gates) is ongoing (Hou et al., 2024).
• Downstream end-to-end optimization: Jointly learning blank-aware compression and end task remains underexplored in the context of neural architectures beyond ASR—particularly where timing alignment is less rigidly structured.

In summary, CTC-based blank filtering encompasses a class of computationally principled, empirically validated methods for leveraging the alignment structure of blank frames in CTC models to improve the efficiency and scalability of sequence modeling, especially in speech recognition and related domains (Zhuang et al., 1 Jan 2026, Jung et al., 2022, Wang et al., 2022, Yang et al., 2023, Hou et al., 2024, Tian et al., 2021, Zhou et al., 2023, Hilmes et al., 2 Jun 2025, Yoshimura et al., 2020).