H-PRM: Hotword Pre-Retrieval for ASR

• H-PRM is a module that reduces a large hotword inventory to a compact candidate set using phonetic matching and acoustic evidence.
• It employs diverse methods such as phoneme-to-phoneme similarity, attention-based filtering, and localized neural retrieval to optimize candidate selection.
• Empirical results indicate that integrating H-PRM improves recognition accuracy and minimizes degradation from distractor hotwords in ASR systems.

Searching arXiv for the cited papers and closely related hotword pre-retrieval work. arXiv search query: "H-PRM hotword pre-retrieval module speech recognition" Hotword Pre-Retrieval Module (H-PRM) denotes a front-end stage that reduces a large hotword inventory to a compact utterance-specific candidate set, or, in the always-on wake-word setting, acts as the lightweight gate that decides whether downstream speech processing should even start. The exact term appears in "H-PRM: A Pluggable Hotword Pre-Retrieval Module for Various Speech Recognition Systems" (Dai et al., 22 Aug 2025), but closely related mechanisms are described elsewhere as Attention Score Filtering (ASF) in SeACo-Paraformer (Shi et al., 2023), coarse CTC-based hotword filtering for prompt-conditioned LLM-ASR (Yang et al., 2024), phoneme-level retrieval-augmented generation in NIM4-ASR (Xie et al., 20 Apr 2026), CIF-localized retrieval in CLAR (Huang et al., 26 Mar 2026), and an always-on hotword gate in EfficientWord-Net that the paper text explicitly maps to a Hotword Pre-Retrieval Module (R et al., 2021). Across these works, H-PRM is best understood as a systems abstraction rather than a single algorithm: it is the stage that screens, ranks, or verifies hotword evidence before the final recognizer commits to contextual biasing or decoding.

1. Scope and conceptual boundaries

The literature describes two closely related functions under the H-PRM umbrella. One is inventory reduction: given a large incoming hotword list, the module retrieves only the most relevant candidates for the current utterance before downstream contextual ASR. This is the explicit role of H-PRM in the pluggable 2025 system, where a large hotword list is screened before SeACo-Paraformer or Audio LLM prompting (Dai et al., 22 Aug 2025). The other is front-end gating: a lightweight detector continuously monitors audio and triggers downstream processing only when the target hotword is present, as in EfficientWord-Net and earlier wake-word detectors (R et al., 2021).

This distinction matters because not every system implements H-PRM as an external retrieval engine. SeACo-Paraformer does not name such a module, but its ASF mechanism is explicitly characterized as the closest corresponding component: it first runs the bias decoder over the full hotword list, derives an attention score matrix $\mathbf{A} \in R^{L \times n}$, sums attention over output steps, and keeps the most active $k$ hotwords for the final effective inference pass (Shi et al., 2023). By contrast, Index-ASR does not describe a dedicated retrieval or indexing subsystem at all; its nearest equivalent is a prompt-side context provisioning mechanism that accepts user-defined cues such as hot word lists or paragraph-level summaries (Song et al., 31 Dec 2025). This suggests that “pre-retrieval” in ASR spans a continuum from explicit candidate search to model-internal pre-selection to externally supplied context prompts.

A further conceptual boundary separates H-PRM from ordinary contextual prompting. In the strong form, H-PRM is not merely a list serializer; it is a module that uses acoustic or acoustically grounded evidence to decide which hotwords should survive into the downstream context. That stronger interpretation is explicit in phoneme-based H-PRM (Dai et al., 22 Aug 2025), CTC-assisted filtering (Yang et al., 2024), NIM4-ASR’s phoneme-level RAG (Xie et al., 20 Apr 2026), and CLAR’s localized dual-encoder retrieval (Huang et al., 26 Mar 2026).

2. Architectural patterns

A canonical H-PRM pipeline appears in the pluggable H-PRM system. It has four stages: hotword preprocessing, initial ASR pass, H-PRM retrieval, and downstream hotword customization. The hotword list is normalized and tokenized into Pinyin or phonemes, their phonemic embeddings are stored in a hotword feature bank using the SAM-BERT TTS frontend, a Paraformer model first produces a recognition result for the speech segment, and H-PRM then scores every hotword against the utterance-side phonemic representation and returns the top-$N$ shortlist (Dai et al., 22 Aug 2025). The shortlist is passed either to SeACo-Paraformer or into an Audio LLM prompt.

A second pattern uses a cheap first-pass transcript as the retrieval query. In "CTC-Assisted LLM-Based Contextual ASR," the speech encoder is WavLM Large with a CTC branch; a coarse Viterbi-decoded sentence is produced, the top 5000 most frequent words in the LibriSpeech training corpus are removed, a character 2-gram index over the bias list retrieves preliminary candidates, and the final F3 algorithm matches every remaining word in the inference sentence with the most similar hotword by edit distance before inserting those hotwords into the prompt (Yang et al., 2024). This design is explicitly inference-time and heuristic rather than jointly trained.

A third pattern uses direct phoneme-database retrieval. NIM4-ASR inserts a phoneme-level CTC head and RAG module between the streaming speech encoder and the final LLM decoder. During inference, the phoneme head greedily decodes phoneme hypotheses, these are converted into index sequences, an Aho-Corasick automaton scans them against a datastore of hotword text $\leftrightarrow$ phoneme sequence key-value pairs, longest-match filtering removes nested matches, and the retrieved hotword texts are appended to the LLM prompt as contextual hints (Xie et al., 20 Apr 2026). The paper emphasizes million-scale hotword customization via RAG with sub-millisecond retrieval latency.

A fourth pattern is localized neural retrieval. CLAR uses a Paraformer speech encoder, a Chinese-RoBERTa-wwm-ext text encoder, modality-specific projection networks, and a CIF predictor that emits monotonic token boundaries without timestamps. Retrieval is then done over candidate-length localized windows rather than one global utterance vector (Huang et al., 26 Mar 2026). This architecture is designed to address short-entity retrieval in full-utterance speech, where global pooling causes representation dilution.

3. Retrieval signals, representations, and scoring

The most distinctive H-PRM designs are differentiated by what they treat as the retrieval signal. In the pluggable H-PRM system, retrieval is explicitly phoneme-to-phoneme. Both the Paraformer recognition result and each candidate hotword are converted into Pinyin or phoneme embeddings, a cosine similarity matrix is computed for each hotword–utterance pair, and a lightweight 5-layer CNN with channels $16, 32, 64, 128, 128$, followed by 2 fully connected layers and a 2-class output, produces a normalized similarity score used for ranking (Dai et al., 22 Aug 2025). The paper’s modality ablation is unusually direct: $p \rightarrow p$ dominates $t \rightarrow t$, $a \rightarrow a$, and $a \rightarrow t$, with $R@1 = 92.57$, $k$0, $k$1, and $k$2 on Aishell-dev, and $k$3, $k$4, $k$5, and $k$6 on Common-voice-zh (Dai et al., 22 Aug 2025). The paper’s stated interpretation is that hotword pre-retrieval is fundamentally a phonetic matching problem.

The same paper describes the similarity matrix in sequence-to-sequence form. If the ASR-result phonemic embedding sequence is

$k$7

and the hotword phonemic embedding sequence is

$k$8

then the similarity matrix $k$9 is

$N$0

The final shortlist is

$N$1

The paper emphasizes the visual intuition of a clear diagonal bright line for true phonetic alignment and a disordered pattern for unrelated hotwords (Dai et al., 22 Aug 2025).

CLAR adopts a different, localized score. Speech-side projected embeddings are

$N$2

text-side candidate embeddings are

$N$3

and the frame-candidate similarity matrix is

$N$4

CIF yields token spans $N$5. For a candidate hotword of token length $N$6, CLAR scores every valid token window

$N$7

then uses

$N$8

as the retrieval score (Huang et al., 26 Mar 2026). This is a length-aware localized matching rule rather than a global embedding similarity.

Other systems use different signals. SeACo-Paraformer’s ASF ranks hotwords by cumulative last-layer cross-attention from bias decoder states to hotword encodings rather than by explicit phonetic similarity (Shi et al., 2023). NIM4-ASR rejects dense ANN retrieval entirely and uses exact phoneme-sequence matches in a trie with Aho-Corasick failure links, followed by longest-match filtering; the paper explicitly prefers a hard-matching strategy because retrieval misses are often less harmful than retrieval errors (Xie et al., 20 Apr 2026). CTC-assisted contextual ASR uses character 2-gram overlap plus edit distance, with the final F3 rule matching each rare word in the coarse transcript to its nearest hotword (Yang et al., 2024). Taken together, these systems show that H-PRM is not tied to one retrieval formalism: it may be pairwise phoneme-similarity classification, internal attention-salience filtering, exact phoneme automaton search, approximate string matching, or localized speech-text dual-encoder scoring.

4. Coupling to downstream ASR and Speech LLMs

In conventional contextual ASR, H-PRM typically sits before the biasing module. In the H-PRM paper, the insertion point for SeACo-Paraformer is explicit: speech enters Paraformer, Paraformer generates a preliminary recognition result, the result is converted to phonemic embeddings, H-PRM ranks all hotwords, and only the top-$N$9 retrieved hotwords are passed to SeACo’s hotword customization stage (Dai et al., 22 Aug 2025). SeACo’s own ASF uses $\leftrightarrow$0 in experiments and reruns the truly effective bias decoder with that filtered list (Shi et al., 2023).

In Audio LLMs, the module usually exports text strings, not graph weights or decoder-state constraints. The H-PRM paper gives two prompting cases. For Whisper-small, the prompt is a natural-language context listing entities; for Qwen2-Audio-Instruct, the prompt explicitly asks the model to perform Chinese ASR while focusing on a hotword list $\leftrightarrow$1 and to output the result in a specified JSON-like format (Dai et al., 22 Aug 2025). The paper’s central observation is that the best shortlist size depends strongly on the downstream model’s tolerance for distractors.

Prompt-based integration also appears in LLM-centric contextual ASR. "CTC-Assisted LLM-Based Contextual ASR" inserts the filtered list into the prompt

$\leftrightarrow$2

Speech embeddings and prompt embeddings are concatenated and fed to Vicuna 7B (Yang et al., 2024). NIM4-ASR places a static instruction prefix at the beginning of context, appends streaming speech embeddings incrementally, and injects retrieved hotwords from the RAG module as dynamic textual context at the end of the prompt; the LLM decoder then performs a single final decoding pass when voice activity detection detects end of speech (Xie et al., 20 Apr 2026). CLAR likewise retrieves top-ranked hotwords and prepends them as contextual prompts for the downstream Speech LLM, without shallow fusion or lattice rescoring (Huang et al., 26 Mar 2026). Index-ASR represents the weakest form of this interface: it supports context prompts, specifically hot word lists or paragraph-level summaries, but leaves retrieval itself outside the model description (Song et al., 31 Dec 2025).

A practical implication repeatedly visible in these systems is that H-PRM is as much an interface discipline as a retriever. The downstream recognizer must be able to consume a small bias list effectively; if it cannot, higher pre-recall can still degrade final recognition.

5. Empirical behavior, scaling, and operating points

The central empirical motivation for H-PRM is large-inventory degradation. The pluggable H-PRM paper reports a dedicated stress test in which the hotword list grows from 150 to 3,800 by adding irrelevant items. Without H-PRM, SeACo-Paraformer PRR falls from 94.6% to 71.9% and MER rises from 5.67% to 9.20%; with H-PRM, PRR remains around 93% and MER increases only slightly to 6.39%, while top-50 PrRR drops only from 99.0% to 97.0% (Dai et al., 22 Aug 2025). On Common-voice-zh, the same paper reports that Paraformer yields MER 9.44 and PRR 70.41, SeACo yields MER 9.45 and PRR 70.63, and H-PRM yields MER 6.57 and PRR 93.90 (Dai et al., 22 Aug 2025). The large-hotword Chinese set is therefore the clearest case where the pre-retrieval stage, rather than the downstream biasing architecture alone, drives the improvement.

The papers also show that best shortlist size is model-dependent rather than monotonic with retrieval recall. In the H-PRM paper, SeACo-Paraformer is best around top-50, Whisper-small around top-3, and Qwen2-Audio-Instruct around top-1; top-50 causes severe degradation for prompt-based Audio LLMs, especially Qwen2-Audio-Instruct, where Aishell-dev MER rises to 29.42 and Common-voice-zh MER to 25.88 (Dai et al., 22 Aug 2025). The paper’s own summary is that improving pre-recall alone is not enough; the shortlist must be matched to the downstream model’s ability to exploit context without being overwhelmed.

Related systems show the same scaling pattern. SeACo-Paraformer enlarges Test-Entity’s hotword list from 231 to 400, 800, 1200, 2000, and 4000 hotwords; all methods degrade as the list expands, but only SeACo-Paraformer + ASF still achieves CER reduction when the list reaches 4000 hotwords, and with ASF it keeps its advantage over CLAS (Shi et al., 2023). NIM4-ASR reports city-specific retrieval databases comprising millions of location name–phoneme pairs, and states that the hotword database can easily scale to millions of entries while maintaining sub-millisecond retrieval latency per query (Xie et al., 20 Apr 2026). CTC-assisted contextual ASR reports that the model still performs well with 2000 biasing words and achieves $\leftrightarrow$3 WER/B-WER on test-clean and $\leftrightarrow$4 on test-other at bias-list size $\leftrightarrow$5 (Yang et al., 2024).

CLAR provides a different kind of evidence: the crucial ablation is not just inventory size but localization quality. With only the global contrastive loss, Test-AISHELL-1-NE retrieval is $\leftrightarrow$6, $\leftrightarrow$7, $\leftrightarrow$8. With local + CIF and no global loss, Test retrieval becomes $\leftrightarrow$9, $16, 32, 64, 128, 128$0, $16, 32, 64, 128, 128$1. With global + local + CIF, Test retrieval reaches $16, 32, 64, 128, 128$2, $16, 32, 64, 128, 128$3, $16, 32, 64, 128, 128$4 (Huang et al., 26 Mar 2026). On downstream contextual ASR, CLAR reduces Test-AISHELL-1-NE CER/B-WER from $16, 32, 64, 128, 128$5 in the vanilla Step-Audio2-mini system to $16, 32, 64, 128, 128$6 with the full candidate pool $16, 32, 64, 128, 128$7 (Huang et al., 26 Mar 2026). This result strongly supports the claim that H-PRM quality is not merely a candidate-management issue; it is also an alignment issue.

6. Misconceptions, limitations, and unresolved questions

A common misconception is that H-PRM must be a conventional retrieval engine. The surveyed systems show otherwise. ASF in SeACo-Paraformer is a model-internal neural pre-selection stage driven by cross-attention salience rather than an external index (Shi et al., 2023). EfficientWord-Net maps directly to H-PRM as a gate that continuously reads 1-second windows with a 0.25-second hop, embeds Log Mel spectrograms of shape $16, 32, 64, 128, 128$8, compares them with a pre-calculated hotword embedding by Euclidean distance, and triggers when $16, 32, 64, 128, 128$9 with $p \rightarrow p$0 (R et al., 2021). Index-ASR, conversely, provides only context prompts and contextual supervised fine-tuning data generation, not a retrieval subsystem at all (Song et al., 31 Dec 2025). The term therefore denotes a function, not a fixed implementation style.

A second misconception is that better retrieval recall always improves final ASR. Multiple papers directly contradict this. The H-PRM paper shows that larger top-$p \rightarrow p$1 often hurts prompt-based models even when pre-recall is higher (Dai et al., 22 Aug 2025). CTC-assisted contextual ASR reaches its best reported filtering variant with F3 rather than the earlier F1 or F2 schemes because removing common words and forcing per-rare-word coverage matter more than naive global ranking (Yang et al., 2024). NIM4-ASR explicitly chooses exact phoneme matching over approximate matching because retrieval precision under large-scale settings is more important than maximizing recall at all costs (Xie et al., 20 Apr 2026).

The present literature also leaves several production questions unresolved. The pluggable H-PRM system depends on a first-pass Paraformer output; if the initial ASR is badly wrong, retrieval quality may suffer, and the paper does not describe ANN acceleration, so per-utterance compute grows roughly linearly with the number of hotwords (Dai et al., 22 Aug 2025). CLAR gives strong retrieval gains but does not discuss very-large-scale ANN indexing or a thresholding policy for retrieval scores (Huang et al., 26 Mar 2026). EfficientWord-Net emphasizes one-shot enrollment without retraining, but its training data is homegrown artificially synthesized speech, and the paper does not specify posterior smoothing, debounce logic, hysteresis, or multi-template aggregation (R et al., 2021). Index-ASR demonstrates large gains from context prompting—average WER $p \rightarrow p$2 and average recall $p \rightarrow p$3—yet it still lacks an explicit retrieval/indexing algorithm, candidate ranking procedure, or large-hotword-set scaling analysis (Song et al., 31 Dec 2025).

These limitations clarify the current state of the field. H-PRM is already a practical design pattern for large-vocabulary hotword customization and always-on hotword gating, but the surveyed systems still expose the same open problems: calibration under distractor-heavy inventories, precision–recall trade-offs for prompt-conditioned LLMs, dependence on weak first-pass hypotheses, missing large-scale indexing strategies in some neural retrievers, and incomplete deployment details around streaming trigger logic, latency accounting, and error recovery. The broad design lesson that remains stable across the literature is narrower and more durable: hotword customization scales better when the recognizer is not asked to reason over the entire hotword inventory at once.