LOGIC: Logit-Space Integration for Biasing

• LOGIC is a decoding-time framework that integrates dynamic entity lists directly into token logits for contextual biasing.
• It uses a trie-based logit modification strategy with Immediate Prefix Boosting and Retroactive Score Rectification to reduce Entity WER across locales.
• LOGIC bypasses prompt stuffing and post-hoc correction by biasing generation in real time, ensuring efficient and accurate speech transcription.

Searching arXiv for the specific LOGIC acronym paper and a closely related context-biasing reference. Searching arXiv for "Logit-Space Integration for Contextual Biasing" and related speech contextual biasing work. LOGIC, short for Logit-Space Integration for Contextual Biasing, is a decoding-time framework for contextual biasing in speech LLMs. It is designed to improve recognition of user-specific and newly emerging entities such as contact names, playlist titles, technical terms, rare locations, and personalized vocabulary by injecting a runtime entity list directly into token logits during autoregressive decoding, rather than through prompt stuffing or post-hoc transcript rewriting (Wang, 21 Jan 2026). In the formulation presented for speech LLMs, LOGIC addresses the mismatch between static training knowledge and dynamic inference-time context, while preserving open-ended generation and keeping the contextualization mechanism outside the main input sequence.

1. Problem setting and motivation

The method is motivated by a persistent limitation of speech LLMs: they are strong at general conversational understanding, but their parameters reflect a fixed training distribution and therefore lag behind both public novelty and private user context. The practical consequence is degraded recognition of domain-specific and personalized entities even when the rest of an utterance is transcribed well. The paper treats this as a contextual biasing problem: at inference time, the system receives a list of currently relevant entities and should prefer them when the acoustics support them (Wang, 21 Jan 2026).

Two existing solution families are identified as unsatisfactory at scale. The first is prompt-based contextual biasing, in which the entity list is appended to the prompt. This is described as suffering from context-window limits, increased inference latency, the quadratic complexity of self-attention with respect to prompt length, and the “lost-in-the-middle” phenomenon. The paper also reports a failure mode called “context leakage” or “list-vomiting,” where a model begins reciting the bias list rather than following the acoustics. The second is Generative Error Correction (GEC), which rewrites a first-pass transcript using the entity list. That approach is characterized as prone to “over-correction” and hallucination, because it operates in text space rather than in the speech model’s own decoding distribution. The illustrative example is a spoken “I like algorithms” being rewritten as “I like Al Gore” when “Al Gore” appears in the entity list (Wang, 21 Jan 2026).

This framing places LOGIC in a narrower but more controlled design space. It does not attempt general symbolic reasoning over entities. Instead, it modifies next-token preferences at the decoding layer so that contextual entities become more likely only when they are locally consistent with the beam’s current token prefix.

2. Logit-space integration and decoding state

The central object is an entity set

$\mathbf{O} = \{E_1, \dots, E_N\}.$

Each entity is tokenized and inserted into a token-level prefix tree, $\mathcal{T}$. During beam-search decoding, each beam $b$ carries a trie state $s_{t-1}^{(b)}$, representing the currently matched entity prefix, together with an accumulated bias bonus $\Phi_t$. Let $\mathbf{z}_t \in \mathbb{R}^{|\mathcal{V}|}$ be the speech LLM’s base logits at step $t$. LOGIC constructs modified logits

$$\mathbf{z}'_t[k] = \mathbf{z}_t[k] + \mathbb{I}\!\big(k \in \mathrm{Children}(s_{t-1}^{(b)})\big)\cdot \lambda,$$

where $k$ is a vocabulary index, $\mathrm{Children}(s)$ is the set of valid next tokens from trie node $\mathcal{T}$0, and $\mathcal{T}$1 is the bias bonus (Wang, 21 Jan 2026).

The effect is multiplicative in probability space: valid continuation tokens are multiplied by $\mathcal{T}$2 before renormalization. The paper emphasizes that this is a soft bias, not a hard lexical constraint. A non-trie token can still win if the acoustic and language-model evidence is sufficiently strong. This is a crucial distinction from grammar-constrained decoding, because speech transcription remains open-ended.

The same update can be written as a sparse mask over the vocabulary: $\mathcal{T}$3 with

$\mathcal{T}$4

Because the intervention occurs after the speech LLM has already produced $\mathcal{T}$5, contextualization is decoupled from the model’s prompt processing. The abstract characterizes this as constant-time complexity relative to prompt length, and the detailed implementation refines the claim: the contextual update is implemented sparsely in $\mathcal{T}$6, where $\mathcal{T}$7 is the maximum trie out-degree, instead of a naïve $\mathcal{T}$8 full-vocabulary modification (Wang, 21 Jan 2026).

3. Trie construction, tokenization, and path management

A nontrivial complication arises from subword tokenization. The same surface form may admit multiple valid token sequences depending on whitespace or segmentation context. The paper gives “Alex” as an example, potentially tokenized as $\mathcal{T}$9, $b$0, or $b$1. LOGIC therefore uses multi-path tokenization: multiple valid tokenizations for each entity are inserted into the trie rather than assuming a single canonical sequence (Wang, 21 Jan 2026).

This design makes the trie tokenizer-aware and prevents contextual biasing from being tied to a brittle tokenization choice. The trie itself is a token-level prefix tree $b$2, whose leaves mark complete entity realizations. During decoding, if the selected token $b$3 continues the active prefix, the beam advances to

$b$4

and its accumulated bias updates to

$b$5

If a leaf is reached, the entity has been fully matched; the trie state resets to the root and $b$6 resets to $b$7, allowing later entities to be recognized in the same utterance (Wang, 21 Jan 2026).

The method also introduces Immediate Prefix Boosting (IPB). Rather than withholding bias until after the first token, LOGIC boosts entity-onset tokens directly from the root: $b$8 The rationale is that in speech LLMs with large subword vocabularies, the first token of a name is often highly informative, especially for short names and scripts where initial segments carry substantial discriminative content. This changes the recall profile of the system, particularly for short entities, but it also creates a risk of entering the wrong entity path too early (Wang, 21 Jan 2026).

4. Retroactive score rectification and implementation

To control false positives created by provisional prefix boosts, LOGIC adds Retroactive Score Rectification (RSR). The failure mode is described as a “sunk cost” effect: a beam may receive several early bonuses for matching the beginning of an entity, then diverge acoustically, yet retain an artificially inflated beam score. LOGIC tracks the cumulative bonus and revokes it if the partial entity is abandoned before completion: $b$9 Thus bonuses are provisional until a full entity leaf is reached (Wang, 21 Jan 2026).

The paper’s implementation uses vLLM with a custom LogitsProcessor, vectorized GPU execution, a sparse CUDA kernel, and zero-copy state management with the trie stored in shared memory. The sparse update is central to the efficiency claim. Since $s_{t-1}^{(b)}$0, the implementation updates only the current trie-child indices rather than all vocabulary logits. The resulting contextualization cost depends on local branching $s_{t-1}^{(b)}$1, typically under 100, not on the size of the full vocabulary and not on the prompt length that a prompt-based solution would require (Wang, 21 Jan 2026).

The single main runtime hyperparameter is the bias bonus $s_{t-1}^{(b)}$2. Two operating regimes are reported. A robust setting uses approximately $s_{t-1}^{(b)}$3 to $s_{t-1}^{(b)}$4, favoring low false alarms. An aggressive setting uses approximately $s_{t-1}^{(b)}$5 to $s_{t-1}^{(b)}$6, favoring entity recall. No learned calibration module, entity-frequency prior, or semantic retrieval score is added; all contextual influence is mediated through trie-consistent additive logit bonuses (Wang, 21 Jan 2026).

5. Empirical evaluation

The evaluation is conducted on an internally curated ASR test suite spanning 11 locales: $s_{t-1}^{(b)}$7 The task focuses on person names (PNAMEs) embedded in natural speech, with entity list sizes ranging from about 460 entities in de-DE to more than 2100 in it-IT. The abstract states that the experiments use Phi-4-MM, whereas the experimental description specifically says Phi-4-mini (Wang, 21 Jan 2026).

In the robust configuration, with IPB and RSR enabled, LOGIC achieves an average 9% relative reduction in Entity WER (EWER) with an average 0.30% increase in False Alarm Rate (FAR) across the 11 locales. The largest reported robust-setting relative EWER gains are 19% for fr-FR, 17% for es-MX, 14% for de-DE, and 11% for en-US. WER remains broadly stable, and entity recall improves across locales (Wang, 21 Jan 2026).

In the aggressive configuration, average relative EWER improvement rises to 17%. Reported aggressive gains include 29% for fr-FR, 28% for es-MX, 24% for de-DE, 21% for pt-BR, 20% for zh-CN, and 18% for en-US. The paper presents this as a recall–precision trade-off controlled by $s_{t-1}^{(b)}$8 (Wang, 21 Jan 2026).

A concise summary is:

Setting	Average relative EWER improvement	Additional note
Robust	9%	0.30% average FAR increase
Aggressive	17%	higher recall-oriented bias
Runtime	—	RTF 0.0990 $s_{t-1}^{(b)}$9 0.1018, a 2.8% overhead

Efficiency is reported on NVIDIA A100 GPUs with a phrase list of 2210 items. The Real-Time Factor changes from 0.0990 without biasing to 0.1018 with LOGIC, corresponding to a 2.8% overhead. The paper presents this as small enough for real-time deployment (Wang, 21 Jan 2026).

6. Interpretation, limitations, and place in the literature

LOGIC is best understood as a decoding-time contextual biasing mechanism rather than a prompting method, a transcript-rewriting method, or a symbolic reasoning system. Its logic is operational: entity prefixes are represented in a trie, and beam scores are altered only when candidate next tokens continue a contextual entity. This suggests a closer relationship to constrained or guided decoding than to open-ended entity reasoning.

Its principal strengths are efficiency, multilingual applicability, training-free deployment, and robustness against two specific failure modes identified in the paper: prompt-induced list recitation and GEC-style over-correction. The qualitative examples emphasize this narrow but practical scope: “that mirror referred to” becomes “that Mira referred to,” and “If I who are we waiting for?” becomes “Yvonne, who are we waiting for?” under contextual biasing (Wang, 21 Jan 2026).

The limitations are equally clear. LOGIC depends on exact token-prefix matching in the trie; it does not perform semantic retrieval, fuzzy entity matching, or learned entity embedding comparison. Its gains are therefore largest when acoustics already partially support the target entity and decoding ambiguity is local. The paper explicitly identifies semantic biasing as future work. A further limitation is that the evaluation emphasizes person names; a broader account of playlist titles, product names, or technical terminology would require additional experiments. Finally, although the abstract frames the method as operating with constant-time complexity relative to prompt length, the detailed complexity statement is more precise: the sparse logit update is $\Phi_t$0, where $\Phi_t$1 is the maximum trie out-degree, so the improvement is relative to prompt-based context injection rather than a universal constant-time decoding claim (Wang, 21 Jan 2026).

In the current form, LOGIC marks a specific shift in speech LLM adaptation: contextual knowledge is no longer inserted through the text prompt or applied after decoding, but integrated into the logit space of the base decoder itself. That design choice explains both its empirical behavior and its practical significance.