Uncertainty-aware Semantic Decoding

• The paper introduces an inference framework that clusters candidate items via logit similarity, yielding an 18.5% HR@3 improvement in sequential recommendation.
• It computes semantic entropy over clusters to adjust item scoring and sampling temperature, ensuring more accurate and uncertainty-aware predictions.
• USD consolidates semantically redundant items to prevent score dilution, demonstrating robust gains across diverse domains and recommendation scenarios.

Uncertainty-aware Semantic Decoding (USD) is an inference-time decoding framework in which model outputs are interpreted at the level of semantic equivalence groups and uncertainty is computed over those groups rather than over isolated tokens or items. In the formulation introduced for LLM-based sequential recommendation, USD clusters candidate items with similar logit vectors into semantic equivalence groups, redistributes probability mass within these clusters, computes entropy across them, and uses that signal to control item scoring and sampling temperature during recommendation inference (Yin et al., 10 Aug 2025). Adjacent work suggests a broader interpretation in which decoding is performed in semantic or task-dependent latent spaces and uncertainty is defined over meanings, behaviors, or structured outputs rather than raw strings (Tomov et al., 29 Jan 2026, Phillips et al., 4 Feb 2026).

1. Problem setting and motivation

In the sequential-recommendation setting studied by USD, each user $u$ has a historical interaction sequence

$$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$

where each $i_j \in \mathcal{I}$ is an item previously interacted with, and the task is to predict the next item $i_{n+1}$ from the unseen item set $\mathcal{I}\setminus\mathcal{H}_u$. The underlying LLM-based recommender is written as

$$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$

with $y_t$ denoting output tokens (Yin et al., 10 Aug 2025).

The motivation for USD is that recommendation is not open-ended text generation. The paper identifies three related mismatches. First, token-level decoding optimizes generation over token sequences, but recommendation requires ranking discrete items. Second, several items may satisfy nearly the same user intent, so standard decoding can treat semantically redundant items as separate competitors and dilute their joint support. Third, conventional decoding is uncertainty-blind: it does not distinguish uncertainty caused by dispersion across semantically equivalent items from uncertainty caused by genuinely different latent intents. In the paper’s formulation, this means standard decoding can overestimate uncertainty, dilute scores among near-duplicate items, and produce suboptimal rankings (Yin et al., 10 Aug 2025).

USD therefore targets a narrower decision object than standard language-model decoding. Rather than asking which token sequence is most likely, it asks how item probabilities should be reinterpreted when several candidates occupy the same semantic neighborhood. This suggests a shift from token-local confidence to uncertainty over semantically meaningful groups.

2. Formalization of the USD framework

USD is explicitly presented as an inference-time method. It does not retrain the backbone model; instead, it changes how candidate outputs are grouped, scored, and sampled during recommendation inference (Yin et al., 10 Aug 2025).

The semantic relation between candidate items $s$ and $s'$ is defined from their logit representations: $$\text{Sim}(s,s') \;=\; \frac{\mathbf{l}_s \cdot \mathbf{l}_{s'}}{\|\mathbf{l}_s\|\;\|\mathbf{l}_{s'}\|},$$ and semantic equivalence is declared by thresholding this similarity: $$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$0 These relations induce semantic clusters

$$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$1

Once clusters are formed, USD aggregates item probabilities into cluster probabilities: $$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$2 Uncertainty is then measured as semantic entropy over the cluster distribution: $$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$3

Item scoring combines the original item probability with a cluster-aware uncertainty term: $$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$4 where

$$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$5

Here $$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$6 denotes the cluster containing item $$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$7. The first term preserves item-specific evidence; the second rewards membership in semantically strong clusters while discounting that contribution when semantic entropy is high (Yin et al., 10 Aug 2025).

USD also adapts the sampling temperature: $$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$8 This introduces an explicit exploration–exploitation mechanism at the semantic-cluster level. Lower semantic uncertainty keeps decoding sharper; higher semantic uncertainty increases exploration. The specific implementation reports $$\mathcal{H}_u = \langle i_1, i_2, \dots, i_n \rangle,$$9 as the semantic-similarity threshold, $i_j \in \mathcal{I}$0, $i_j \in \mathcal{I}$1, and $i_j \in \mathcal{I}$2 (Yin et al., 10 Aug 2025).

3. Inference pipeline and semantic interpretation

USD operates on a small inference-time candidate pool rather than on the full catalog. The paper states that Monte Carlo sampling of $i_j \in \mathcal{I}$3 candidate items is used to compute semantic entropy efficiently during inference, with $i_j \in \mathcal{I}$4. Candidate logit vectors are compared by cosine similarity, and an agglomerative clustering procedure forms semantic clusters under the thresholded equivalence rule (Yin et al., 10 Aug 2025).

The pipeline can be summarized in nine stages. The backbone LLM first produces candidate next-item probabilities and logit vectors. A small candidate set is then sampled. Pairwise logit-space similarities are computed, semantic clusters are formed, and probability mass is aggregated within clusters. Semantic entropy over clusters is computed. USD then adjusts item scores through the hybrid rule above and adapts sampling temperature through semantic entropy. Final ranking or sampling is performed with these uncertainty-aware scores (Yin et al., 10 Aug 2025).

The semantic interpretation is central. If probability mass is split across many items in one cluster, semantic entropy remains relatively low because the model is concentrated on one latent preference mode. If probability mass is spread across several different clusters, semantic entropy is high because the model is uncertain between different intents. The paper illustrates this contrast with product examples: mass spread across several brands of headphones expresses a semantically concentrated preference, whereas mass spread across headphones, baby toys, and sports supplements indicates uncertainty over different intents (Yin et al., 10 Aug 2025).

This cluster-level reading of uncertainty is what distinguishes USD from ordinary next-token decoding. The method assumes that semantic redundancy should not always be treated as indecision. Instead, it should be consolidated before downstream ranking. This suggests that USD is less a new recommender architecture than a new semantic interpretation layer over LLM outputs.

4. Empirical evidence in sequential recommendation

The main experiments use six Amazon Product domains—Baby, Beauty, Clothing, Grocery, Sports, and Toys—with leave-one-out splitting, and apply USD on top of RedPajama-INCITE-Instruct-3B, a decoder-only transformer with 2.8B parameters and 32 layers. The reported evaluation metrics are HR@3, NDCG@3, MRR@3, HR@5, NDCG@5, and MRR@5 (Yin et al., 10 Aug 2025).

The central quantitative claim is that USD improves over the strongest baseline by 18.5% in HR@3, 11.9% in NDCG@3, and 10.8% in MRR@3. In the reported average Amazon results, UniMP achieves $i_j \in \mathcal{I}$5 HR@3, $i_j \in \mathcal{I}$6 NDCG@3, and $i_j \in \mathcal{I}$7 MRR@3, whereas USD achieves $i_j \in \mathcal{I}$8, $i_j \in \mathcal{I}$9, and $i_{n+1}$0, respectively. At cutoff 5, UniMP records $i_{n+1}$1 HR@5, $i_{n+1}$2 NDCG@5, and $i_{n+1}$3 MRR@5, whereas USD records $i_{n+1}$4, $i_{n+1}$5, and $i_{n+1}$6 (Yin et al., 10 Aug 2025).

The paper also isolates decoding policy rather than architecture. For HR@3, the reported decoding baselines are approximately: Beam Search around $i_{n+1}$7, Nucleus Sampling $i_{n+1}$8, Best-of-$i_{n+1}$9 $\mathcal{I}\setminus\mathcal{H}_u$0, Self-consistency $\mathcal{I}\setminus\mathcal{H}_u$1, and USD $\mathcal{I}\setminus\mathcal{H}_u$2. The stated HR@3 improvement over these decoding methods ranges from 2.4% to 23.5% (Yin et al., 10 Aug 2025).

Ablation studies identify semantic clustering as the dominant component. Complete USD reports HR@3 $\mathcal{I}\setminus\mathcal{H}_u$3, NDCG@3 $\mathcal{I}\setminus\mathcal{H}_u$4, and MRR@3 $\mathcal{I}\setminus\mathcal{H}_u$5. Removing uncertainty estimation yields HR@3 $\mathcal{I}\setminus\mathcal{H}_u$6, NDCG@3 $\mathcal{I}\setminus\mathcal{H}_u$7, and MRR@3 $\mathcal{I}\setminus\mathcal{H}_u$8. Removing semantic clustering yields HR@3 $\mathcal{I}\setminus\mathcal{H}_u$9, NDCG@3 $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$0, and MRR@3 $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$1. The paper states that removing semantic clustering causes a 10.5% decrease in HR@3, while removing uncertainty estimation causes a 7.1% decrease in HR@3 (Yin et al., 10 Aug 2025).

Cross-domain results are reported on H{data}M and Netflix. On H{data}M, UniMP records HR@5 $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$2, NDCG@5 $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$3, and MRR@5 $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$4, while USD reaches $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$5, $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$6, and $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$7, corresponding to improvements of 17.6% HR@5, 17.5% NDCG@5, and 18.0% MRR@5. On Netflix, UniMP records HR@5 $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$8, NDCG@5 $$p_\theta(i_{n+1}|\mathcal{H}_u) = \prod_{t=1}^{T} p_\theta(y_t|y_{<t}, \mathcal{H}_u),$$9, and MRR@5 $y_t$0, whereas USD records $y_t$1, $y_t$2, and $y_t$3, corresponding to improvements of 6.5% HR@5, 6.3% NDCG@5, and 6.5% MRR@5 (Yin et al., 10 Aug 2025).

The hyperparameter analysis reports best settings at $y_t$4, $y_t$5, and $y_t$6. The interpretation given is that $y_t$7 balances item specificity and semantic aggregation, $y_t$8 gives the best confidence/exploration balance, and $y_t$9 yields clusters of the right granularity (Yin et al., 10 Aug 2025).

5. Broader research context

Although the term “USD” is used explicitly for sequential recommendation in (Yin et al., 10 Aug 2025), adjacent work shows that the underlying idea belongs to a wider family of semantic-space decoding methods. The paper "Task-Awareness Improves LLM Generations and Uncertainty" models outputs directly in a task-dependent latent structure, equips that structure with a dissimilarity measure, and computes Bayes-optimal responses by minimizing expected task-level loss rather than selecting the most likely string (Tomov et al., 29 Jan 2026). The paper "Semantic Self-Distillation for LLM Uncertainty" learns a prompt-conditioned semantic distribution $s$0, uses the entropy of that distribution as a pre-generation uncertainty signal, and uses posterior density for candidate-answer reliability scoring (Phillips et al., 4 Feb 2026). The paper "Using Semantic Distance to Estimate Uncertainty in LLM-Based Code Generation" defines uncertainty as the probability-weighted semantic spread of sampled programs and introduces the distance-aware scores

$s$1

thereby replacing binary disagreement with graded semantic dissimilarity (He et al., 9 May 2026).

A second line of work treats uncertainty as an inference-time control signal rather than merely a post-hoc diagnostic. "Towards Better Code Generation: Adaptive Decoding with Uncertainty Guidance" triggers lookahead reranking only when token-level entropy exceeds a learned threshold (He et al., 10 Jun 2025). "Towards Uncertainty-Aware Language Agent" uses answer-level uncertainty to decide whether to answer directly, invoke tools, or defer (Han et al., 2024). "UT-ACA: Uncertainty-Triggered Adaptive Context Allocation for Long-Context Inference" fuses semantic embeddings with logit-based confidence and uses that signal to trigger rollback, expand context, and regenerate tokens (Zhou et al., 19 Mar 2026). "Semantic uncertainty in advanced decoding methods for LLM generation" shows that structured decoding can increase semantic diversity while lowering predictive entropy in some settings, especially for summarization and code generation (Foodeei et al., 17 Jun 2025). "R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents" goes further by decoding structured suggestions that maximize expected utility over sampled latent goals rather than maximizing token probability (Johnson et al., 2023).

Taken together, these papers suggest that uncertainty-aware semantic decoding is not reducible to one implementation detail. It can appear as semantic clustering over item logits, Bayes-risk minimization in latent space, density modeling over answer embeddings, distance-aware aggregation over sampled behaviors, or adaptive control over tools, context, or search depth. What unifies these variants is that uncertainty is defined over semantic alternatives rather than over surface strings alone.

6. Terminological scope, limitations, and open questions

The most immediate terminological caution is that the acronym “USD” is not unique. The paper "Unbiased Semantic Decoding with Vision Foundation Models for Few-shot Segmentation" uses “USD” to denote Unbiased Semantic Decoding, but that work explicitly states that it does not introduce uncertainty estimation, confidence calibration, Bayesian decoding, entropy-based rejection, or uncertainty-aware loss design (Wang et al., 19 Nov 2025). In other words, “Unbiased Semantic Decoding” and “Uncertainty-aware Semantic Decoding” name different ideas.

Within the sequential-recommendation USD framework, several limitations are explicit or strongly implied. Cluster quality depends on logit geometry; if logit similarity is a poor proxy for semantic equivalence, cluster assignments can be noisy. The method is sensitive to the similarity threshold $s$2: values that are too high fragment clusters, whereas values that are too low merge distinct intents. The paper does not fully specify the exact agglomerative linkage rule, the detailed candidate-sampling procedure, or the selected $s$3 for adaptive temperature. It also does not report explicit calibration metrics such as confidence–accuracy alignment or calibration error, so claims about “better uncertainty estimation” are supported mainly through downstream recommendation metrics rather than through dedicated uncertainty evaluation (Yin et al., 10 Aug 2025).

The candidate-pool dependence is equally important. Because USD clusters only sampled candidates, its effectiveness depends on whether relevant items enter the $s$4-candidate set at all. The paper also notes that the method may be less useful when catalogs exhibit little semantic redundancy. This suggests that USD should be most effective in domains with many substitutable or near-duplicate items, such as retail catalogs, and less decisive when items are already highly distinct (Yin et al., 10 Aug 2025).

A broader implication, suggested by the surrounding literature, is that future USD systems may move beyond logit-space clustering alone. Task-aware latent structures (Tomov et al., 29 Jan 2026), prompt-conditioned semantic densities (Phillips et al., 4 Feb 2026), graded semantic distances (He et al., 9 May 2026), and uncertainty-triggered adaptive control (He et al., 10 Jun 2025, Zhou et al., 19 Mar 2026) all point toward richer versions of semantic decoding in which uncertainty can govern clarification, reranking, abstention, retrieval, or context allocation. This suggests that USD is best understood not as a single algorithmic recipe, but as a research program for making decoding decisions in semantic space under explicit uncertainty.