Expressiveness Limits of Autoregressive Semantic ID Generation in Generative Recommendation

Abstract: Generative recommendation (GR) models generate items by autoregressively producing a sequence of discrete tokens that jointly index the target item. However, this autoregressive generation process also induces a structured decoding space whose impact on model expressiveness remains underexplored. Specifically, token-by-token generation can be viewed as traversing a decoding tree induced by semantic ID tokens, where leaf nodes correspond to candidate items. We observe that the item probabilities produced by GR models are strongly correlated with this tree structure: items that are close in the tree tend to receive similar probabilities for any given user, making it difficult to distinguish among them based on user-specific preferences. We further show theoretically that such structural correlations prevent GR models from representing even simple patterns that can be well captured by conventional collaborative filtering models. To mitigate this issue, we propose Latte, a simple modification that injects a latent token before each semantic ID, reshaping the decoding space from a single tree into multiple latent-token-conditioned trees. This design creates multiple paths with varying tree distances between items, relaxing tree-induced probability coupling and yielding an average of 3.45% relative improvement on NDCG@10. Our code is available at https://github.com/hyp1231/Latte.

• The paper presents a theoretical analysis of decoding tree limits that induce probability coupling between items.
• The paper introduces Latte, a latent token mechanism that decouples probability distributions and yields a 3.45% improvement in NDCG@10.
• The paper demonstrates practical extensions to multimodal recommendation by enabling adaptive modality ordering for enhanced performance.

Expressiveness Limits of Autoregressive Semantic ID Generation in Generative Recommendation

Introduction

The paper "Expressiveness Limits of Autoregressive Semantic ID Generation in Generative Recommendation" (2605.06331) presents a thorough analysis of the structural constraints embedded within autoregressive Semantic ID (SID) generation in generative recommendation models (GR). The authors provide both empirical and theoretical characterizations of the probability coupling induced by the decoding tree formed by sequential SID generation. The manuscript introduces the Latte architecture, which injects latent tokens to relax those expressiveness bottlenecks, resulting in consistent improvements in top-N recommendation metrics. The investigation addresses scenarios beyond user-item scoring to include item-item similarity limitations and demonstrates practical extensions such as modality ordering in multimodal recommendation.

Structural Constraints of Autoregressive SID Generation

Autoregressive GR models represent items as multi-token SIDs and model recommendation as sequential token generation. The probability of generating an item is constructed as a product of conditional token probabilities, which, as the authors elaborate, defines a decoding tree: each branch at depth $j$ reflects a prefix of $j$ tokens, and each leaf node corresponds to a concrete item. This construction endows the GR paradigm with an inherent geometry wherein tree path distances encode the prefix sharing between items.

Figure 1: Illustration of the expressive limitations induced by autoregressive SID generation—tree structure imposes strong probability coupling between items sharing long prefixes.

The authors empirically demonstrate that this structural arrangement causes item generation probabilities to be highly correlated for tree-adjacent items, regardless of the user context. In other words, items that are close in the tree (high prefix overlap) are nearly indistinguishable in user ranking. This violates a central requirement for recommender systems: the ability to capture cross-user rank reversals, i.e., instances where different users prefer different item permutations even among highly similar or adjacent items.

Figure 2: Correlation between tree distance and item generation probability similarity—correlation decreases monotonically with increasing tree distance.

The observed monotonic relationship between tree distance and user-based score correlation is formalized via an ultrametric on the item set, establishing that expressive limitations are rooted in the decoding procedure itself and not only in the tokenization design.

Theoretical Characterization: Rank Reversal and Item-Item Similarity

The paper rigorously proves that the probability of cross-user rank reversals between tree-coupled items is sharply upper-bounded by the structural correlation imposed by the decoding tree. When the Pearson correlation $\rho$ between two items' generation probabilities across users approaches 1 (as is the case for tree-adjacent items), the ability to express distinct user preferences for those items vanishes. The provided bound quantifies the suppressive effect of tree-induced correlation on personalized ranking.

Similarly, the structural bias induces forced transitivity in item-item similarity relationships, which is fundamentally misaligned with real-world collaborative filtering behavior where intransitive similarities (e.g., item $i_1$ and $i_2$ both co-occur with $i_3$ but not with each other) are prevalent. This is analytically guaranteed by the ultrametric inequality of the decoding tree: items $i_1$ and $i_3$ cannot be dissimilar if both share high similarity with $i_2$ under the tree geometry.

Latte: Relaxing Tree-Induced Expressiveness Limits

To mitigate these constraints, the authors introduce Latte, a structural augmentation of GR that injects a latent discrete token $\ell$ prior to every SID sequence. This transforms the decoding geometry from a single tree into a latent-token-conditioned forest of trees, with generation for each item and user proceeding via selection of a root latent token and subsequent SID tokens.

During training, a latent token is uniformly sampled for each target, ensuring all latent-token-conditioned paths are traversed. At inference, aggregation (sum or max) across all latent tokens is performed efficiently using beam search. This modification equips the model with context-sensitive generation paths: two items may have small standard tree distance but be assigned to different latent tokens for different users, effectively decoupling their probability computations. The expected effect is a sublinear collapse of shared probability mass and a reduction in forced similarity.

Empirically, Latte consistently yields improved results in NDCG@10 and Recall@10 over strong generative and conventional sequential baselines, with average NDCG@10 gains of $j$0 over the base PSID model.

Figure 3: Analysis of Latte across tokenization methods, aggregation strategies, and latent token vocabulary size, confirming broad generalizability and robustness to design choices.

The uncoupling effect is confirmed by a systematic reduction in Kendall's rank correlation between tree distance and item-item similarity. Notably, introducing even a moderate number of latent tokens (4–8) is sufficient to measurably mitigate the original structural coupling, without significant inference cost.

Application: Dynamic Modality Ordering in Multimodal Settings

The authors further demonstrate the flexibility of the latent token idea in multimodal generative recommendation. By treating each latent token as an indicator of a permutation over item modality order (e.g., playlist, tag, metadata), the model automatically discovers and adopts high-performing modality sequences on a per-user basis, regularly outperforming any fixed sequence baseline. Selection frequency analysis of latent tokens supports that the mechanism adaptively exploits inductive biases induced by permutation-based modeling.

Figure 4: Permutation-indicating latent tokens in multimodal SID generation allow the model to dynamically prioritize modality orders, improving NDCG and recall and revealing selection patterns aligned with optimal fixed-order performance.

Practical and Theoretical Implications

From a theoretical stance, the analysis exposes that tree-structured autoregressive decoding intrinsically limits the class of user-item and item-item distributions that generative recommenders can express. This has implications for both model selection and tokenization protocol design in future SID-based recommenders: without structural interventions, even optimal tokenization cannot overcome coupling-induced expressiveness bottlenecks.

Practically, the proposed solution is minimal in system complexity and inference overhead, and extensible to further context-aware modeling strategies (e.g., conditioning latent tokens on user, time, or side information). The latent token paradigm is not restricted to GR: analogous approaches may be leveraged in other autoregressive structured output settings—such as generative retrieval or semantic hashing—where tree-based dependencies suppress distributional flexibility.

Conclusion

This work delivers a precise identification and quantification of expressiveness bottlenecks in autoregressive SID-based generative recommenders, rooted in the decoding tree geometry. The proposed introduction of latent tokens (Latte) demonstrably relaxes tree-induced coupling, enabling more flexible, user-personalized ranking and similarity relationships, with stable top-N accuracy improvements. Extensions to inductive-bias-aware and multimodal scenarios further highlight the breadth of applicability of the approach. Future directions include designing more structured forms of latent-token conditioning, optimizing the decoder geometry, and theoretically characterizing optimal trade-offs between expressiveness and efficiency in generative recommendation modeling (2605.06331).