Gryphon: A Unified Architecture for Semantic-ID Generation and Item-Level Scoring in Industrial Recommendations

Abstract: Generative retrieval (GR) has become a scalable approach to candidate generation: each item is assigned a short hierarchical token sequence called a Semantic ID (SID), and the next item's SID is decoded autoregressively. A practical limitation is that the decoder's beam search optimizes the likelihood of token sequences, not the relevance of the underlying items. These objectives diverge when sequence likelihood is poorly calibrated due to beam search error accumulation, and when several items collapse onto a single SID and receive identical scores. We introduce Gryphon, an encoder-decoder generative recommendation architecture that adds a jointly trained item-level scoring component alongside SID generation, reusing the encoder's user representation computed in a single forward pass. Instead of ranking SIDs by accumulated token likelihood, Gryphon resolves each generated SID to its concrete items and re-scores those items directly, which sidesteps miscalibrated sequence scores and separates items that collide on the same identifier. On an industrial music service, with item-level scoring trained under a next-item-prediction objective, Gryphon attains the highest item-level Recall@1000, above the strongest baselines (+3.7% over vanilla GR and +2.5% over collision-resolved GR) at comparable parameter count and latency. Gryphon's item-level ranking also surpasses its beam-likelihood ranking of the same candidates (+4.2% gain), demonstrating the benefit of item-level scoring in GR. Deployed as the sole candidate source in a 7-day A/B test, Gryphon produced no statistically significant change in total listening time (+0.25%) while replacing a pipeline of more than 15 candidate generators and a separate preranking stage, substantially simplifying the candidate-generation system.

• The paper introduces a unified architecture that integrates semantic-ID generation with an item-level scoring module to resolve beam search miscalibration and ID collisions.
• It decouples candidate generation from relevance estimation using a Transformer encoder–decoder and a shallow scoring tower, achieving a +3.7% improvement in Recall@1000 over vanilla GR.
• Online A/B testing validates Gryphon’s impact with improved user engagement and reduced system complexity, highlighting its practical advantage in large-scale recommendation.

Gryphon: Unified Semantic-ID Generation and Item-Level Scoring for Generative Industrial Recommender Systems

Introduction and Motivation

Generative Retrieval (GR) has emerged as an efficient architecture for large-scale recommender systems by leveraging compact Semantic IDs (SIDs) as surrogates for item representation. SIDs, generated via hierarchical quantizers, enable scalable candidate generation by casting recommendation as an autoregressive sequence modeling problem. However, structural deficiencies—specifically, the disconnect between sequence-level beam likelihood and item-level utility—impede direct applicability in high-stakes, industrial contexts.

The key failure modes are (1) sequence likelihood miscalibration, due to error accumulation in beam search over autoregressively generated SIDs, and (2) semantic ID collisions, where multiple items share a SID and thus become indistinguishable under pure sequence-likelihood-based ranking. Gryphon introduces a unified encoder–decoder architecture with a jointly trained Item-Level Scoring Module (ILSM), allowing item-level re-scoring that does not rely on potentially miscalibrated beam sequence probabilities and resolves relevance disparities within SID collisions.

Model Architecture

Gryphon enhances a vanilla encoder–decoder GR system with an item-level scorer, explicitly decoupling candidate generation from relevance estimation.

1. Encoder and Decoder: User interaction history is embedded using a Transformer encoder, producing latent user states $E_u$. The decoder predicts SIDs autoregressively from $E_u$, mirroring existing GR protocols.
2. Item-Level Scoring Module (ILSM): Candidate SIDs from the beam search are resolved into collision groups (sets of items with identical SIDs). For each item, an embedding is produced using a shallow tower (taking tokenized SIDs and item features), then cross-attended with $E_u$ in the ILSM. The result is a scalar item relevance score.
3. Training Objective: Gryphon is learned using a composite loss:
   • $\mathcal{L}_{\mathrm{gen}}$: Next-SID generation loss with teacher forcing.
   • $\mathcal{L}_{\mathrm{NIP}}$: Next-item-prediction loss using sampled softmax with negative log-likelihood and sampling correction.
   • The total objective is $$\mathcal{L} = \mathcal{L}_{\mathrm{gen}} + \lambda\mathcal{L}_{\mathrm{NIP}}$$, with a shared encoder encouraging representations well-suited to both candidate generation and item relevance.

Inference proceeds by first generating a set of SIDs using beam search, expanding each to its collision group, and then applying the ILSM to all resolved items; the top-$N$ items by ILSM score form the candidate set delivered to downstream ranking.

Figure 1: Gryphon inference: beam search generates candidate SIDs, which are resolved to all matching items and re-scored with ILSM, while beam likelihood is discarded from ranking.

Methodological Implications

Gryphon's architecture directly addresses SEMANTIC ID collisions and likelihood miscalibration that plague vanilla GR:

• By sidestepping reliance on the (potentially miscalibrated) product of sequence token likelihoods, Gryphon enables order-sensitive, context-aware relevance estimation without the need for unique terminal tokens or collision resolution layers, which are unscalable as catalog size grows.
• The re-use of encoder states ensures computational efficiency, keeping model latency and parameter count nearly constant versus pure GR baselines.
• Provided the candidate SIDs contain relevant items, the downstream ILSM can recover and promote these items even if their corresponding sequence-level likelihoods are suboptimal.

Experimental Results

Offline Evaluation

Gryphon's performance was measured on one week of logs from a large-scale music recommendation platform, comparing against:

• ARGUS: Production autoregressive two-tower Transformer.
• Vanilla GR: Pure SID generative retrieval with collision-expansion.
• Vanilla GR Resolved: SID collision resolving with appended unique tokens.

Key metric is Recall@1000. Numerical results:

Method	Recall@1000
ARGUS	0.6582
Vanilla GR	0.8245
Vanilla GR Resolved	0.8343
Gryphon	0.8552

Gryphon exhibits a +3.7% absolute gain over vanilla GR and a +2.5% gain over resolved GR at comparable inference cost and parameter count. Notably, Gryphon’s item-level scorer outperforms beam-likelihood-based ranking of the same candidate pool by +4.2% Recall@1000, indicating that the main bottleneck in vanilla GR is not candidate generation but incorrect scoring due to miscalibrated likelihoods and collision-induced ambiguity.

Ablation

A further ablation demonstrated that rescoring candidates with ILSM raises Recall@1000 above the SID-level ceiling established by beam search, directly quantifying the theoretical and empirical value of item-level re-scoring.

Online A/B Test

A 7-day A/B test on the production stack, replacing a heterogeneous pipeline of >15 candidate generators (with preranking) with Gryphon as the sole source, yielded the following:

• Total Listening Time (TLT): +0.25% (no statistical significance vs control)
• Active User Ratio: +0.43% (statistically significant, $p<0.001$)
• Unfinished Tracks: -1.3% (statistically significant, $p<0.001$)
• System complexity and computational load for candidate generation were significantly reduced.

Theoretical and Practical Implications

Gryphon provides a formal separation between candidate generation and item relevance, enabling more robust handling of semantic identifier collisions in dynamic catalogs and improved alignment with deployment metrics. This unified framework paves the way for efficient, production-compatible generative recommenders that can flexibly incorporate richer item-specific features and objectives, such as multi-objective utility estimation or long-horizon user engagement modeling.

The empirical demonstration that Recall@1000 can surpass the SID-level beam ceiling establishes item-level scoring as an essential component in closing the deployment gap for GR-based recommendation architectures.

Future Directions

Potential extensions include training ILSM under more complex objectives, such as:

• Multi-objective engagement modeling (e.g., TLT, skip rate)
• Distillation from full-ranker outputs to better align candidate generation with end-to-end user experiences
• Direct incorporation of long-term value/reward estimation, potentially leveraging reinforcement learning approaches.

Additionally, harmonizing ILSM supervision with downstream ranker feedback remains a promising avenue to further increase system robustness and utility, especially in non-stationary or fast-evolving item catalogs.

Conclusion

Gryphon introduces a practically viable, unified approach to bridging the gap between generative retrieval and item-level utility in industrial recommender systems. By integrating item-level scoring with SID-based candidate generation, it achieves both substantial quantitative improvements and system simplification. Gryphon’s architectural decoupling of candidate generation and item relevance scoring has direct implications for the design of scalable, effective, and maintainable recommender systems and constitutes a significant methodological advancement in generative recommendation architectures (2606.08604).