Semantic ID Representation

• Semantic ID Representation is a method that encodes the semantic attributes of entities into structured, discrete identifiers to capture content-driven insights.
• It employs continuous embedding, residual vector quantization, and hierarchical tokenization to derive semantically rich and transferable IDs.
• Its practical applications span recommendation systems, generative retrieval, and IoT discovery, offering improved interpretability and scalability.

Semantic ID Representation refers to the practice of encoding the semantic attributes of entities—items, documents, users, devices—into structured, discrete, and often sequential identifiers, rather than random, opaque IDs. This representation paradigm enables transfer of rich semantic information, alignment across modalities, improved generalization to long-tail or unseen entities, and increased interpretability in information retrieval, recommendation, generation, and indexing systems. Recent work has established semantic ID representation as a foundational layer for generative retrieval, content-based and collaborative filtering, multimodal fusion, and personalized synthesis, spanning domains from large-scale recommendation to IoT discovery.

1. Foundations and Motivations

Traditional ID representations assign random, platform-specific ID tokens (e.g., item IDs, one-hot, randomly hashed indices), which are effective for memorization within collaborative models but fail to capture semantic similarity, suppress statistical sharing among related entities, and underperform in cold-start regimes. Semantic ID representation solves these deficiencies by deriving IDs from the content or multimodal features of items, embedding semantics, hierarchical context, and structural priors directly into the identifier space (Singh et al., 2023, Zheng et al., 2 Apr 2025, Liu et al., 11 Dec 2025, Zhang et al., 2024).

Key motivations:

• Generalization: Semantic IDs allow unseen or rare entities to benefit from shared sub-structures with similar items (Singh et al., 2023, Liu et al., 11 Dec 2025).
• Transferability: IDs constructed from rich metadata, multimodal signals, or learned representations facilitate knowledge transfer across domains, platforms, and modalities (Huang et al., 2 Dec 2025, Tan et al., 2024, Penha et al., 14 Aug 2025).
• Interpretability and Stability: Content-derived IDs offer interpretable tokenizations and reduce embedding drift in dynamic environments (Zheng et al., 2 Apr 2025).
• Scalability: Structured IDs can index exponentially large catalogs via compact hierarchical or compositional token representations (Liu et al., 3 Nov 2025, Zhan et al., 21 Nov 2025, Fernandez et al., 2021).

2. Methodological Foundations and Quantization Schemes

Nearly all recent state-of-the-art semantic ID systems employ a multi-stage pipeline consisting of content embedding, quantization into discrete tokens, and (optionally) downstream adaptation:

2.1 Continuous Embedding

Semantic features are acquired by encoding textual, visual, or multimodal attributes through pretrained or fine-tuned models (LLM, CLIP, BERT/Sentence-T5, ResNet, Swin) (Huang et al., 2 Dec 2025, Zhan et al., 21 Nov 2025, Zhou et al., 12 Oct 2025, Xu et al., 21 Aug 2025). For IoT, structured metadata fields may be packed directly (Fernandez et al., 2021).

2.2 Discrete Tokenization

A core technique is residual vector quantization, typically instantiated as RQ-VAE or multi-head VQ-VAE, which recursively decomposes the continuous embedding into a coarse-to-fine sequence of codewords from hierarchical codebooks (Singh et al., 2023, Liu et al., 3 Nov 2025, Huang et al., 2 Dec 2025, Liu et al., 11 Dec 2025, Shi et al., 9 Nov 2025, Fernandez et al., 2021).

Given embedding $z_0$, codebook layers $\{\mathcal{C}^{(l)}\}_{l=1}^L$, tokens are derived as:

$$c_l = \arg\min_{k} \|r_{l-1} - \mathcal{C}^{(l)}_k\|_2, \qquad r_l = r_{l-1} - \mathcal{C}^{(l)}_{c_l}$$

with $r_0 = z_0$ and the semantic ID $[c_1, \ldots, c_L]$.

Parallel and bidirectional tokenizations, e.g., in LLaDA-Rec, split the latent embedding into $M$ sub-vectors quantized independently for symmetric modeling (Shi et al., 9 Nov 2025).

Alternative code assignment mechanisms include mixture-of-codes (MoC), which operates with $M$ independent codebooks to scale up the semantic embedding capacity and improve discriminability (Zhang et al., 2024), and hybrid tokenization with fused ID+semantics (Lin et al., 23 Feb 2025, Liu et al., 11 Dec 2025).

2.3 Losses and Constraints

Most frameworks optimize for reconstruction fidelity plus quantization commitment losses as in VQ-VAE. Category-aware and cluster-scale losses (e.g., CAT-ID$^2$'s hierarchical class constraint, cluster scale constraint, dispersion loss) are employed to enforce that semantically similar entities share code tokens and that the codebook is fully utilized without collapse (Liu et al., 3 Nov 2025).

Contrastive, InfoNCE, or alignment losses are frequent, especially for cross-modal and user-behavior adaptation, ensuring that semantic IDs align with downstream behavioral, category, or multi-view preferences (Xu et al., 21 Aug 2025, Zhou et al., 12 Oct 2025, Liu et al., 11 Dec 2025).

2.4 Unique Assignment and Conflict Resolution

Standard quantization can result in ID conflicts (multiple items mapped to the same token sequence), especially in high-density codebooks. Purely semantic indexing frameworks introduce exhaustive candidate matching (ECM) and recursive residual searching (RRS) to guarantee globally unique, semantic-preserving assignments without auxiliary random tokens (Zhang et al., 19 Sep 2025).

Table: Comparison of Core Quantization Approaches

Scheme	Token Structure	Uniqueness Enforcement	Notable Use
Hierarchical RQ-VAE	Sequential L-tuple	Optionally post-hoc	CAT-ID$^2$, YouTube SID, Meta Ads SID
Parallel VQ-VAE / MoC	M-way code concatenation	Inherently higher capacity	LLaDA-Rec (Shi et al., 9 Nov 2025), MoC (Zhang et al., 2024)
Purely semantic (ECM/RRS)	Candidate enumeration	Enumerative assignment	Uniqueness without random codes (Zhang et al., 19 Sep 2025)
Platform-agnostic textual	NL tag sequence	Autoregressive generation	IDGenRec (Tan et al., 2024)

3. Architectural Variants and Alignment Paradigms

3.1 Cross-Modal Semantic IDs

Advanced models such as MMQ (Xu et al., 21 Aug 2025), Q-BERT4Rec (Huang et al., 2 Dec 2025), and SICSRec (Zhou et al., 12 Oct 2025) explicitly fuse multimodal inputs (text, vision, structure), often with mixture-of-expert tokenizers and cross-modal orthogonal regularizations, to encode both shared and modality-specific semantics.

Behavior-aware adaptation is achieved by fine-tuning semantic IDs under final user-behavioral losses, leading to direct alignment with actual interaction patterns (Xu et al., 21 Aug 2025, Zhan et al., 21 Nov 2025, Liu et al., 11 Dec 2025).

3.2 ID-Semantics Decoupling and Harmonization

Several systems recognize the trade-off between the uniqueness and memorization capacity of hash IDs (HID) and the generalization of semantic IDs (SID). Approaches such as H²Rec (Liu et al., 11 Dec 2025) and unified semantic–ID tokenization (Lin et al., 23 Feb 2025) deploy dual-branch or concatenated embeddings to harmonize collaborative and content-based information, with explicit code-alignment (contrastive) and masked-sequence granularity losses to ensure robust representations across the head and tail of the catalog.

ID-free recommendation replaces explicit ID tokens with pure content- and position-based encodings, dynamically building relational graphs and achieving superior generalization in multimodal settings (Li et al., 8 Jul 2025).

3.3 Generative and Retrieval Contexts

In generative retrieval (DSI, TIGER, CAT-ID$^2$), the entire search or recommendation process is reframed as a sequence-to-sequence generation task, where semantic IDs function as retrieval targets in LLM-centric pipelines. Uniqueness, semantic prefix sharing, and codebook balance are paramount for high-precision direct generation (Liu et al., 3 Nov 2025, Zhang et al., 19 Sep 2025, Jin et al., 2023).

LLaDA-Rec (Shi et al., 9 Nov 2025) demonstrates that discrete diffusion and bidirectional generation over parallel semantic IDs alleviate error accumulation and modeling constraints inherent in autoregressive frameworks.

4. Applications, Use Cases, and Deployment

Semantic ID representation has demonstrated empirical and operational benefits across a range of domains:

• Recommendation and Ranking: Enhanced generalization, cold-start, stability, and reduced overfitting in large-scale systems (Meta Ads, YouTube, Amazon) (Singh et al., 2023, Zheng et al., 2 Apr 2025, Liu et al., 11 Dec 2025, Lin et al., 23 Feb 2025).
• Generative Search and Retrieval: Efficient, interpretable, platform-agnostic identifiers facilitating cross-domain transfer and zero-shot retrieval (Penha et al., 14 Aug 2025, Liu et al., 3 Nov 2025, Zhang et al., 19 Sep 2025, Tan et al., 2024).
• Personalized Generation: Disentangled or jointly embedded identity–semantic spaces enable style-consistent, ID-preserving image synthesis and personalization (Wu et al., 2024, Liu et al., 19 Apr 2025).
• IoT Discovery: Compact, base32-encoded semantic IDs support DNS-based range queries and device lookup by semantic context, logical, or geographic partitioning (Fernandez et al., 2021).
• Generative POI Recommendation: SIDs for POI modeling leverage collaborative and semantic signals, with diversity losses to promote uniform code assignment and inter-domain transfer (Wang et al., 2 Jun 2025).

Empirical analyses consistently show improved AUC, Recall@K, NDCG@K, head–tail balance, and stability metrics upon deployment of semantic IDs, especially when SIDs are hybridized with HIDs or are constructed via advanced quantization/fusion methods (Liu et al., 11 Dec 2025, Zhang et al., 2024, Liu et al., 3 Nov 2025).

5. Scalability, Efficiency, and Robustness

Semantic ID representations are architected to maintain scalability and manageable model complexity, even at dataset scales of O(10^7–10^8) items:

• Token Table Sizing: SIDs, through hierarchical or parallel codebooks, encode exponentially large entity spaces with log-scale token representations, controlling memory and computation via prefix n-gram or SentencePiece-like subtoken strategies (Singh et al., 2023, Zheng et al., 2 Apr 2025, Liu et al., 11 Dec 2025).
• Codebook Utilization: Losses such as CSCL enforce nearly uniform occupation of the codebook, mitigating collapse and preserving discriminability (Liu et al., 3 Nov 2025, Zhang et al., 2024).
• Pruning and Selection: Techniques such as representation-aware token pruning (RASTP) reduce complexity by dropping low-importance tokens, improving efficiency without loss in performance (Zhan et al., 21 Nov 2025).
• Fusion and Bottleneck Modules: MoC and similar fusion architectures allow adaptive scaling of semantic dimensions while maintaining dimension robustness and information preservation (Zhang et al., 2024).
• Stable Online Inference: Prefix n-gram and hierarchical cluster assignment provide stable, interpretable lookups and structured sharing for new, tail, or drifted IDs (Zheng et al., 2 Apr 2025, Liu et al., 11 Dec 2025).
• Conflict Resolution: Global uniqueness in code sequence assignment (ECM/RRS) ensures collision-free semantic IDs without expanding the code vocabulary unnecessarily (Zhang et al., 19 Sep 2025).

6. Recent Developments and Open Challenges

Research has advanced towards more general and robust semantic ID representations:

• Pure Semantic Indexing: Relaxing strict nearest-centroid rules for conflict resolution, moving entirely away from non-semantic tokens or random conflict indices for code uniqueness (Zhang et al., 19 Sep 2025).
• Self-supervised Generative Indexers: End-to-end models (LMIndexer) jointly learn document representations and hierarchical semantic IDs under self-supervised, contrastive, and reconstruction objectives, outperforming two-stage pipelines (Jin et al., 2023).
• Behavioral and Modality Alignment: Behavior-aware fine-tuning and dual-level alignment in SID/HID architectures explicitly transfer collaborative signal to semantically grouped items for both interpretability and recommendation quality (Xu et al., 21 Aug 2025, Liu et al., 11 Dec 2025, Zhou et al., 12 Oct 2025).
• Cross-Modal and Task-General ID Spaces: Unified multi-task training and quantization enable shared semantic ID spaces jointly optimized for search and recommendation (Penha et al., 14 Aug 2025).

Remaining open challenges include proportional scaling of codebooks, end-to-end online codebook training aligned to downstream losses, dynamic code assignment for unseen entities, and extending semantic-ID schemes to session-, event-, or fully cross-domain regimes (Zhang et al., 2024, Li et al., 8 Jul 2025, Huang et al., 2 Dec 2025).

---

References:

• (Singh et al., 2023): Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations
• (Zheng et al., 2 Apr 2025): Enhancing Embedding Representation Stability in Recommendation Systems with Semantic ID
• (Liu et al., 11 Dec 2025): The Best of the Two Worlds: Harmonizing Semantic and Hash IDs for Sequential Recommendation
• (Xu et al., 21 Aug 2025): MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation
• (Liu et al., 3 Nov 2025): CAT-ID$^2$: Category-Tree Integrated Document Identifier Learning for Generative Retrieval In E-commerce
• (Shi et al., 9 Nov 2025): LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendation
• (Zhou et al., 12 Oct 2025): Self-Supervised Representation Learning with ID-Content Modality Alignment for Sequential Recommendation
• (Zhan et al., 21 Nov 2025): RASTP: Representation-Aware Semantic Token Pruning for Generative Recommendation with Semantic Identifiers
• (Zhang et al., 19 Sep 2025): Purely Semantic Indexing for LLM-based Generative Recommendation and Retrieval
• (Tan et al., 2024): IDGenRec: LLM-RecSys Alignment with Textual ID Learning
• (Wu et al., 2024): Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm
• (Fernandez et al., 2021): Semantic Identifiers and DNS Names for IoT
• (Zhang et al., 2024): Towards Scalable Semantic Representation for Recommendation
• (Wang et al., 2 Jun 2025): Generative Next POI Recommendation with Semantic ID
• (Huang et al., 2 Dec 2025): Q-BERT4Rec: Quantized Semantic-ID Representation Learning for Multimodal Recommendation
• (Lin et al., 23 Feb 2025): Unified Semantic and ID Representation Learning for Deep Recommenders
• (Penha et al., 14 Aug 2025): Semantic IDs for Joint Generative Search and Recommendation
• (Li et al., 8 Jul 2025): From ID-based to ID-free: Rethinking ID Effectiveness in Multimodal Collaborative Filtering Recommendation
• (Jin et al., 2023): LLMs As Semantic Indexers