Semantic ID Prefix-ngram in Recommender Systems

• Semantic ID Prefix-ngram is a token parameterization method that uses multi-level residual quantization of multimodal embeddings to generate semantically informed tokens at various prefix granularities.
• It enhances performance by reducing normalized entropy and overfitting, as demonstrated by ablation studies and successful production-scale deployments in systems like Meta Ads Ranking.
• The approach integrates with sequential and attention-based models, providing robust item representations and efficient recommendations in dynamic, large-scale environments.

Semantic ID Prefix-ngram is a token parameterization and embedding scheme for recommendation systems, introduced to address instability and inefficiency issues inherent in random hash-based embedding tables for large and dynamic item ID spaces. The technique leverages hierarchical clustering—specifically, multi-level residual quantization of modality-rich content embeddings—to construct semantically informed tokens representing items at various prefix granularities. Extensive ablation and production-scale results demonstrate that Semantic ID prefix-ngram (abbreviated here as "SemID prefix-ngram") improves embedding stability, tail-ID modeling, overfitting resistance, and deployment robustness over prior ID-based and content-hash methods (Zheng et al., 2 Apr 2025). The technique has been deployed in Meta's flagship Ads Ranking system.

1. Formalism and Token Construction

SemID prefix-ngram commences with precomputed multimodal content embeddings for each item $x_i\in\mathbb R^{D}$, generated by a text+image/video foundation model. These are quantized via a Residual Vector Quantized Variational Autoencoder (RQ-VAE) with $L$ layers and codebook size $K$ per layer. The RQ-VAE encoder produces $z_i\in\mathbb R^{D'}$; quantization proceeds recursively: $$r_{i,1}=z_i, \quad c_{i,\ell} = \arg\min_{c\in[0..K-1]} \|v_c^\ell - r_{i,\ell}\|_2^2, \quad r_{i,\ell+1}=r_{i,\ell}-v_{c_{i,\ell}}^\ell$$ with codewords $v_c^\ell$ learned jointly.

For an item with codes $(c_{i,1},...,c_{i,L})$, all prefixes of length up to $n$ are extracted: $$(c_{i,1}),\; (c_{i,1},c_{i,2}),\; ...,\; (c_{i,1},...,c_{i,n})$$ Each prefix is mapped to a unique integer token: $$p_{i,k} = \sum_{t=1}^k (c_{i,t} + 1) K^{k-t} - 1, \quad k=1,...,n$$ This yields up to $L$0 distinct tokens.

Pseudocode: $K$0 Hierarchical clustering via RQ-VAE means that each layer’s code assignment refines its predecessor’s cluster; identical k-prefixes mean the items share the same cluster token at step k.

2. Embedding Parameterization and Downstream Integration

Each distinct prefix token is represented by a learnable embedding $L$1 in a table $L$2, initialized (e.g., by normal or uniform distribution) and trained by standard backpropagation.

Given item $L$3's sequence of prefix tokens $L$4, its embedding is the sum: $L$5 This summed embedding is used in all reported experiments due to its empirical stability and parameter efficiency.

In DLRM-style recommenders, the traditional ID one-hot is replaced with the sequence $L$6, and the sum embedding feeds directly into the model's sparse module. For sequential user histories, the same process applies per clicked item, preserving compatibility with contextual model architectures such as attention or PMA modules.

3. Empirical Results and Ablations

Experiments conducted on large-scale, production-like settings utilize normalized entropy (NE) as the evaluation metric (lower is better). A summary of main findings includes:

Tokenization Ablations:

Param	NE Gain (train) vs. random hash
[2048]×3 Trigram	-0.028%
[2048]×3 Prefix-3gram	-0.141%
[2048]×5 Prefix-5gram	-0.208%
[2048]×6 Prefix-6gram	-0.215%

Segmented NE (Prefix-3gram, K=2048, L=3, collision factor≈3):

Segment	RH	IE	SemID	ΔSemID–RH	ΔSemID–IE
Head 0.1%	0.80105	0.80101	0.80108	+0.00%	+0.01%
Torso 5.6%	0.83589	0.83583	0.83580	-0.01%	-0.00%
Tail 100%	0.83904	0.83886	0.83872	-0.04%	-0.02%
New items	0.83524	0.83453	0.83180	-0.41%	-0.33%
All items	0.82663	0.82645	0.82621	-0.05%	-0.03%

ID Drift Stability: For all items, SemID (0.0073) outperforms RH (0.0083) and IE (0.0074) (Table 2b).

Retention: 20-day training improves NE_eval by -0.18% (RH) and -0.23% (SemID).

Prefix/Codebook size Effects: Increasing n (prefix length) or K (codebook size) further reduces NE, e.g., n=3→5→6 improves NE from -0.141% to -0.215%; K=512→1024→2048 improves NE from -0.034% to -0.141% (reported at L=3).

No increase in overfitting or instability was observed as depths increased.

4. Integration in Attention-based User Histories

Item embeddings from SemID prefix-ngram are compatible with various sequence aggregators.

User History NE Gains (relative to RH baseline):

Aggregator	ΔNE_train	ΔNE_eval
Bypass	-0.056%	-0.085%
Transformer	-0.071%	-0.110%
PMA	-0.073%	-0.100%

Attention-score statistics on 1,000 eval sequences:

	First	Pad	Entropy	Self
Transf+RH	0.030	0.460	2.149	0.052
Transf+SemID	0.043	0.418	1.967	0.045
PMA+RH	0.071	0.351	3.075	N/A
PMA+SemID	0.074	0.313	3.025	N/A

SemID yields lower entropy (more focused attention) and less spurious pad or self-attention. This suggests improved integration in sequence-aware recommender architectures.

5. Large-scale Production Deployment

Offline RQ-VAE is trained on three months of foundation-model embeddings. Production use cases set L=6, K=2048 ($L$7 million, n=5). The online pipeline is as follows:

1. At ad creation, content embedding $L$8 is computed.
2. RQ-VAE encoder produces codes, which are mapped to prefix tokens.
3. Tokens are persisted in the entity store.
4. At serving, features are constructed by fetching the semantic tokens for target item and history.
5. Model retrieves embedding vectors from the prebuilt table for inference.

Production NE Gains (flagship Ads Ranking):

Feature	ΔNE_train	ΔNE_eval
+6 sparse SemID	-0.063%	-0.071%
+1 sequential SemID	-0.110%	-0.123%

An online A/B test yielded a +0.15% improvement in the key business metric.

In click-loss rate tests, swapping items within the same k-prefix monotonically reduces CTR loss as k increases, indicating semantic consistency. Adding six semantic sparse features reduced average A/A variance (AAR) by 43%.

6. Limitations

Scaling n requires an exponential growth in embedding table size (up to $L$9); this places practical constraints on prefix depth and codebook size. The RQ-VAE codebooks are static; semantic drift in item content or taxonomy necessitates periodic off-line retraining to maintain alignment. Evaluation is conducted solely via normalized entropy; AUC/log-loss and statistical significance analyses are not reported (Zheng et al., 2 Apr 2025).

7. Prospects and Future Directions

Potential enhancements include adaptive codebook sizes per RQ-VAE layer to respect semantic complexity at each hierarchical level, online fine-tuning of VQ codebooks to capture semantic drift, combining semantic prefix tokens with learned hash buckets for heavy-tailed regimes, and application across cross-domain (multi-category) user histories. These avenues aim to further improve embedding utility and deployment resilience in dynamic, large-scale recommender environments.