Quantizing Intent: Cross-Domain Semantic IDs from Organic Activity for Industrial Ranking

Abstract: Ads click-through rate (CTR) prediction is constrained by sparse user supervision: most users engage with ads infrequently while generating dense behavioral evidence in organic surfaces such as feed. Transferring these cross-domain signals into ads ranking is difficult due to domain mismatch, serving cost, and production complexity. We introduce cross-domain user Semantic IDs (SIDs) derived from organic feed activity and show that behavioral activity richness governs cross-domain transfer quality: SIDs from user profile text yield +0.036% AUC, SIDs from an activity-tuned LLaMA-based user embedding model yield +0.107%, and SIDs from direct feed activity behavioral embeddings yield +0.213%. We further propose RQ-FSQ, a residual finite scalar quantization method that discretizes pre-trained embeddings while matching dense-embedding AUC at substantially smaller storage. Across two heterogeneous sources, RQ-FSQ matches or slightly exceeds dense source embeddings, achieving +0.351% AUC for Feed Activity at about 30x smaller storage and +0.265% AUC for Activity-Tuned LLaMA at about 280x smaller storage. We also introduce a Hierarchical Discrete Embedding module that encodes multi-level SIDs through prefix n-gram sparse embedding tables trained end-to-end under the CTR objective. In a large-scale industrial ads ranking system, cold-start segment analysis shows gains up to +1.522% for users with near-zero ad interaction history, validating cross-domain behavioral transfer as an effective bridge for sparse-history ranking.

• The paper introduces cross-domain Semantic IDs that quantize organic user activity into compact tokens for industrial ad CTR models, achieving up to +1.522% AUC for the coldest users and +0.296% with structured multi-source IDs.
• RQ-FSQ combines scalar and residual quantization to match or slightly exceed dense-embedding performance while reducing storage by roughly 30× for feed activity and 280× for LLM-based representations.
• The results show that richer behavioral sources produce larger transfer gains, while hierarchical embeddings and backbone-based imputation preserve coverage with minimal serving cost across sparse ad-domain histories.

Overview

This paper studies cross-domain viewer Semantic IDs (SIDs) for industrial ads click-through rate (CTR) prediction. The central idea is to discretize user representations derived from organic feed activity into compact token sequences and use them as input features in a production decoder-only Transformer ranking model, thereby transferring dense behavioral evidence from organic surfaces into the ad-sparse ads domain (2606.01396). To the authors' knowledge, this is the first empirical study of cross-domain viewer SIDs for industrial ads CTR modeling; prior SID work (TIGER, YouTube ranking, LC-Rec, Meta's production deployment) operates almost exclusively within a single domain.

The paper makes four contributions: RQ-FSQ, a hybrid quantization scheme; the behavioral activity richness principle governing source selection; the Hierarchical Discrete Embedding (HDE) Module; and a structured Multi-Source SID with backbone-based imputation.

Behavioral activity richness governs transfer quality

The most consequential finding is a monotonic relationship between the amount of behavioral activity encoded in the source embedding and downstream AUC gain, all measured against a no-SID production baseline with $K{=}3$ codes per source:

Source	Signal type	ΔAUC
Profile Qwen SID	Text semantics only	+0.036%
Activity-Tuned LLaMA SID	Activity-trained, profile-prompted	+0.107%
Feed Activity SID	Direct 1-year behavioral aggregation	+0.213%

This progression supports what the authors term the behavioral activity richness principle: transfer quality is determined by how much behavioral activity the source encodes, whether directly or implicitly through an activity-trained encoder. Notably, the Activity-Tuned LLaMA result shows that implicit behavioral signal injected during contrastive fine-tuning survives even when the input prompt is profile text alone. The mechanism proposed is implicit domain adaptation: quantization strips domain-specific geometry while preserving semantic cluster structure, and the CTR gradient re-specializes the embedding tables toward ad engagement probability without any explicit alignment loss — qualitatively analogous to pre-train/fine-tune transfer but operating in discrete token space.

Cold-start stratification directly validates this mechanism. Partitioning users by trailing ad-impression history, Feed Activity SID gains are strongly non-uniform: +1.522% for the most cold-start segment (bottom 8% by history size), +0.874% for infrequent users, and only +0.131% for frequent users who already possess rich first-party ads signals. This is the strongest numerical claim in the paper and confirms that cross-domain SIDs act as a behavioral bridge precisely where ads-domain history fails.

Multi-Source SID with backbone imputation

Naïvely summing three independently indexed single-source SIDs yields +0.260%. The structured Multi-Source SID — a 9-code representation combining Activity-Tuned LLaMA ($c_1$–$c_3$), Profile Qwen ($c_4$–$c_6$), and Feed Activity ($c_7$–$c_9$) — achieves +0.296%, a +0.036% gain at identical parameter budget. Two factors drive the improvement: per-source prefix n-gram tables avoid cross-source hash collisions, and backbone-based imputation preserves population coverage. When Profile Qwen or Feed Activity embeddings are missing, they are reconstructed from the Activity-Tuned LLaMA embedding via a dedicated residual VAE trained on co-present users; when the backbone itself is missing, padding codes map to zero embeddings so the representation degrades cleanly. Coverage preservation matters because no single source achieves full population coverage in production.

RQ-FSQ: matching dense embeddings at large storage reductions

RQ-FSQ pairs FSQ's per-dimension scalar quantization (preserving fine structure) with RQ-VAE residual quantization (preserving global geometry), fusing the two streams additively in the downstream model. Against the dense float32 baseline:

Method	Feed Activity storage / ΔAUC	LLaMA storage / ΔAUC
Dense float	1× / +0.349%	1× / +0.264%
RQ-KMeans	~0.004× / +0.213%	~0.0004× / +0.107%
FSQ	~0.03× / +0.343%	~0.003× / +0.248%
RQ-FSQ	~0.03× / +0.351%	~0.003× / +0.265%

RQ-FSQ matches or slightly exceeds the dense baseline on both heterogeneous sources at roughly 30× (Feed Activity) and 280× (Activity-Tuned LLaMA) smaller storage. The benefit scales with source dimensionality, making it particularly attractive for high-dimensional LLM-based encoders where float storage is prohibitive. A public replication on MovieLens-100K confirms the ordering: RQ-FSQ reaches 0.8343 AUC (+6.54%) versus 0.8215 for RQ-KMeans and 0.8078 for the dense embedding, exceeding even the dense baseline. The authors frame the RQ-KMeans vs. RQ-FSQ choice as a storage–fidelity trade-off rather than a binary preference; RQ-KMeans remains preferable when storage dominates and modest AUC loss is acceptable.

HDE Module and integration

The HDE Module maps any $K$-level SID into a dense vector via prefix n-gram sparse tables: level 1 uses direct unigram lookup, levels $\geq 2$ use polynomial hashing of prefix $k$-grams into tables capped at $c_1$0. Unlike SIDE's collision-free positional base-$c_1$1 encoding, which scales as $c_1$2, controlled collisions bound memory — a trade-off the authors argue is necessary at industrial scale. All tables train end-to-end under the CTR objective with a higher learning rate than Transformer weights. User SIDs are request-level, broadcast across all sequence positions, materialized offline, and summed with event-level embeddings before LayerNorm, leaving the backbone, attention stack, and prediction head untouched. Serving cost is negligible since lookups are local memory accesses over precomputed tokens.

Limitations and open questions

Several caveats bear on the results. Absolute AUC values are withheld under confidentiality policy, so all production claims rest on relative deltas validated only by the authors' internal correspondence between offline AUC and online CTR impact. Hyperparameters ($c_1$3, $c_1$4, $c_1$5) were selected via internal sweeps described only summarily, and the claim that larger $c_1$6 or $c_1$7 yields no measurable improvement is specific to their deployment scale. The three embedding sources are platform-specific; while the authors map them to reproducible categories, the behavioral richness principle has been demonstrated on exactly three points along one monotonic axis rather than established as a general law. The MovieLens replication validates only the RQ-FSQ > RQ-KMeans ordering, not the cross-domain transfer results themselves. Finally, the discussion raises a privacy consideration — cross-domain SIDs increase the granularity at which inferred interests can be linked across domains — which the authors address only through the low resolution of the discrete bottleneck and reliance on existing consent controls.

Conclusion

This paper establishes cross-domain viewer SIDs as an effective feature class for industrial ads CTR, demonstrating monotonic gains with source behavioral richness (+0.036%/+0.107%/+0.213%), structured multi-source combination reaching +0.296%, and cold-start gains up to +1.522%. RQ-FSQ provides a general-purpose discretizer that matches dense-float quality at 30×–280× storage reduction, validated on both proprietary and public data. The open question the work leaves is whether the behavioral richness principle and the unified discrete-token interface extend to other ranking surfaces and multi-task objectives beyond CTR, as its outlook suggests.