TencentGR-10M: Generative Recommendation Benchmark

• TencentGR-10M is an all-modality benchmark designed for generative recommendation in industrial advertising using real Tencent Ads logs.
• It integrates collaborative identifiers with six continuous multi-modal embeddings to represent ads across textual and visual modalities.
• The benchmark employs a Transformer-based autoregressive model with a weighted evaluation protocol that emphasizes high-value click and conversion events.

Searching arXiv for the benchmark paper and directly related work. TencentGR-10M is an all-modality generative-recommendation benchmark for industrial advertising, introduced as the final track of the Tencent Advertising Algorithm Challenge 2025. It is built from real de-identified Tencent Ads logs and is designed for generative recommender systems in which collaborative identifiers and multi-modal content are mapped into tokenized or embedded representations and user behavior is modeled with autoregressive sequence models. In the benchmark definition, TencentGR-10M scales to 10 million users, distinguishes click and conversion events at both the sequence and target level, and uses weighted evaluation to emphasize high-value conversion events (Pan et al., 4 Apr 2026).

1. Benchmark context and problem formulation

The benchmark was introduced in response to a stated gap in public datasets for generative recommendation: existing multi-modal recommendation datasets did not jointly provide large-scale, realistic, and fully all-modality data designed specifically for generative recommendation in industrial advertising (Pan et al., 4 Apr 2026). TencentGR-10M therefore serves both as a dataset and as a task specification for sequence-based ad retrieval under realistic logged user behavior.

The benchmark is situated within a generative-recommendation framework in which user history is represented as an ordered sequence of interactions and the learning objective is next-item prediction. In TencentGR-10M, the target is a single exposure that is retrospectively linked to a qualifying downstream event. Specifically, within a global time window $[t_{\rm begin}, t_{\rm end}]$, the dataset samples users who have at least one click or conversion; for each user, the first qualifying event after $t_{\rm begin}$ is selected, with conversion preferred over click, and that event is attributed back to its triggering exposure. That exposure becomes the single ground-truth target, while all interactions strictly before it form the model input sequence, truncated to a maximum of 100 items plus one user-profile token (Pan et al., 4 Apr 2026).

This formulation is notable because the benchmark is not framed as generic implicit-feedback recommendation. It is explicitly tied to an advertising pipeline in which exposures, clicks, and conversions are distinct event types, and where conversion-aware evaluation is part of the task definition. A plausible implication is that TencentGR-10M is intended not merely for engagement prediction but for retrieval systems that must preserve utility under heterogeneous action semantics.

2. Data construction, de-identification, and corpus statistics

TencentGR-10M is constructed from Tencent Ads interaction logs containing item exposures, clicks, and conversion attributions. All personally identifying fields are removed; the description specifies that user names, IP addresses, and raw images, videos, or ad text are stripped. Users, ads, and other categorical fields are replaced by 64-bit SHA-hashed IDs. Multi-modal content is not released in raw form; instead, text, image, and video information is encoded offline by production embedding models (Pan et al., 4 Apr 2026).

The user sampling procedure yields $N = 10{,}139{,}575$ users with at least one click or conversion in the time window. The candidate pool contains $3{,}637{,}720$ distinct ads. The average observed sequence length before truncation is 97.29, with a maximum of 100. Across all sequences, total logged events are distributed as exposures (94.63%), clicks (2.85%), and conversions (2.52%) (Pan et al., 4 Apr 2026).

These corpus-level statistics establish the benchmark as both large-scale and highly imbalanced across action types. The predominance of exposures is structurally typical of advertising logs, while the presence of explicit click and conversion signals enables action-aware modeling and weighted evaluation. A potential misconception is that “all-modality” implies release of raw creative assets; in TencentGR-10M, the released representation is instead based on pre-computed embeddings rather than raw text, image, or video media (Pan et al., 4 Apr 2026).

3. Feature schema and all-modality representation

TencentGR-10M combines collaborative identifiers with six pre-computed continuous multi-modal embeddings per ad. On the collaborative side, each token is a tuple of sparse fields, each with its own embedding table. The dataset includes 12 ad-side categorical fields with feature IDs 100–102 and 112, 114–122, such as advertiser ID, campaign ID, product category, and creative ID, with cardinalities ranging from 6 up to 5 million. It also includes 8 user-side categorical fields with feature IDs 103–110, including demographics and long-term interest categories, with cardinalities ranging from 2 up to 19 distinct values per field. Each sparse field $f$ is represented by a learnable lookup embedding $\mathbf{e}_f \in \mathbb{R}^{d_f}$ (Pan et al., 4 Apr 2026).

On the multi-modal side, each ad may contain up to six embedding vectors produced by state-of-the-art models. The benchmark description identifies the following embeddings.

Emb. ID	Model	Modality / Dim
81	Bert-finetune	T / 32
82	Conan-embedding	T / 1,024
83	gte-Qwen2-7B	T / 3,584
84	hunyuan_mm_7B	I / 32
85	QQMM-embed-v1	I / 3,584
86	UniME-LLaVA-7B	I / 3,584

For TencentGR-10M, the original 4,096-dimensional hunyuan image embedding is reduced to 32 dimensions via a learned projection for storage and efficient downstream training. The other continuous embeddings are not further quantized or discretized (Pan et al., 4 Apr 2026).

The benchmark therefore couples sparse collaborative structure with dense semantic side information. This suggests a design space in which representation learning can proceed either by direct fusion of continuous embeddings or by downstream discretization into semantic identifiers, a strategy later adopted by several top-ranked solutions.

4. Tokenization, sequence encoding, and baseline model

TencentGR-10M does not tokenize content into subword units. Instead, each sparse ID or categorical field defines its own discrete vocabulary, and each continuous multi-modal embedding is used directly as a vector. For the user token, the representation collects user-side fields; for each item token at time $t$, it collects user-item fields together with the multi-modal embeddings. All field embeddings are concatenated and passed through a small feed-forward network to produce a unified token vector in $\mathbb{R}^d$ (Pan et al., 4 Apr 2026).

A single user-profile token acts as an implicit CLS-like token, and there is no explicit SEP token between items. Temporal order is represented by learnable positional embeddings, yielding the Transformer input

$$\mathbf{H}^0 \;=\;\bigl[\;\mathbf{x}_u^0+\mathbf{p}_0,\;\mathbf{x}_{u,1}^0+\mathbf{p}_1,\;\ldots,\;\mathbf{x}_{u,T_u}^0+\mathbf{p}_{T_u}\bigr].$$

The provided baseline implementation follows a next-token prediction paradigm using a causal Transformer and an InfoNCE retrieval loss (Pan et al., 4 Apr 2026). The architecture uses Transformer depth $L=1$, hidden size $t_{\rm begin}$0, a single attention head, dropout 0.2, and token embedding dimension 32 after the MLP. Sequence length is padded or truncated to 101 tokens, corresponding to one user token and 100 item tokens.

For training, each position $t_{\rm begin}$1 uses the next item $t_{\rm begin}$2 at time $t_{\rm begin}$3 as the positive target, while one or more negatives $t_{\rm begin}$4 are sampled uniformly from the full candidate pool. If $t_{\rm begin}$5 denotes the dot-product score between the user state $t_{\rm begin}$6 and the candidate item embedding, the loss is

$t_{\rm begin}$7

Here $t_{\rm begin}$8 is action-dependent: it is typically 1 for clicks and $t_{\rm begin}$9 for conversions, while exposures receive weight 0 in the second round. The baseline uses Adam with learning rate 0.001 on a single GPU, and one negative per positive by default (Pan et al., 4 Apr 2026).

Inference proceeds in three steps: compute the user embedding $N = 10{,}139{,}575$0, pre-compute and index all candidate-item embeddings with FAISS, and retrieve top-$N = 10{,}139{,}575$1 nearest neighbors under inner-product search (Pan et al., 4 Apr 2026). This makes the benchmark explicitly retrieval-oriented rather than purely generative in the language-model sense, despite its “generative recommendation” framing.

5. Evaluation protocol and weighted relevance

TencentGR-10M uses a weighted evaluation protocol that reflects the differing value of exposures, clicks, and conversions. In the final track, relevance is assigned by

$N = 10{,}139{,}575$2

The benchmark then defines weighted HitRate@K and weighted NDCG@K over the top-$N = 10{,}139{,}575$3 predictions $N = 10{,}139{,}575$4 and the singleton ground-truth set $N = 10{,}139{,}575$5 (Pan et al., 4 Apr 2026).

The weighted hit rate is

$N = 10{,}139{,}575$6

The weighted discounted cumulative gain is

$N = 10{,}139{,}575$7

with ideal DCG

$N = 10{,}139{,}575$8

where $N = 10{,}139{,}575$9 denotes the ideal ordering, and

$3{,}637{,}720$0

The final leaderboard score uses the same HitRate/NDCG weighting pattern as the preliminary round, but applied to these weighted metrics (Pan et al., 4 Apr 2026). Data splitting is temporal: for each user, the target exposure at time $3{,}637{,}720$1 is withheld, while all interactions with $3{,}637{,}720$2 are provided as input. Participants receive the training data with held-out targets and submit inference code; the test answers remain private and evaluation is performed within Tencent Angel (Pan et al., 4 Apr 2026).

The evaluation design is significant because it makes the benchmark explicitly utility-sensitive. Unlike standard top-$3{,}637{,}720$3 retrieval benchmarks where relevance is often binary, TencentGR-10M privileges conversion-bearing targets through the $3{,}637{,}720$4 weighting. This suggests that improvements in ranking quality are intended to be interpreted in the context of action value rather than click-only relevance.

6. Top-ranked solutions and emergent design patterns

The benchmark paper reports several convergent patterns among the top-ranked and award-winning solutions (Pan et al., 4 Apr 2026). The first and second place teams adopted explicit per-token action conditioning, drawing on PinRec (Badrinath et al., 9 Apr 2025), so that each item token’s representation is modulated by action type—exposure, click, or conversion—via gated-fusion or FiLM layers and attention bias terms. According to the benchmark report, this disentangles signals associated with different user intents.

Both first and second place solutions applied residual-quantized k-means to multi-modal embeddings in order to generate compact “semantic IDs.” A random-$3{,}637{,}720$5 regularization strategy was used to prevent collapse of rare codes. Rich temporal encoding also emerged as a common strategy: winning systems engineered hierarchical time features, including absolute timestamp embeddings, inter-event gaps, session-level markers, and multi-frequency Fourier features, and fused them into the Transformer either as additional tokens or through time-biased attention (Pan et al., 4 Apr 2026).

The second place team used two-stage training, first pre-training on exposures alone and then fine-tuning on clicks and conversions under InfoNCE. The third place team scaled contrastive negatives to 380 K per batch and studied trade-offs among model depth, width, and negative-sample count under fixed compute, reporting that larger negative sets yielded strong gains in recall and NDCG. The Technical Innovation Award solution combined semantic ID generation and action prediction in a unified decoder-only Transformer trained with a multi-task loss, consisting of InfoNCE for ID generation and cross-entropy for action type, and introduced collision-resolution in the secondary codebook (Pan et al., 4 Apr 2026).

The benchmark report also states that, although no winner published a full modality-by-modality ablation, all winners reported a 3–5% drop in weighted-NDCG when visual or audio embeddings were removed, and that the high-dimensional Qwen/text embeddings produced the largest single-modality gains (Pan et al., 4 Apr 2026). Because the released feature schema described in the same report enumerates text and image embeddings but does not separately enumerate an audio embedding field, this point is best read as a report about solution behavior rather than as a full specification of the public feature list.

Taken together, these results indicate that performance on TencentGR-10M depends not only on larger models or more negatives, but on preserving the semantics of action type, temporal structure, and multi-modal content. The benchmark paper summarizes this convergence as a shift toward generative-retrieval systems that explicitly encode action types, quantize multi-modal embeddings into semantic codes, and inject rich temporal and session structure (Pan et al., 4 Apr 2026).