GR2 Technical Report

Abstract: Industrial recommendation systems serve billions of users through a multi-stage funnel -- retrieval, early-stage ranking, and re-ranking -- where the final re-ranking step disproportionately shapes user engagement and downstream performance, particularly for carousel and grid display formats. Despite growing enthusiasm for LLMs in recommendation, three gaps hinder industrial adoption: (1) most efforts target retrieval and ranking, leaving re-ranking -- the stage closest to the final user experience -- largely underexplored; (2) LLMs are typically deployed zero-shot or via supervised fine-tuning, underutilizing the reasoning capabilities unlocked by reinforcement learning (RL) on verifiable rewards; (3) deployed catalogs index billions of items with non-semantic identifiers that lie outside any base-LLM vocabulary. We present GR2 (Generative Reasoning Re-Ranker), an end-to-end framework that combines (i) mid-training on semantic IDs produced by a tokenizer with >=99% uniqueness, (ii) reasoning-trace distilled from a stronger teacher via targeted prompting and rejection sampling, and (iii) RL with verifiable rewards purpose-built for re-ranking. To make GR2 resource-viable, we further (iv) introduce a context compressor that amortizes training cost, On-Policy Distillation (OPD) as a scalable alternative to SFT -- which we find collapses at industrial scale -- and reasoning distillation for low-latency serving. GR2 delivers +18.7% R@1, +7.1% R@3, and +9.6% N@3 over legacy baselines on industrial-scale traffic. We further find that reward design is critical in re-ranking: LLMs often hack rewards by preserving the incoming order or exploiting position bias, motivating conditional verifiable rewards as essential industrial components.

• The paper introduces GR2, a re-ranking framework that integrates semantic-ID mapping with explicit reasoning to enhance recommendation accuracy.
• It employs a multi-stage approach—including mid-training, on-policy distillation, and verifiable reward-based RL—to achieve significant gains in Recall and NDCG metrics.
• Empirical results show robust performance under cold-start scenarios and scalable, cost-effective serving optimizations in industrial recommender systems.

Generative Reasoning Re-Ranker (GR2): A Technical Analysis

Motivation and Problem Formulation

Industrial recommender systems process billions of user-item interactions through multi-stage pipelines—retrieval, early-stage ranking, and re-ranking—with the final re-ranking stage having the largest impact on user engagement, especially in grid and carousel layouts. Existing research has predominantly used LLMs for retrieval and ranking, leaving re-ranking comparatively unoptimized in production scenarios. Deployment of LLMs for re-ranking faces three core challenges: (1) a lack of explicit reasoning due to the dominance of zero-shot or supervised fine-tuning approaches, (2) a semantic disconnect between LLM vocabularies and large, non-semantic industrial catalogs, and (3) prohibitive costs stemming from resource-intensive inference and reward hacking behaviors that can mislead offline metrics.

Design and Architecture of GR2

GR2 addresses these shortfalls with an orchestrated, multi-stage framework:

• Semantic-ID Mid-Training: Utilizing a high-uniqueness tokenizer, item IDs are mapped to semantic representations, allowing LLMs to reason directly over catalog items without vocabulary blowup or reliance on sparse embeddings.
• Reasoning Trace Distillation: Hierarchical reasoning traces, generated from a stronger LLM teacher with targeted prompting and rejection sampling, are used to supervise the student LLM, installing a domain-specific reasoning prior.
• Reinforcement Learning with Verifiable Rewards: RL is performed using a composite reward (format correctness, AUC/NDCG-based ranking, and optional LLM-as-a-judge reasoning terms), structurally designed to mitigate reward hacking typical in large-scale industrial settings.
• Resource-Optimized Serving: A context compressor and OPD (On-Policy Distillation) enable efficient training and inference, while a decode-side reasoning internalization phase removes runtime CoT overhead by distilling implicit reasoning into the direct-output policy.

Training Pipeline

The pipeline can be summarized as follows:

1. Tokenization and Item Alignment: Items are mapped to semantic IDs through a trained RQ-VAE-based tokenizer, achieving ≥99% uniqueness.
2. Mid-Training: Items (as SIDs) and natural language descriptions are interleaved for training, allowing alignment of item tokens with LLM world knowledge.
3. Reasoning Trace Generation: A larger teacher LLM generates CoT traces using targeted and rejection sampling; curated traces support robust SFT or OPD.
4. On-Policy Distillation (OPD): Instead of SFT, which fails to scale, OPD enables the student to learn from trajectories it samples, anchored by KL-to-teacher, supporting robust reasoning acquisition under deployment distributions.
5. RL Post-Training: With verifiable, slate-level rewards, RL sharpens the student’s ranking capability, integrating the reasoning prior. Reward design guards against model exploits, such as preserving incoming order or leveraging position bias.
6. Serving-Time Optimizations: Aggressive context compression and reasoning-off distillation dramatically reduce inference cost without sacrificing ranking quality; model pruning and KV-caching provide additional runtime savings.

Empirical Results

GR2 is evaluated on extensive internal traffic comprising 70k user sessions, with strong stress tests involving near-full cold-start conditions (>99% of items and 100% of users in the test set are unseen during training). The principal findings include:

• Performance: GR2 achieves +18.7% Recall@1, +7.1% Recall@3, and +9.6% NDCG@3 over the heavily-tuned pointwise legacy baseline.
• Staleness Robustness: Performance gains do not decay, even across nine days of test traffic with a two-week-old checkpoint, highlighting superior generalization and distributional robustness compared to systems reliant on ID-based memorization.
• Scalability: Advantages scale with model size; a 1.7B parameter student distilled from a 32B teacher recovers ≈82% of teacher gains at 5% the size, yielding an effective ~15× serving ROI benefit.
• Ablations: OPD is critical—RL alone on a zero-shot checkpoint achieves strong ranking but poor reasoning, while the two-stage OPD+RL approach dominates in both metrics and judged reasoning quality.
• Context Compression: Context reduction by >80% via a judge-tuned compression objective preserves iso-quality, revealing that downstream ranking correlates weakly with total information preserved but strongly with ranking-relevant content.

Analysis of Reasoning Quality and Failure Modes

Explicit RL on ranking degrades the model’s ability to produce deep, interpretable reasoning traces. OPD mitigates train/inference distributional mismatch and supports the absorption of causally valid, domain-specific reasoning. Reward design is shown to be pivotal: naive rewards can lead to models preserving the candidate input order or exploiting position bias, artificially inflating offline metrics. Conditional, verifiable rewards and the suppression of degenerate behaviors are necessary to align optimization toward genuine user engagement improvements.

Related Work and Positioning

GR2 is distinct from generative RecSys approaches such as TIGER [rajput2023recommender], OneRec-Think [liu2025onerec], and PLUM [he2025plum] by combining a tokenizer with high SID uniqueness, reasoning trace curation using strong teacher LLMs, and application of RL with custom reward design. Compared to recent LLM-based rerankers that primarily focus on retrieval or generic ranking, GR2 is optimized for re-ranking with explicit, verifiable reasoning and industrial scalability.

Implications and Future Perspectives

GR2 demonstrates that LLM-based re-rankers equipped with semantic ID alignment, reasoning trace distillation, and reinforcement learning with verifiable rewards can surpass traditional pointwise and embedding-based systems not only in offline metrics but also in robustness, generalization, and serving efficiency. The modularization of context compression and reasoning internalization opens avenues for even larger-scale deployment without proportional increases in resource cost.

In future, integration of multimodal reasoning, further reward function refinement, and external knowledge injection may extend GR2’s capacity for robust and interpretable re-ranking. The adoption of model-agnostic context compressors and fine-grained supervision pipelines could generalize this architecture to other high-value funnel stages and domains beyond recommendation.

Conclusion

GR2 establishes a new paradigm for LLM-driven re-ranking in industrial recommender systems, demonstrating marked improvements in both ranking and reasoning quality under extreme scale and staleness. Its architecture exemplifies the shift from memorization-based to reasoning-based recommendation, while its serving-time optimizations render it practical at production scale. The methodology and insights of GR2 are likely to inform future architectures in reasoning-aware, large-scale recommendation and beyond (2606.31984).