From Local Indices to Global Identifiers: Generative Reranking for Recommender Systems via Global Action Space

Abstract: In modern recommender systems, list-wise reranking serves as a critical phase within the multi-stage pipeline, finalizing the exposed item sequence and directly impacting user satisfaction by modeling complex intra-list item dependencies. Existing methods typically formulate this task as selecting indices from the local input list. However, this approach suffers from a semantically inconsistent action space: the same output neuron (logits) represents different items across different samples, preventing the model from establishing a stable, intrinsic understanding of the items. To address this, we propose GloRank (Global Action Space Ranker), a generative framework that shifts reranking from selecting local indices to generating global identifiers. Specifically, we represent items as sequences of discrete tokens and reformulate reranking as a token generation task. This design effectively decouples the scoring mechanism from the variable input order, ensuring that items are evaluated against a consistent global standard. We further enhance this with a two-stage optimization pipeline: a supervised pre-training phase to initialize the model with high-quality demonstrations, followed by a reinforcement learning-based post-training phase to directly maximize list-wise utility. Extensive experiments on two public benchmarks and a large-scale industrial dataset, coupled with online A/B tests, demonstrate that GloRank consistently outperforms state-of-the-art baselines and achieves superior robustness in cold-start scenarios.

• The paper presents a novel global action space formulation that stabilizes listwise reranking by eliminating label permutation noise.
• It introduces a generative framework using SID-based tokenization with a Transformer encoder-decoder and prefix-tree constrained decoding.
• Empirical results demonstrate 3-9% performance improvements and robust cold-start handling compared to traditional local-index approaches.

Generative Reranking with Global Action Spaces: The GloRank Framework

Introduction and Motivation

List-wise reranking serves as a critical component in industrial recommender systems, optimizing candidate item permutations for maximal user satisfaction by modeling inter-item dependencies. Traditional approaches predominantly select item indices from a locally ordered candidate list, which leads to a semantically inconsistent action space: the correspondence between output logits and item semantics is input-dependent. This inherent label permutation noise impedes stable learning and degrades generalization, as the model cannot assign fixed intrinsic meaning to each output dimension.

Figure 1: Illustration of the semantic inconsistency in conventional position-dependent action spaces where output logits map to different semantic items across samples.

To address this, GloRank introduces a paradigm shift by defining item selection over a fixed global action space, representing each item by a compact, hierarchical global identifier instead of a transient position. This substantive change ensures that the model establishes a stable, semantics-aware evaluation basis, eliminating mapping-induced gradient noise.

Theoretical Analysis: Semantic Consistency in Action Spaces

A key analysis introduced in the paper is the quantification of optimization instability arising from traditional local-index-based mappings. Under random permutations of candidate lists, the supervisory signal for a given output neuron becomes stochastic even for fixed item targets, introducing irreducible variance in gradient estimates. Proposition 1 in the paper shows that this “mapping-induced variance” is strictly positive unless candidate lists are always ordered, which compounds training difficulty, especially in high-cardinality action spaces.

Conversely, when action spaces are globally anchored—every action corresponds to a fixed semantic token—the only remaining stochasticity in supervision arises from the encoder output variation, not label assignment. This leads to more stable and efficient learning, particularly when models are scaled to industrial settings.

Architecture: GloRank's Generative Sequence Framework

GloRank operationalizes the global action formulation via a generative framework composed of:

• Semantic Tokenization: Items are mapped to hierarchical, content-derived semantic identifier (SID) sequences using residual quantization of pre-encoded semantic vectors, enabling compact global vocabularies (RQ-Kmeans, RQ-VAE).
• Input/Output Sequence Construction: Candidate SIDs and user context are serialized; reranking is cast as sequence generation over the global SID vocabulary, making output semantics position-invariant.
• Transformer Encoder-Decoder Backbone: This supports full listwise context modeling and autoregressive generation of SID sequences.
• Prefix-Tree Constrained Decoding: At each generation step, outputs are restricted to SIDs present in the current candidate set, maintaining validity and uniqueness.

  Figure 2: GloRank uses SID-based global tokenization and autoregressive decoding over the global action space, with constrained decoding ensuring only candidates are selected.

Optimization Pipeline: Supervised Pre-training and RL-based Post-training

To maximize performance and stability, GloRank decouples training into two synergistic stages:

• Supervised Pre-training: The model is initialized via next-token prediction on high-quality demonstration lists, which are themselves selected from candidate permutations using a listwise evaluator or heuristic, not simply replayed logging-policy outputs.
• Reinforcement Learning Post-training: Listwise utility is directly maximized through GRPO (Group Relative Policy Optimization), with listwise evaluation metrics as reward signals. This alignments the generation distribution toward sequences with globally superior utility.

Empirical Results

Standard and Counterfactual Evaluations

GloRank achieves consistent quantitative improvements (3-9% relative) over state-of-the-art generator-evaluator architectures and generator-only baselines—including GoalRank and MG-E—on MovieLens-1M, Amazon Books, and a large-scale industrial dataset. Gains are pronounced on NDCG and MAP, demonstrating benefits for both accuracy and ranking order.

Figure 3: Distribution of simulation-based reward scores shows GloRank attaining higher median and upper quantiles compared to leading baselines.

Ablations confirm that both pre-training and post-training are necessary for optimality, with catastrophic performance drops observed if pre-training is omitted. Theoretical insights are empirically validated: replacing the global action space with a local-index variant results in substantial accuracy loss due to increased training noise and overfitting to index patterns rather than item content.

Figure 4: GloRank’s global action space substantially outperforms an otherwise identical local action space variant (LAS) across all standard metrics.

Robustness and Cold-Start Generalization

GloRank demonstrates marked robustness to cold-start scenarios. By constructing item representations from semantic content rather than atomic IDs, GloRank can infer the utility of previously unseen items at inference time, achieving only marginal drops in performance. Legacy architectures such as GoalRank, which operate over atomic IDs, fail catastrophically when encountering unseen items due to embedding random initialization noise contaminating the entire listwise context.

Figure 5: Under cold-start (unseen item) conditions, GloRank maintains high performance while GoalRank degrades sharply.

Hyperparameter Analysis and Online Deployment

Hyperparameter sweeps highlight the monotonic value of increased GRPO group size and confirm that KL regularization is best omitted for maximal reward maximization in list-wise settings. Beam search size impacts ranking-dependent metrics, indicating a precision/reordering tradeoff.

Figure 6: Hyperparameter sensitivity analysis showing consistently stable and superior precision and MAP under varying training and inference configurations.

Online A/B testing on a major content platform demonstrates statistically significant gains (e.g., +0.095% Watch Time, +0.447% Forwards) over a strong GoalRank baseline, directly translating architectural advantages into business metrics.

Implications and Future Directions

GloRank’s decoupling of action space semantics from input ordering enables scalable, robust, semantically compositional reranking. Practically, the SID-based design facilitates dynamic corpus adaptation, cold-start resolution, and improved transferability between retrieval and reranking modules by aligning identifiers. Theoretically, this work suggests that consistent action space semantics should be a first-class consideration in large-scale sequence modeling tasks, not merely an implementation detail.

Potential future research includes more granular SID designs integrating collaborative filtering signals, unified generative modeling across both retrieval and reranking stages, efficient constrained decoding for extremely large global vocabularies, and reinforcement learning with richer reward models reflecting business-aware multi-objective optimization.

Conclusion

GloRank addresses a fundamental bottleneck in current reranking architectures by operationalizing a global action space via SID tokenization and generative decoding. This reformulation eliminates the mapping-induced supervision noise inherent in local-index-based approaches, stabilizing optimization and enhancing transferability, robustness, and downstream utility. Extensive empirical results—offline, online, and under cold-start—support both the theoretical argument and practical impact of this architectural innovation.