Cold-Starts in Generative Recommendation: A Reproducibility Study

Abstract: Cold-start recommendation remains a central challenge in dynamic, open-world platforms, requiring models to recommend for newly registered users (user cold-start) and to recommend newly introduced items to existing users (item cold-start) under sparse or missing interaction signals. Recent generative recommenders built on pre-trained LLMs (PLMs) are often expected to mitigate cold-start by using item semantic information (e.g., titles and descriptions) and test-time conditioning on limited user context. However, cold-start is rarely treated as a primary evaluation setting in existing studies, and reported gains are difficult to interpret because key design choices, such as model scale, identifier design, and training strategy, are frequently changed together. In this work, we present a systematic reproducibility study of generative recommendation under a unified suite of cold-start protocols.

• The paper demonstrates that identifier design is the dominant factor in cold-start robustness, outweighing the benefits of model scaling and RL enhancements.
• It employs a rigorous methodology with diverse generative recommender benchmarks across various identifier codings and training protocols.
• The study reveals a performance asymmetry between user and item cold-starts, emphasizing the need for semantic compositional identifiers and tailored training objectives.

A Comprehensive Analysis of Cold-Starts in Generative Recommendation: A Reproducibility Study

Introduction

This paper provides a systematic and controlled study of generative recommenders under cold-start scenarios, focusing on how key model factors—model scale, item identifier design, and training strategies—govern cold-start robustness and generalization (2603.29845). Generative recommendation methods leveraging PLMs have shown promise in modeling both user and item semantics, but prior work has not rigorously isolated their behavior in cold-start situations. The analysis specifically addresses recommendation challenges for newly registered users (user cold-start) and newly introduced items (item cold-start), presenting unified cold-start evaluation protocols.

Figure 1: Generative recommendation pipeline highlights the identifier generation process conditioned on user history.

Experimental Protocols and Benchmarks

A suite of representative generative recommenders is reproduced, spanning diverse identifier codings (atomic IDs, textual, compositional semantic codes via RQ-VAE, balanced k-means, OPQ), model architectures (encoder--decoder, decoder-only, diffusion-based), and training strategies (SFT, RL). Evaluation is conducted on three large-scale datasets representative of e-commerce, content, and gaming domains (Amazon-Toys, MicroLens, Steam), with carefully curated warm-start, user cold-start, and item cold-start splits. Evaluation focuses on top-$K$ ranking metrics (Recall@10 and NDCG@10).

Cold-Start Generalization: Asymmetry and Key Findings

Cold-Start Asymmetry

The results demonstrate a stark asymmetry in cold-start generalization. User cold-starts, where the model is tasked to recommend to unseen users given sparse behavioral history, incur only moderate performance degradation relative to warm-starts. In contrast, item cold-starts result in severe performance collapse. Even state-of-the-art generative recommenders see Recall@10 drop to near-zero on cold items. This highlights the unique difficulty of new-item recommendation, as cold items lack collaborative signals and require robust semantic grounding.

Figure 2: Recall@10 trends for TIGER (with various Flan-T5 scales) demonstrate limited gains for cold-start despite model size increases.

Impact of Model Scale

Contrary to the scaling hypothesis for LLMs, model scale provides only marginal improvements in cold-start settings. As demonstrated by scaling variants of TIGER with Flan-T5 backbones, performance gains under warm or user cold-start are monotonic but modest; for item cold-start, the gain saturates quickly and does not address the performance gap relative to warm settings. Thus, scaling semantic capacity of PLMs is insufficient by itself to bridge the distribution shift of unseen items.

Identifier Design: Trade-offs and Robustness

Identifier design emerges as the dominant factor influencing cold-start robustness. Atomic IDs perform well on in-distribution items/users but fail completely for item cold-start—unseen IDs are unproducible and result in identifier space mismatch. Textual identifiers (e.g., titles) significantly improve item cold-start, as PLMs leverage semantic cues, but cause warm and user cold-start performance to deteriorate due to loss of discriminability and increased prediction confusion (token overlap or ambiguity). Semantic codes, particularly compositional quantizations like OPQ, yield a favorable trade-off: they retain warm and user cold-start accuracy while offering greater robustness compared to atomic IDs under item cold-start.

Figure 3: Identifier design comparison reveals that semantic codes (RQ-VAE, balanced k-means, OPQ) better balance generalization and discriminability than atomic or textual IDs.

Training Strategy: Limited Value of Reinforcement Learning

Introducing RL post SFT (e.g., in OneRec or TIGER variants) does not consistently improve cold-start generalization—sometimes leading to minor drops under distribution shift. Since RL reward objectives are optimized on the training item space, they fail to extrapolate to unseen items or users in cold-start, emphasizing the need for reward functions tailored to out-of-distribution scenarios. The empirical evidence suggests that cold-start-aware objectives, diversity-facing rewards, or training set augmentation are required for shifting cold-start robustness.

Practical and Theoretical Implications

The explicit result that model scaling alone is suboptimal and that identifier selection dominates cold-start robustness bears important design implications for PLM-based recommenders. For practitioners, balancing identifier discriminability and semantic generalizability is critical—textual identifiers should be used only when item cold-start is paramount, while semantic coding (especially OPQ or similarly compositional schemes) should be preferred for holistic coverage. The limited transferability of RL optimization suggests reevaluating training strategies to explicitly regularize for robustness outside the observed interaction distribution.

Theoretically, the asymmetry between user and item cold-start reinforces collaborative signal dependency in recommendation settings, diverging from document retrieval where semantic representations suffice for unseen instances. This finding motivates further analysis of identifier quantization schemes, decoding error propagation, and the design of cold-start-recognizing training objectives.

Future Directions

Directions for future research include:

• Developing theoretical models relating identifier structure to cold-start extrapolation error and aligning quantization/factorization methods to minimize cold-item distribution shift.
• Formulating reward functions and augmentation strategies explicitly focusing on unseen item discoverability and cold user characterization.
• Studying cold-start performance under additional modalities (image, video, multi-modal item representations) and in online adaptation scenarios.

Conclusion

This paper establishes unified cold-start protocols, isolates key generative recommendation design factors, and provides clear empirical evidence that identifier design, not model scaling or RL, is the decisive lever for cold-start robustness. Item cold-start remains an open and challenging problem, with semantic compositional identifiers offering the best trade-off between generalization and discriminability. The study provides both practical guidance for production recommender deployment and a rigorous blueprint for controlled evaluation in generative recommendation research.