Robustness of semantic-ID findings across evaluation settings

Determine whether the observed patterns in semantic-ID design—including dataset-dependent recommendation performance, non-monotonic scaling, and differing semantic-neighborhood preservation—survive under other generative backbones and decoding rules, systematic dataset-specific tuning, larger item catalogs, and cold-start evaluation.

Background

The study derives its conclusions primarily from a fixed TIGER generative-recommendation protocol, a limited set of datasets and catalog sizes, largely official hyperparameters, and warm-only evaluation. The authors therefore leave unresolved whether the reported empirical patterns generalize to alternative generative backbones and decoding procedures, more systematic tuning, larger catalogs, and cold-start recommendation settings. Establishing this robustness would clarify which findings reflect general properties of semantic-ID design and which depend on the experimental protocol.