Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation (2306.14462v1)

Published 26 Jun 2023 in cs.IR and cs.LG

Abstract: Recommendation systems suffer in the strict cold-start (SCS) scenario, where the user-item interactions are entirely unavailable. The ID-based approaches completely fail to work. Cold-start recommenders, on the other hand, leverage item contents to map the new items to the existing ones. However, the existing SCS recommenders explore item contents in coarse-grained manners that introduce noise or information loss. Moreover, informative data sources other than item contents, such as users' purchase sequences and review texts, are ignored. We explore the role of the fine-grained item attributes in bridging the gaps between the existing and the SCS items and pre-train a knowledgeable item-attribute graph for SCS item recommendation. Our proposed framework, ColdGPT, models item-attribute correlations into an item-attribute graph by extracting fine-grained attributes from item contents. ColdGPT then transfers knowledge into the item-attribute graph from various available data sources, i.e., item contents, historical purchase sequences, and review texts of the existing items, via multi-task learning. To facilitate the positive transfer, ColdGPT designs submodules according to the natural forms of the data sources and coordinates the multiple pre-training tasks via unified alignment-and-uniformity losses. Our pre-trained item-attribute graph acts as an implicit, extendable item embedding matrix, which enables the SCS item embeddings to be easily acquired by inserting these items and propagating their attributes' embeddings. We carefully process three public datasets, i.e., Yelp, Amazon-home, and Amazon-sports, to guarantee the SCS setting for evaluation. Extensive experiments show that ColdGPT consistently outperforms the existing SCS recommenders by large margins and even surpasses models that are pre-trained on 75-224 times more, cross-domain data on two out of four datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. CB2CF: a neural multiview content-to-collaborative filtering model for completely cold item recommendations. In Proc. RecSys.
  2. Language models are few-shot learners. In Proc. NeurIPS.
  3. Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences. In Proc. TheWebConf.
  4. Rich Caruana. 1997. Multitask learning. Machine learning 28, 1 (1997), 41–75.
  5. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In Proc. ICML.
  6. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. Proc. NeurIPS.
  7. Wide & deep learning for recommender systems. In Proc. DLRS.
  8. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
  9. Mamo: Memory-augmented meta-optimization for cold-start recommendation. In Proc. SIGKDD.
  10. Metakg: Meta-learning on knowledge graph for cold-start recommendation. IEEE TKDE (2022).
  11. Graph neural networks for social recommendation. In Proc. TheWebConf.
  12. Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training. arXiv preprint arXiv:2305.07633 (2023).
  13. Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5). In Proc. RecSys.
  14. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proc. SIGKDD.
  15. Michael Gutmann and Aapo Hyvärinen. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proc. AISTATS.
  16. Pre-training graph neural networks for cold-start users and items representation. In Proc. WSDM.
  17. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proc. SIGIR.
  18. Towards Universal Sequence Representation Learning for Recommender Systems. In Proc. SIGKDD.
  19. Strategies for pre-training graph neural networks. Proc. ICLR.
  20. Gpt-gnn: Generative pre-training of graph neural networks. In Proc. SIGKDD.
  21. Unsupervised pre-training of graph convolutional networks. In ICLR Workshop: Representation Learning on Graphs and Manifolds.
  22. Adrián Javaloy and Isabel Valera. 2022. RotoGrad: Gradient Homogenization in Multitask Learning. Proc. ICLR.
  23. Understanding dimensional collapse in contrastive self-supervised learning. arXiv preprint arXiv:2110.09348 (2021).
  24. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In Proc. ICDM.
  25. Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proc. ICLR.
  26. In defense of the unitary scalarization for deep multi-task learning. Proc. NeurIPS.
  27. Melu: Meta-learned user preference estimator for cold-start recommendation. In Proc. SIGKDD.
  28. From zero-shot learning to cold-start recommendation. In Proc. AAAI.
  29. Task-adaptive neural process for user cold-start recommendation. In Proc. TheWebConf.
  30. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021).
  31. Graph Neural Pre-Training for Recommendation with Side Information. ACM Trans. Inf. Syst. (dec 2022).
  32. Leveraging distribution alignment via stein path for cross-domain cold-start recommendation. Proc. NeurIPS.
  33. Augmenting sequential recommendation with pseudo-prior items via reversely pre-training transformer. In Proc. SIGIR.
  34. Meta-learning on heterogeneous information networks for cold-start recommendation. In Proc. SIGKDD.
  35. Learning to pre-train graph neural networks. In Proc. AAAI.
  36. Distributed representations of words and phrases and their compositionality. Proc. NeurIPS.
  37. Warm up cold-start advertisements: Improving ctr predictions via learning to learn id embeddings. In Proc. SIGIR.
  38. Pnmta: A pretrained network modulation and task adaptation approach for user cold-start recommendation. In Proc. TheWebConf.
  39. Aleksandr Petrov and Craig Macdonald. 2022. A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation. In Proc. RecSys.
  40. Attribute graph neural networks for strict cold start recommendation. IEEE TKDE (2020).
  41. Gcc: Graph contrastive coding for graph neural network pre-training. In Proc. SIGKDD.
  42. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 140 (2020), 1–67.
  43. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proc. EMNLP.
  44. Steffen Rendle. 2010. Factorization machines. In Proc. ICDM. IEEE.
  45. BPR: Bayesian personalized ranking from implicit feedback. In Proc. UAI.
  46. Ozan Sener and Vladlen Koltun. 2018. Multi-task learning as multi-objective optimization. Proc. NeurIPS.
  47. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proc. CIKM.
  48. Gppt: Graph pre-training and prompt tuning to generalize graph neural networks. In Proc. SIGKDD.
  49. Line: Large-scale information network embedding. In Proc. TheWebConf.
  50. Ext5: Towards extreme multi-task scaling for transfer learning. Proc. ICLR.
  51. A meta-learning perspective on cold-start recommendations for items. In Proc. NeurIPS.
  52. Attention is all you need. Proc. NeurIPS.
  53. Graph attention networks. In Proc. ICLR.
  54. Dropoutnet: Addressing cold start in recommender systems. In Proc. NeurIPS.
  55. Pre-training graph neural network for cross domain recommendation. In 2021 IEEE Third International Conference on Cognitive Machine Intelligence (CogMI). IEEE, 140–145.
  56. Towards Representation Alignment and Uniformity in Collaborative Filtering. In Proc. SIGKDD.
  57. Knowledge graph convolutional networks for recommender systems. In Proc. TheWebConf.
  58. MetaKRec: Collaborative Meta-Knowledge Enhanced Recommender System. In Proc. BigData. IEEE.
  59. Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proc. ICML.
  60. Kgat: Knowledge graph attention network for recommendation. In Proc. SIGKDD.
  61. Neural graph collaborative filtering. In Proc. SIGIR.
  62. Contrastive learning for cold-start recommendation. In Proc. Multimedia.
  63. Self-supervised graph learning for recommendation. In Proc. SIGIR.
  64. Towards open-world recommendation: An inductive model-based collaborative filtering approach. In Proc. ICML.
  65. Attentional factorization machines: Learning the weight of feature interactions via attention networks. In Proc. IJCAI.
  66. Do Current Multi-Task Optimization Methods in Deep Learning Even Help? Proc. NeurIPS.
  67. Consisrec: Enhancing gnn for social recommendation via consistent neighbor aggregation. In Proc. SIGIR.
  68. Mine your own anatomy: Revisiting medical image segmentation with extremely limited labels. arXiv preprint arXiv:2209.13476 (2022).
  69. ACTION++: Improving Semi-supervised Medical Image Segmentation with Adaptive Anatomical Contrast. arXiv preprint arXiv:2304.02689 (2023).
  70. Bootstrapping semi-supervised medical image segmentation with anatomical-aware contrastive distillation. In Proc. IPMI. Springer.
  71. Rethinking semi-supervised medical image segmentation: A variance-reduction perspective. arXiv preprint arXiv:2302.01735 (2023).
  72. Gradient surgery for multi-task learning. Proc. NeurIPS.
  73. Where to Go Next for Recommender Systems? ID- vs. Modality-based recommender models revisied. In Proc. SIGIR.
  74. Do users rate or review? Boost phrase-level sentiment labeling with review-level sentiment classification. In Proc. SIGIR.
  75. Improving Item Cold-start Recommendation via Model-agnostic Conditional Variational Autoencoder. In Proc. SIGIR.
  76. Cold brew: Distilling graph node representations with incomplete or missing neighborhoods. In Proc. ICLR.
  77. Learning to warm up cold item embeddings for cold-start recommendation with meta scaling and shifting networks. In Proc. SIGIR.
  78. Recommendation for new users and new items via randomized training and mixture-of-experts transformation. In Proc. SIGIR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Yuwei Cao (13 papers)
  2. Liangwei Yang (46 papers)
  3. Chen Wang (600 papers)
  4. Zhiwei Liu (114 papers)
  5. Hao Peng (291 papers)
  6. Chenyu You (66 papers)
  7. Philip S. Yu (592 papers)
Citations (16)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now