Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Foundation Models for Recommender Systems: A Survey and New Perspectives (2402.11143v1)

Published 17 Feb 2024 in cs.IR

Abstract: Recently, Foundation Models (FMs), with their extensive knowledge bases and complex architectures, have offered unique opportunities within the realm of recommender systems (RSs). In this paper, we attempt to thoroughly examine FM-based recommendation systems (FM4RecSys). We start by reviewing the research background of FM4RecSys. Then, we provide a systematic taxonomy of existing FM4RecSys research works, which can be divided into four different parts including data characteristics, representation learning, model type, and downstream tasks. Within each part, we review the key recent research developments, outlining the representative models and discussing their characteristics. Moreover, we elaborate on the open problems and opportunities of FM4RecSys aiming to shed light on future research directions in this area. In conclusion, we recap our findings and discuss the emerging trends in this field.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (110)
  1. A bi-step grounding paradigm for large language models in recommendation systems. CoRR, abs/2308.08434, 2023.
  2. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, Singapore, pages 1007–1014. ACM, 2023.
  3. Large language models for recommendation: Progresses and future directions. In Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, Beijing, China, pages 306–309. ACM, 2023.
  4. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020.
  5. On the opportunities and risks of foundation models. CoRR, abs/2108.07258, 2021.
  6. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.
  7. Instruction mining: High-quality instruction data selection for large language models. CoRR, abs/2307.06290, 2023.
  8. The lottery ticket hypothesis for pre-trained BERT networks. In Advances in Neural Information Processing Systems: Annual Conference on Neural Information Processing Systems, virtual, 2020.
  9. Recommendation unlearning. In The ACM Web Conference, Virtual Event, Lyon, France, pages 2768–2777. ACM, 2022.
  10. Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning. CoRR, abs/2305.09246, 2023.
  11. M6-rec: Generative pretrained language models are open-ended recommender systems. arXiv preprint arXiv:2205.08084, 2022.
  12. Uncovering chatgpt’s capabilities in recommender systems. In Proceedings of the 17th ACM Conference on Recommender Systems, Singapore, pages 1126–1132. ACM, 2023.
  13. Attack prompt generation for red teaming and defending large language models. In Findings of the Association for Computational Linguistics, Singapore, pages 2176–2189. Association for Computational Linguistics, 2023.
  14. Zero-shot recommender systems. CoRR, abs/2105.08318, 2021.
  15. Agent ai: Surveying the horizons of multimodal interaction. arXiv preprint arXiv:2401.03568, 2024.
  16. Recommender systems in the era of large language models (llms). CoRR, abs/2307.02046, 2023.
  17. Exploring adapter-based transfer learning for recommender systems: Empirical studies and practical insights. CoRR, abs/2305.15036, 2023.
  18. A unified framework for multi-domain CTR prediction via large language models. CoRR, abs/2312.10743, 2023.
  19. Advances and challenges in conversational recommender systems: A survey. AI Open, 2:100–126, 2021.
  20. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524, 2023.
  21. Retrieval-augmented generation for large language models: A survey. CoRR, abs/2312.10997, 2023.
  22. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In Proceedings of the 16th ACM Conference on Recommender Systems, pages 299–315, 2022.
  23. Vip5: Towards multimodal foundation models for recommendation. arXiv preprint arXiv:2305.14302, 2023.
  24. Span selection pre-training for question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pages 2773–2782. Association for Computational Linguistics, 2020.
  25. An unified search and recommendation foundation model for cold-start scenario. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Birmingham, United Kingdom, pages 4595–4601. ACM, 2023.
  26. Large language models are zero-shot time series forecasters. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  27. Leveraging large language models for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, Singapore, pages 1096–1102. ACM, 2023.
  28. Large language models as zero-shot conversational recommenders. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Birmingham, United Kingdom, pages 720–730. ACM, 2023.
  29. Learning vector-quantized item representation for transferable sequential recommenders. In Proceedings of the ACM Web Conference, pages 1162–1171, 2023.
  30. Large language models are zero-shot rankers for recommender systems. arXiv preprint arXiv:2305.08845, 2023.
  31. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR, 2022.
  32. In-context analogical reasoning with pre-trained language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Toronto, Canada, July 9-14, 2023, pages 1953–1969. Association for Computational Linguistics, 2023.
  33. UP5: unbiased foundation model for fairness-aware recommendation. CoRR, abs/2305.12090, 2023.
  34. Tutorial on large language models for recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, Singapore, pages 1281–1283. ACM, 2023.
  35. How to index item ids for recommendation foundation models. arXiv preprint arXiv:2305.06569, 2023.
  36. Recommender AI agent: Integrating large language models for interactive recommendations. CoRR, abs/2308.16505, 2023.
  37. Recsim: A configurable simulation platform for recommender systems. CoRR, abs/1909.04847, 2019.
  38. A survey on conversational recommender systems. ACM Comput. Surv., 54(5):105:1–105:36, 2022.
  39. Online distillation-enhanced multi-modal transformer for sequential recommendation. In Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada, pages 955–965. ACM, 2023.
  40. Tinybert: Distilling BERT for natural language understanding. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, Findings of ACL, pages 4163–4174. Association for Computational Linguistics, 2020.
  41. Large models for time series and spatio-temporal data: A survey and outlook. CoRR, abs/2310.10196, 2023.
  42. Reformer: The efficient transformer. In International Conference on Learning Representations, 2019.
  43. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In WSDM ’20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, pages 304–312. ACM, 2020.
  44. Generate neural template explanations for recommendation. In The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, pages 755–764. ACM, 2020.
  45. Text is all you need: Learning language representations for sequential recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, CA, USA, pages 1258–1267. ACM, 2023.
  46. Large language models for generative recommendation: A survey and visionary discussions. CoRR, abs/2309.01157, 2023.
  47. HAMUR: hyper adapter for multi-domain recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Birmingham, United Kingdom, pages 1268–1277. ACM, 2023.
  48. A preliminary study of chatgpt on news recommendation: Personalization, provider fairness, and fake news. In Benjamin Kille, editor, Proceedings of the International Workshop on News Recommendation and Analytics co-located with the 2023 ACM Conference on Recommender Systems, Singapore, volume 3561 of CEUR Workshop Proceedings. CEUR-WS.org, 2023.
  49. Loftq: Lora-fine-tuning-aware quantization for large language models. CoRR, abs/2310.08659, 2023.
  50. Llara: Aligning large language models with sequential recommenders. CoRR, abs/2312.02445, 2023.
  51. Sparks of artificial general recommender (AGR): early experiments with chatgpt. CoRR, abs/2305.04518, 2023.
  52. AWQ: activation-aware weight quantization for LLM compression and acceleration. CoRR, abs/2306.00978, 2023.
  53. How can recommender systems benefit from large language models: A survey. CoRR, abs/2306.05817, 2023.
  54. Rella: Retrieval-enhanced large language models for lifelong sequential behavior comprehension in recommendation. CoRR, abs/2308.11131, 2023.
  55. A multi-facet paradigm to bridge large language model and recommendation. CoRR, abs/2310.06491, 2023.
  56. Is chatgpt a good recommender? a preliminary study. arXiv preprint arXiv:2304.10149, 2023.
  57. Llmrec: Benchmarking large language models on recommendation task. CoRR, abs/2308.12241, 2023.
  58. Pre-train, prompt and recommendation: A comprehensive survey of language modelling paradigm adaptations in recommender systems. CoRR, abs/2302.03735, 2023.
  59. A first look at llm-powered generative news recommendation. arXiv preprint arXiv:2305.06566, 2023.
  60. Text matching improves sequential recommendation by reducing popularity biases. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Birmingham, United Kingdom, pages 1534–1544. ACM, 2023.
  61. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, pages 9802–9822. Association for Computational Linguistics, 2023.
  62. Language-guided music recommendation for video via prompt analogies. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, pages 14784–14793. IEEE, 2023.
  63. Learning to compress prompts with gist tokens. CoRR, abs/2304.08467, 2023.
  64. OpenAI. Gpt-4 technical report. OpenAI, 2023.
  65. Logic-scaffolding: Personalized aspect-instructed recommendation explanation generation using llms. CoRR, abs/2312.14345, 2023.
  66. Recommender systems: Introduction and challenges. In Recommender Systems Handbook, pages 1–34. Springer, 2015.
  67. Outfittransformer: Learning outfit representations for fashion recommendation. In IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, pages 3590–3598. IEEE, 2023.
  68. Autotransition: Learning to recommend video transition effects. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, volume 13698 of Lecture Notes in Computer Science, pages 285–300. Springer, 2022.
  69. Chatgpt for conversational recommendation: Refining recommendations by reprompting with feedback. CoRR, abs/2401.03605, 2024.
  70. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024.
  71. Yueming Sun and Yi Zhang. Conversational recommender system. In Kevyn Collins-Thompson, Qiaozhu Mei, Brian D. Davison, Yiqun Liu, and Emine Yilmaz, editors, The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, MI, USA, pages 235–244. ACM, 2018.
  72. A survey of reasoning with foundation models. CoRR, abs/2312.11562, 2023.
  73. One model for all: Large language models are domain-agnostic recommendation systems. CoRR, abs/2310.14304, 2023.
  74. Finetuning large-scale pre-trained language models for conversational recommendation with knowledge graph. CoRR, abs/2110.07477, 2021.
  75. Missrec: Pre-training and transferring multi-modal interest-aware sequence representation for recommendation. In Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada, pages 6548–6557. ACM, 2023.
  76. Recagent: A novel simulation paradigm for recommender systems, 2023.
  77. Rethinking the evaluation for conversational recommendation in the era of large language models. arXiv preprint arXiv:2305.13112, 2023.
  78. Enhancing recommender systems with large language model reasoning graphs. CoRR, abs/2308.10835, 2023.
  79. Recmind: Large language model powered agent for recommendation. arXiv preprint arXiv:2308.14296, 2023.
  80. DRDT: dynamic reflection with divergent thinking for llm-based sequential recommendation. CoRR, abs/2312.11336, 2023.
  81. Aligning large language models with human: A survey. CoRR, abs/2307.12966, 2023.
  82. Ptum: Pre-training user model from unlabeled user behaviors via self-supervision. arXiv preprint arXiv:2010.01494, 2020.
  83. Personalized prompts for sequential recommendation. CoRR, abs/2205.09666, 2022.
  84. A survey on large language models for recommendation. CoRR, abs/2305.19860, 2023.
  85. Towards open-world recommendation with knowledge augmentation from large language models. CoRR, abs/2306.10933, 2023.
  86. Data selection for language models via importance resampling. CoRR, abs/2302.03169, 2023.
  87. Cvalues: Measuring the values of chinese large language models from safety to responsibility. CoRR, abs/2307.09705, 2023.
  88. Openp5: Benchmarking foundation models for recommendation. CoRR, abs/2306.11134, 2023.
  89. Prompting large language models for recommender systems: A comprehensive framework and empirical analysis. CoRR, abs/2401.04997, 2024.
  90. Knowledge plugins: Enhancing large language models for domain-specific recommendations. CoRR, abs/2311.10779, 2023.
  91. Clip-actor: Text-driven recommendation and stylization for animating human meshes. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, volume 13663 of Lecture Notes in Computer Science, pages 173–191. Springer, 2022.
  92. Untargeted attack against federated recommendation systems via poisonous item embeddings and the defense. In Thirty-Seventh AAAI Conference on Artificial Intelligence, Washington, DC, USA, pages 4854–4863. AAAI Press, 2023.
  93. Where to go next for recommender systems? ID- vs. modality-based recommender models revisited. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Taipei, Taiwan, pages 2639–2649. ACM, 2023.
  94. Knowledge prompt-tuning for sequential recommendation. In Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada, pages 6451–6461. ACM, 2023.
  95. Yongfeng Zhang and Xu Chen. Explainable recommendation: A survey and new perspectives. Found. Trends Inf. Retr., 14(1):1–101, 2020.
  96. Prompt learning for news recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Taipei, Taiwan, pages 227–237. ACM, 2023.
  97. Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv., 52(1):5:1–5:38, 2019.
  98. Explainable recommendation: A survey and new perspectives. Foundations and Trends® in Information Retrieval, 14(1):1–101, 2020.
  99. On generative agents in recommendation. CoRR, abs/2310.10108, 2023.
  100. Interactive interior design recommendation via coarse-to-fine multimodal reinforcement learning. In Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada, pages 6472–6480. ACM, 2023.
  101. Is chatgpt fair for recommendation? evaluating fairness in large language model recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, Singapore, pages 993–999. ACM, 2023.
  102. Agentcf: Collaborative learning with autonomous language agents for recommender systems. CoRR, abs/2310.09233, 2023.
  103. Recommendation as instruction following: A large language model empowered recommendation approach. CoRR, abs/2305.07001, 2023.
  104. Gangyi Zhang. User-centric conversational recommendation: Adapting the need of user with large language models. In Jie Zhang, Li Chen, Shlomo Berkovsky, Min Zhang, Tommaso Di Noia, Justin Basilico, Luiz Pizzato, and Yang Song, editors, Proceedings of the 17th ACM Conference on Recommender Systems, Singapore, pages 1349–1354. ACM, 2023.
  105. Improving conversational recommender systems via knowledge graph based semantic fusion. In The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, CA, USA, pages 1006–1014. ACM, 2020.
  106. LIMA: less is more for alignment. CoRR, abs/2305.11206, 2023.
  107. Exploring recommendation capabilities of gpt-4v(ision): A preliminary case study. CoRR, abs/2311.04199, 2023.
  108. What to do next: Modeling user behaviors by time-lstm. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, Melbourne, pages 3602–3608, 2017.
  109. Cross-domain recommendation: Challenges, progress, and prospects. In Zhi-Hua Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, Virtual Event / Montreal, Canada, pages 4721–4728. ijcai.org, 2021.
  110. Collaborative large language model for recommender systems. CoRR, abs/2311.01343, 2023.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now