Papers
Topics
Authors
Recent
2000 character limit reached

Causality-Enhanced Behavior Sequence Modeling in LLMs for Personalized Recommendation (2410.22809v1)

Published 30 Oct 2024 in cs.IR and cs.AI

Abstract: Recent advancements in recommender systems have focused on leveraging LLMs to improve user preference modeling, yielding promising outcomes. However, current LLM-based approaches struggle to fully leverage user behavior sequences, resulting in suboptimal preference modeling for personalized recommendations. In this study, we propose a novel Counterfactual Fine-Tuning (CFT) method to address this issue by explicitly emphasizing the role of behavior sequences when generating recommendations. Specifically, we employ counterfactual reasoning to identify the causal effects of behavior sequences on model output and introduce a task that directly fits the ground-truth labels based on these effects, achieving the goal of explicit emphasis. Additionally, we develop a token-level weighting mechanism to adjust the emphasis strength for different item tokens, reflecting the diminishing influence of behavior sequences from earlier to later tokens during predicting an item. Extensive experiments on real-world datasets demonstrate that CFT effectively improves behavior sequence modeling. Our codes are available at https://github.com/itsmeyjt/CFT.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. A bi-step grounding paradigm for large language models in recommendation systems. arXiv preprint arXiv:2308.08434 (2023).
  2. Decoding matters: Addressing amplification bias and homogeneity issue for llm-based recommendation. EMNLP (2024).
  3. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems. 1007–1014.
  4. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024).
  5. A multi-view deep learning approach for cross domain user modeling in recommendation systems. In Proceedings of the 24th international conference on world wide web. 278–288.
  6. Causal inference in recommender systems: A survey and future directions. ACM Transactions on Information Systems 42, 4 (2024), 1–32.
  7. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524 (2023).
  8. Causer: Causal session-based recommendations for handling popularity bias. In Proceedings of the 30th ACM international conference on information & knowledge management. 3048–3052.
  9. Addressing Confounding Feature Issue for Causal Recommendation. ACM Trans. Inf. Syst. 41, 3, Article 53 (Feb. 2023), 23 pages. https://doi.org/10.1145/3559757
  10. Session-based Recommendations with Recurrent Neural Networks. In 4th International Conference on Learning Representations.
  11. Large language models are zero-shot rankers for recommender systems. In European Conference on Information Retrieval. Springer, 364–381.
  12. A critical study on data leakage in recommender system offline evaluation. ACM Transactions on Information Systems 41, 3 (2023), 1–27.
  13. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM). IEEE, 197–206.
  14. Do llms understand user preferences? evaluating llms on user rating prediction. arXiv preprint arXiv:2305.06474 (2023).
  15. Propensity matters: Measuring and enhancing balancing for recommendation. In International Conference on Machine Learning. PMLR, 20182–20194.
  16. GPT4Rec: A generative framework for personalized recommendation and user interests interpretation. arXiv preprint arXiv:2304.03879 (2023).
  17. Prompt distillation for efficient llm-based recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 1348–1357.
  18. Causal inference for recommendation. In Causation: Foundation to Application, Workshop at UAI. AUAI, Vol. 6. 108.
  19. How Can Recommender Systems Benefit from Large Language Models: A Survey. ACM Trans. Inf. Syst. (July 2024). https://doi.org/10.1145/3678004 Just Accepted.
  20. Bridging Items and Language: A Transition Paradigm for Large Language Model-Based Recommendation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1816–1826.
  21. Efficient Inference for Large Language Model-based Generative Recommendation. arXiv preprint arXiv:2410.05165 (2024).
  22. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In International Conference on Learning Representations. https://openreview.net/forum?id=Bkg6RiCqY7
  23. Hoang Ngo and Dat Quoc Nguyen. 2024. RecGPT: Generative Pre-training for Text-based Recommendation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024 - Student Research Workshop, Bangkok, Thailand, August 11-16, 2024. Association for Computational Linguistics, 302–313.
  24. Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 188–197.
  25. J Pearl. 2009. Causality. Cambridge university press.
  26. Judea Pearl. 2016. Causal Inference in Statistics: A Primer. John Wiley & Sons.
  27. Recommendation on Live-Streaming Platforms: Dynamic Availability and Repeat Consumption. In Proceedings of the 15th ACM Conference on Recommender Systems (Amsterdam, Netherlands) (RecSys ’21). Association for Computing Machinery, New York, NY, USA, 390–399. https://doi.org/10.1145/3460231.3474267
  28. Unbiased recommender learning from missing-not-at-random implicit feedback. In Proceedings of the 13th International Conference on Web Search and Data Mining. 501–509.
  29. Large Language Models for Intent-Driven Session Recommendations. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR ’24). Association for Computing Machinery, New York, NY, USA, 324–334. https://doi.org/10.1145/3626772.3657688
  30. IDGenRec: LLM-RecSys Alignment with Textual ID Learning. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 355–364.
  31. Jiaxi Tang and Ke Wang. 2018. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (Marina Del Rey, CA, USA) (WSDM ’18). Association for Computing Machinery, New York, NY, USA, 9 pages. https://doi.org/10.1145/3159652.3159656
  32. Counterfactual Video Recommendation for Duration Debiasing. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Long Beach, CA, USA) (KDD ’23). Association for Computing Machinery, New York, NY, USA, 4894–4903. https://doi.org/10.1145/3580305.3599797
  33. Lei Wang and Ee-Peng Lim. 2023. Zero-shot next-item recommendation using large pretrained language models. arXiv preprint arXiv:2304.03153 (2023).
  34. Learnable Item Tokenization for Generative Recommendation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. 2400–2409.
  35. Deconfounded recommendation for alleviating bias amplification. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 1717–1725.
  36. Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 1288–1297. https://doi.org/10.1145/3404835.3462962
  37. RecMind: Large Language Model Powered Agent For Recommendation. In Findings of the Association for Computational Linguistics: NAACL 2024. Association for Computational Linguistics, 4351–4364.
  38. Llmrec: Large language models with graph augmentation for recommendation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining. 806–815.
  39. Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models. In Proceedings of the 18th ACM Conference on Recommender Systems (RecSys ’24). Association for Computing Machinery, New York, NY, USA, 12–22.
  40. A Decoding Acceleration Framework for Industrial Deployable LLM-based Recommender Systems. arXiv preprint arXiv:2408.05676 (2024).
  41. Dually Enhanced Propensity Score Estimation in Sequential Recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (Atlanta, GA, USA) (CIKM ’22). Association for Computing Machinery, New York, NY, USA, 2260–2269. https://doi.org/10.1145/3511808.3557299
  42. Qwen2 Technical Report. arXiv preprint arXiv:2407.10671 (2024).
  43. Deconfounding Duration Bias in Watch-time Prediction for Video Recommendation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Washington DC, USA) (KDD ’22). Association for Computing Machinery, New York, NY, USA, 4472–4481. https://doi.org/10.1145/3534678.3539092
  44. On generative agents in recommendation. In Proceedings of the 47th international ACM SIGIR conference on research and development in Information Retrieval. 1807–1817.
  45. Agentcf: Collaborative learning with autonomous language agents for recommender systems. In Proceedings of the ACM on Web Conference 2024. 3679–3689.
  46. Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach. CoRR abs/2305.07001 (2023). https://doi.org/10.48550/ARXIV.2305.07001
  47. Text-like Encoding of Collaborative Information in Large Language Models for Recommendation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Bangkok, Thailand, 9181–9191. https://doi.org/10.18653/v1/2024.acl-long.497
  48. Causal Intervention for Leveraging Popularity Bias in Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 11–20. https://doi.org/10.1145/3404835.3462875
  49. Collm: Integrating collaborative embeddings into large language models for recommendation. arXiv preprint arXiv:2310.19488 (2023).
  50. Yu Zhang and Qiang Yang. 2021. A survey on multi-task learning. IEEE transactions on knowledge and data engineering 34, 12 (2021), 5586–5609.
  51. Adapting large language models by integrating collaborative semantics for recommendation. In 2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 1435–1448.
  52. Harnessing large language models for text-rich sequential recommendation. In Proceedings of the ACM on Web Conference 2024. 3207–3216.
  53. Collaborative Large Language Model for Recommender Systems. In Proceedings of the ACM Web Conference 2024 (Singapore, Singapore) (WWW ’24). 3162–3172.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - itsmeyjt/CFT: The code of CFT (1 star)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 3 tweets and received 0 likes.

Upgrade to Pro to view all of the tweets about this paper:

Start a free 7-day Pro trial