Papers
Topics
Authors
Recent
Search
2000 character limit reached

NoteLLM: A Retrievable Large Language Model for Note Recommendation

Published 4 Mar 2024 in cs.IR | (2403.01744v2)

Abstract: People enjoy sharing "notes" including their experiences within online communities. Therefore, recommending notes aligned with user interests has become a crucial task. Existing online methods only input notes into BERT-based models to generate note embeddings for assessing similarity. However, they may underutilize some important cues, e.g., hashtags or categories, which represent the key concepts of notes. Indeed, learning to generate hashtags/categories can potentially enhance note embeddings, both of which compress key note information into limited content. Besides, LLMs have significantly outperformed BERT in understanding natural languages. It is promising to introduce LLMs into note recommendation. In this paper, we propose a novel unified framework called NoteLLM, which leverages LLMs to address the item-to-item (I2I) note recommendation. Specifically, we utilize Note Compression Prompt to compress a note into a single special token, and further learn the potentially related notes' embeddings via a contrastive learning approach. Moreover, we use NoteLLM to summarize the note and generate the hashtag/category automatically through instruction tuning. Extensive validations on real scenarios demonstrate the effectiveness of our proposed method compared with the online baseline and show major improvements in the recommendation system of Xiaohongshu.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. A bi-step grounding paradigm for large language models in recommendation systems. arXiv preprint arXiv:2308.08434 (2023).
  2. TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation. In RecSys.
  3. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
  4. Hashtag-Guided Low-Resource Tweet Classification. In WWW. 1415–1426.
  5. Recommender systems in the era of large language models (llms). arXiv preprint arXiv:2307.02046 (2023).
  6. Large language models as zero-shot conversational recommenders. In CIKM. 720–730.
  7. Large language models are zero-shot rankers for recommender systems. arXiv preprint arXiv:2305.08845 (2023).
  8. Learning deep structured semantic models for web search using clickthrough data. In CIKM. 2333–2338.
  9. Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations. arXiv preprint arXiv:2308.16505 (2023).
  10. Scaling Sentence Embeddings with Large Language Models. arXiv preprint arXiv:2307.16645 (2023).
  11. PromptBERT: Improving BERT Sentence Embeddings with Prompts. In EMNLP. 8826–8837.
  12. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data 7, 3 (2019), 535–547.
  13. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906 (2020).
  14. Hashtag recommendation based on multi-features of microblogs. JCST 33 (2018), 711–726.
  15. Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights. arXiv preprint arXiv:2305.11700 (2023).
  16. User trust in recommendation systems: A comparison of content-based, collaborative and demographic filtering. In CHI. 1–14.
  17. Quantifying and mitigating popularity bias in conversational recommender systems. In CIKM. 1238–1247.
  18. How Can Recommender Systems Benefit from Large Language Models: A Survey. arXiv preprint arXiv:2306.05817 (2023).
  19. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing 7, 1 (2003), 76–80.
  20. Is chatgpt a good recommender? a preliminary study. arXiv preprint arXiv:2304.10149 (2023).
  21. A First Look at LLM-Powered Generative News Recommendation. arXiv preprint arXiv:2305.06566 (2023).
  22. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688 (2023).
  23. Llm-rec: Personalized recommendation via prompting large language models. arXiv preprint arXiv:2307.15780 (2023).
  24. CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models. arXiv preprint arXiv:2401.17043 (2024).
  25. Pre-train a discriminative text encoder for dense retrieval via contrastive span prediction. In SIGIR. 848–858.
  26. Fine-Tuning LLaMA for Multi-Stage Text Retrieval. arXiv preprint arXiv:2310.08319 (2023).
  27. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).
  28. Niklas Muennighoff. 2022. Sgpt: Gpt sentence embeddings for semantic search. arXiv preprint arXiv:2202.08904 (2022).
  29. Large Language Model Augmented Narrative Driven Recommendations. arXiv preprint arXiv:2306.02250 (2023).
  30. Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005 (2022).
  31. OpenAI. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 (2023).
  32. Training language models to follow instructions with human feedback. NeurIPS 35 (2022), 27730–27744.
  33. Large Language Model based Long-tail Query Rewriting in Taobao Search. arXiv preprint arXiv:2311.03758 (2023).
  34. Are gpt embeddings useful for ads and recommendation?. In KSEM. Springer, 151–162.
  35. Juan Ramos et al. 2003. Using tf-idf to determine word relevance in document queries. In Proceedings of the first instructional conference on machine learning, Vol. 242. Citeseer, 29–48.
  36. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In EMNLP-IJCNLP.
  37. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389.
  38. Learning to summarize with human feedback. NeurIPS 33 (2020), 3008–3021.
  39. One embedder, any task: Instruction-finetuned text embeddings. ACL Findings (2023).
  40. An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159 (2018).
  41. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).
  42. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023).
  43. Recmind: Large language model powered agent for recommendation. arXiv preprint arXiv:2308.14296 (2023).
  44. Topic-Aware Neural Keyphrase Generation for Social Media Language. In ACL. 2516–2526.
  45. Microblog hashtag generation via encoding conversation contexts. arXiv preprint arXiv:1905.07584 (2019).
  46. AnyPredict: Foundation Model for Tabular Prediction. arXiv preprint arXiv:2305.12081 (2023).
  47. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021).
  48. Personalized news recommendation: Methods and challenges. ACM Transactions on Information Systems 41, 1 (2023), 1–50.
  49. Empowering news recommendation with pre-trained language models. In SIGIR. 1652–1656.
  50. A Survey on Large Language Models for Recommendation. arXiv preprint arXiv:2305.19860 (2023).
  51. Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models. arXiv preprint arXiv:2306.10933 (2023).
  52. Progressively optimized bi-granular document representation for scalable embedding based retrieval. In WWW. 286–296.
  53. Training large-scale news recommenders with pretrained language models in the loop. In KDD. 4215–4225.
  54. Large Language Models for Generative Information Extraction: A Survey. arXiv preprint arXiv:2312.17617 (2023).
  55. Large scale product graph construction for recommendation in e-commerce. arXiv preprint arXiv:2010.05525 (2020).
  56. A Survey on Multimodal Large Language Models. arXiv preprint arXiv:2306.13549 (2023).
  57. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045 (2023).
  58. Topic Memory Networks for Short Text Classification. In EMNLP. 3120–3131.
  59. Recommendation as instruction following: A large language model empowered recommendation approach. arXiv preprint arXiv:2305.07001 (2023).
  60. Hashtag Recommendation for Multimodal Microblog Using Co-Attention Network.. In IJCAI. 3420–3426.
  61. Keyphrase extraction using deep recurrent neural networks on twitter. In EMNLP. 836–845.
  62. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792 (2023).
  63. Encoding conversation context for neural keyphrase extraction from microblog posts. In NAACL. 1676–1686.
  64. Hierarchical Multi-modal Attention Network for Time-sync Comment Video Recommendation. IEEE Transactions on Circuits and Systems for Video Technology (2023).
  65. Bootstrapping Contrastive Learning Enhanced Music Cold-Start Matching. In WWW. 351–355.
  66. GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond. arXiv preprint arXiv:2309.16583 (2023).
  67. Learning tree-based deep model for recommender systems. In KDD. 1079–1088.
Citations (15)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We found no open problems mentioned in this paper.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 3 tweets with 110 likes about this paper.

Sign Up for Free