Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning (2402.04401v2)

Published 6 Feb 2024 in cs.CL

Abstract: Personalization in LLMs is increasingly important, aiming to align the LLMs' interactions, content, and recommendations with individual user preferences. Recent advances have highlighted effective prompt design by enriching user queries with non-parametric knowledge through behavior history retrieval and textual profiles. However, these methods faced limitations due to a lack of model ownership, resulting in constrained customization and privacy issues, and often failed to capture complex, dynamic user behavior patterns. To address these shortcomings, we introduce One PEFT Per User (OPPU), employing personalized parameter-efficient fine-tuning (PEFT) modules to store user-specific behavior patterns and preferences. By plugging in personal PEFT parameters, users can own and use their LLMs individually. OPPU integrates parametric user knowledge in the personal PEFT parameters with non-parametric knowledge from retrieval and profiles, adapting LLMs to user behavior shifts. Experimental results demonstrate that OPPU significantly outperforms existing prompt-based methods across seven diverse tasks in the LaMP benchmark. Further studies reveal OPPU's enhanced capabilities in handling user behavior shifts, modeling users at different activity levels, maintaining robustness across various user history formats, and displaying versatility with different PEFT methods.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Learning technology models that support personalization within blended learning environments in higher education. TechTrends 65 (2021), 62–78.
  2. Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestion. arXiv preprint arXiv:2311.06318 (2023).
  3. Junyi Chen. 2023. A Survey on Large Language Models for Personalized and Explainable Recommendations. arXiv preprint arXiv:2311.12338 (2023).
  4. When large language models meet personalization: Perspectives of challenges and opportunities. arXiv preprint arXiv:2307.16376 (2023).
  5. Everyone Deserves A Reward: Learning Customized Human Preferences. arXiv:2309.03126 [cs.CL]
  6. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023) (2023).
  7. Large Language Models for User Interest Journeys. arXiv preprint arXiv:2305.15498 (2023).
  8. Uncovering ChatGPT’s Capabilities in Recommender Systems. arXiv preprint arXiv:2305.02182 (2023).
  9. Approaches to Laparoscopic Training in Veterinary Medicine: A Review of Personalized Simulators. Animals 13, 24 (2023), 3781.
  10. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. arXiv preprint arXiv:2208.07339 (2022).
  11. On the effectiveness of parameter-efficient fine-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 12799–12807.
  12. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524 (2023).
  13. Personalization in practice: Methods and applications. In Proceedings of the 14th ACM international conference on web search and data mining. 1123–1126.
  14. Hierarchical user profiling for e-commerce recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining. 223–231.
  15. RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture. arXiv preprint arXiv:2401.08406 (2024).
  16. Array programming with NumPy. Nature 585, 7825 (2020), 357–362.
  17. Towards a Unified View of Parameter-Efficient Transfer Learning. In International Conference on Learning Representations.
  18. DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. In The Eleventh International Conference on Learning Representations.
  19. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning. PMLR, 2790–2799.
  20. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations.
  21. Personalized soups: Personalized large language model alignment via post-hoc parameter merging. arXiv preprint arXiv:2310.11564 (2023).
  22. Precision medicine, AI, and the future of personalized health care. Clinical and translational science 14, 1 (2021), 86–93.
  23. Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction. arXiv:2305.06474 [cs.IR]
  24. The Power of Scale for Parameter-Efficient Prompt Tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 3045–3059.
  25. Teach LLMs to Personalize–An Approach inspired by Writing Education. arXiv preprint arXiv:2308.07968 (2023).
  26. Text Is All You Need: Learning Language Representations for Sequential Recommendation. arXiv preprint arXiv:2305.13731 (2023).
  27. Xiang Lisa Li and Percy Liang. 2021. Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, Online, 4582–4597. https://doi.org/10.18653/v1/2021.acl-long.353
  28. Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74–81. https://aclanthology.org/W04-1013
  29. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems 35 (2022), 1950–1965.
  30. Is chatgpt a good recommender? a preliminary study. arXiv preprint arXiv:2304.10149 (2023).
  31. ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models. arXiv:2305.06566 [cs.IR]
  32. P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ireland, 61–68. https://doi.org/10.18653/v1/2022.acl-short.8
  33. Are Emergent Abilities in Large Language Models just In-Context Learning? arXiv preprint arXiv:2309.01809 (2023).
  34. One chatbot per person: Creating personalized chatbots based on implicit user profiles. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 555–564.
  35. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 11048–11064.
  36. PEARL: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers. arXiv preprint arXiv:2311.09180 (2023).
  37. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730–27744.
  38. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).
  39. How context affects language models’ factual predictions. arXiv preprint arXiv:2005.04611 (2020).
  40. AdapterFusion: Non-Destructive Task Composition for Transfer Learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 487–503.
  41. Revolutionizing education: harnessing the power of artificial intelligence for personalized learning. Klasikal: Journal of Education, Language Teaching and Science 5, 2 (2023), 350–357.
  42. Personalized recommendation combining user interest and social circle. IEEE transactions on knowledge and data engineering 26, 7 (2013), 1763–1777.
  43. Integrating summarization and retrieval for enhanced personalization via large language models. arXiv preprint arXiv:2310.20081 (2023).
  44. LaMP: When Large Language Models Meet Personalization. arXiv preprint arXiv:2304.11406 (2023).
  45. Multitask Prompted Training Enables Zero-Shot Task Generalization. In ICLR 2022-Tenth International Conference on Learning Representations.
  46. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning. PMLR, 31210–31227.
  47. Personalized chatbot trustworthiness ratings. IEEE Transactions on Technology and Society 1, 4 (2020), 184–192.
  48. Zhaoxuan Tan and Meng Jiang. 2023. User Modeling in the Era of Large Language Models: Current Research and Future Directions. arXiv preprint arXiv:2312.11518 (2023).
  49. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023).
  50. Improvements to BM25 and language models examined. In Proceedings of the 19th Australasian Document Computing Symposium. 58–65.
  51. Learning personalized story evaluation. arXiv preprint arXiv:2310.03304 (2023).
  52. Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 5085–5109.
  53. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021).
  54. Emergent Abilities of Large Language Models. Transactions on Machine Learning Research (2022).
  55. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837.
  56. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations. 38–45.
  57. Personalized news recommendation: Methods and challenges. ACM Transactions on Information Systems 41, 1 (2023), 1–50.
  58. Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment. arXiv preprint arXiv:2312.12148 (2023).
  59. Shuo Zhang and Krisztian Balog. 2020. Evaluating conversational recommender systems via user simulation. In Proceedings of the 26th acm sigkdd international conference on knowledge discovery & data mining. 1512–1520.
  60. BookGPT: A General Framework for Book Recommendation Empowered by Large Language Model. arXiv preprint arXiv:2305.15673 (2023).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Zhaoxuan Tan (35 papers)
  2. Qingkai Zeng (28 papers)
  3. Yijun Tian (29 papers)
  4. Zheyuan Liu (35 papers)
  5. Bing Yin (56 papers)
  6. Meng Jiang (126 papers)
Citations (23)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets