Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LiMAML: Personalization of Deep Recommender Models via Meta Learning (2403.00803v1)

Published 23 Feb 2024 in cs.IR, cs.AI, and cs.LG

Abstract: In the realm of recommender systems, the ubiquitous adoption of deep neural networks has emerged as a dominant paradigm for modeling diverse business objectives. As user bases continue to expand, the necessity of personalization and frequent model updates have assumed paramount significance to ensure the delivery of relevant and refreshed experiences to a diverse array of members. In this work, we introduce an innovative meta-learning solution tailored to the personalization of models for individual members and other entities, coupled with the frequent updates based on the latest user interaction signals. Specifically, we leverage the Model-Agnostic Meta Learning (MAML) algorithm to adapt per-task sub-networks using recent user interaction data. Given the near infeasibility of productionizing original MAML-based models in online recommendation systems, we propose an efficient strategy to operationalize meta-learned sub-networks in production, which involves transforming them into fixed-sized vectors, termed meta embeddings, thereby enabling the seamless deployment of models with hundreds of billions of parameters for online serving. Through extensive experimentation on production data drawn from various applications at LinkedIn, we demonstrate that the proposed solution consistently outperforms the baseline models of those applications, including strong baselines such as using wide-and-deep ID based personalization approach. Our approach has enabled the deployment of a range of highly personalized AI models across diverse LinkedIn applications, leading to substantial improvements in business metrics as well as refreshed experience for our members.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Yi Wang Aden. 2012. KDD Cup 2012, Track 2. https://kaggle.com/competitions/kddcup2012-track2
  2. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7–10.
  3. Sequential scenario-specific meta learner for online recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2895–2904.
  4. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 1126–1135. https://proceedings.mlr.press/v70/finn17a.html
  5. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 (2017).
  6. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247 (2017).
  7. F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) 5, 4 (2015), 1–19.
  8. Meta-Learning Online Adaptation of Language Models. arXiv preprint arXiv:2305.15076 (2023).
  9. TIANZE HU. 2021. Hybrid Meta-Learning for Cold-Start Recommendation. (2021).
  10. Meta-learning for online update of recommender systems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 4065–4074.
  11. Meta-Learning with Adaptive Weighted Loss for Imbalanced Cold-Start Recommendation. arXiv preprint arXiv:2302.14640 (2023).
  12. Melu: Meta-learned user preference estimator for cold-start recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1073–1082.
  13. Metaselector: Meta-learning for recommendation with user-level adaptive model selection. In Proceedings of The Web Conference 2020. 2507–2513.
  14. Deep learning recommendation model for personalization and recommendation systems. arXiv preprint arXiv:1906.00091 (2019).
  15. A dynamic meta-learning model for time-sensitive cold-start recommendations. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 7868–7876.
  16. Learning graph meta embeddings for cold-start ads in click-through rate prediction. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1157–1166.
  17. Warm up cold-start advertisements: Improving ctr predictions via learning to learn id embeddings. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 695–704.
  18. PNMTA: A pretrained network modulation and task adaptation approach for user cold-start recommendation. In Proceedings of the ACM Web Conference 2022. 348–359.
  19. Learning an adaptive meta model-generator for incrementally updating recommender systems. In Proceedings of the 15th ACM Conference on Recommender Systems. 411–421.
  20. Multi-objective Optimization of Notifications Using Offline Reinforcement Learning. arXiv:2207.03029 [cs.LG]
  21. Product-based neural networks for user response prediction. In 2016 IEEE 16th international conference on data mining (ICDM). IEEE, 1149–1154.
  22. Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv preprint arXiv:1909.09157 (2019).
  23. Meta-learning with implicit gradients. Advances in neural information processing systems 32 (2019).
  24. Meta-learning with memory-augmented neural networks. In International conference on machine learning. PMLR, 1842–1850.
  25. Prototypical networks for few-shot learning. Advances in neural information processing systems 30 (2017).
  26. A meta-learning perspective on cold-start recommendations for items. Advances in neural information processing systems 30 (2017).
  27. Attention is all you need. Advances in neural information processing systems 30 (2017).
  28. Deep Meta-learning in Recommendation Systems: A Survey. arXiv:2206.04415 [cs.IR]
  29. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. In Proceedings of the web conference 2021. 1785–1797.
  30. G-Meta: Distributed Meta Learning in GPU Clusters for Large-Scale Recommender Systems. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 4365–4369.
  31. Personalized adaptive meta learning for cold-start user preference prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 10772–10780.
  32. Learning and transferring ids representation in e-commerce. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1031–1039.
  33. Improving recommendation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web. 22–32.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now