D2K: Turning Historical Data into Retrievable Knowledge for Recommender Systems (2401.11478v2)
Abstract: A vast amount of user behavior data is constantly accumulating on today's large recommendation platforms, recording users' various interests and tastes. Preserving knowledge from the old data while new data continually arrives is a vital problem for recommender systems. Existing approaches generally seek to save the knowledge implicitly in the model parameters. However, such a parameter-centric approach lacks scalability and flexibility -- the capacity is hard to scale, and the knowledge is inflexible to utilize. Hence, in this work, we propose a framework that turns massive user behavior data to retrievable knowledge (D2K). It is a data-centric approach that is model-agnostic and easy to scale up. Different from only storing unary knowledge such as the user-side or item-side information, D2K propose to store ternary knowledge for recommendation, which is determined by the complete recommendation factors -- user, item, and context. The knowledge retrieved by target samples can be directly used to enhance the performance of any recommendation algorithms. Specifically, we introduce a Transformer-based knowledge encoder to transform the old data into knowledge with the user-item-context cross features. A personalized knowledge adaptation unit is devised to effectively exploit the information from the knowledge base by adapting the retrieved knowledge to the target samples. Extensive experiments on two public datasets show that D2K significantly outperforms existing baselines and is compatible with a major collection of recommendation algorithms.
- Gediminas Adomavicius and Alexander Tuzhilin. 2011. Context-aware recommender systems. In Recommender systems handbook. Springer, 217–253.
- CAN: Feature Co-Action Network for Click-Through Rate Prediction. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 57–65.
- Coresets via bilevel optimization for continual learning and streaming. Advances in Neural Information Processing Systems 33 (2020), 14879–14890.
- Sequential recommendation with user memory networks. In Proceedings of the eleventh ACM international conference on web search and data mining. 108–116.
- Selection via proxy: Efficient data selection for deep learning. arXiv preprint arXiv:1906.11829 (2019).
- POSO: Personalized Cold Start Modules for Large-scale Recommender Systems. arXiv preprint arXiv:2108.04690 (2021).
- A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).
- Neural turing machines. arXiv preprint arXiv:1410.5401 (2014).
- Hybrid computing using a neural network with dynamic external memory. Nature 538, 7626 (2016), 471–476.
- Deepfm: a factorization-machine based neural network for ctr prediction. In IJCAI.
- Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks. In SIGIR.
- Ask me anything: Dynamic memory networks for natural language processing. In International conference on machine learning. PMLR, 1378–1387.
- AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction. In KDD.
- Ader: Adaptively distilled exemplar replay towards continual learning for session-based recommendation. In Fourteenth ACM Conference on Recommender Systems. 408–413.
- Incremental Learning for Personalized Recommender Systems. arXiv preprint arXiv:2108.13299 (2021).
- Continual lifelong learning with neural networks: A review. Neural Networks 113 (2019), 54–71.
- Practice on long sequential user behavior modeling for click-through rate prediction. In KDD. 2671–2679.
- Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction. In CIKM.
- Learning to Retrieve User Behaviors for Click-Through Rate Estimation. ACM Transactions on Information Systems (2023).
- User Behavior Retrieval for Click-Through Rate Prediction. In SIGIR.
- Product-based neural networks for user response prediction over multi-field categorical data. TOIS 37, 1 (2018), 1–35.
- Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction. SIGIR.
- On sampling collaborative filtering datasets. arXiv preprint arXiv:2201.04768 (2022).
- Attention is all you need. In Advances in neural information processing systems. 5998–6008.
- Display Advertising with Real-Time Bidding (RTB) and Behavioural Targeting. Now Publisher (2017).
- Neural memory streaming recommender networks with adversarial training. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2467–2475.
- Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. In Proceedings of the Web Conference 2021. 1785–1797.
- A practical incremental method to train deep ctr models. arXiv preprint arXiv:2009.02147 (2020).
- Structure aware incremental learning with personalized imitation weights for recommender systems. In AAAI.
- Graphsail: Graph structure aware incremental learning for recommender systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2861–2868.
- One person, one model, one world: Learning continual user representation without forgetting. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 696–705.
- How to retrain recommender system? A sequential meta-learning method. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1479–1488.
- Deep interest network for click-through rate prediction. In KDD.
- Jiarui Qin (24 papers)
- Weiwen Liu (59 papers)
- Ruiming Tang (171 papers)
- Weinan Zhang (322 papers)
- Yong Yu (219 papers)