Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Graph-Based Model-Agnostic Data Subsampling for Recommendation Systems (2305.16391v2)

Published 25 May 2023 in cs.IR, cs.LG, and cs.SI

Abstract: Data subsampling is widely used to speed up the training of large-scale recommendation systems. Most subsampling methods are model-based and often require a pre-trained pilot model to measure data importance via e.g. sample hardness. However, when the pilot model is misspecified, model-based subsampling methods deteriorate. Since model misspecification is persistent in real recommendation systems, we instead propose model-agnostic data subsampling methods by only exploring input data structure represented by graphs. Specifically, we study the topology of the user-item graph to estimate the importance of each user-item interaction (an edge in the user-item graph) via graph conductance, followed by a propagation step on the network to smooth out the estimated importance value. Since our proposed method is model-agnostic, we can marry the merits of both model-agnostic and model-based subsampling methods. Empirically, we show that combing the two consistently improves over any single method on the used datasets. Experimental results on KuaiRec and MIND datasets demonstrate that our proposed methods achieve superior results compared to baseline approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Fundamentals of electric circuits. McGraw-Hill Higher Education Boston.
  2. Graph coarsening with neural networks. arXiv preprint arXiv:2102.01350 (2021).
  3. The electrical resistance of a graph captures its commute and cover times. computational complexity 6, 4 (1996), 312–340.
  4. Nitesh V Chawla. 2009. Data mining for imbalanced datasets: An overview. Data mining and knowledge discovery handbook (2009), 875–886.
  5. On sampling strategies for neural network-based collaborative filtering. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 767–776.
  6. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7–10.
  7. Fan RK Chung. 1997. Spectral graph theory. Vol. 92. American Mathematical Soc.
  8. Simplify and robustify negative sampling for implicit collaborative filtering. Advances in Neural Information Processing Systems 33 (2020), 1094–1105.
  9. William Fithian and Trevor Hastie. 2014. Local case-control sampling: Efficient subsampling in imbalanced data sets. Annals of statistics 42, 5 (2014), 1693.
  10. KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems. arXiv preprint arXiv:2202.10842 (2022).
  11. Minimizing effective resistance of a graph. SIAM review 50, 1 (2008), 37–66.
  12. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247 (2017).
  13. Structure-preserving sparsification methods for social networks. Social Network Analysis and Mining 6, 1 (2016), 1–22.
  14. Local uncertainty sampling for large-scale multiclass logistic regression. The Annals of Statistics 48, 3 (2020), 1770–1788.
  15. Frank Harary and Robert Z Norman. 1960. Some properties of line digraphs. Rendiconti del circolo matematico di palermo 9, 2 (1960), 161–168.
  16. Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. 355–364.
  17. Daniel G Horvitz and Donovan J Thompson. 1952. A generalization of sampling without replacement from a finite universe. Journal of the American statistical Association 47, 260 (1952), 663–685.
  18. Embedding-based retrieval in facebook search. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2553–2561.
  19. Combining label propagation and simple models out-performs graph neural networks. arXiv preprint arXiv:2010.13993 (2020).
  20. Junteng Jia and Austion R Benson. 2020. Residual correlation in graph neural network regression. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 588–598.
  21. Can M Le. 2021. Edge Sampling Using Local Network Information. J. Mach. Learn. Res. 22 (2021), 88–1.
  22. Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2636–2645.
  23. Measuring Calibration in Deep Learning.. In CVPR workshops, Vol. 2.
  24. Untangling hairballs. In International Symposium on Graph Drawing. Springer, 101–112.
  25. Contrastive learning with hard negative samples. arXiv preprint arXiv:2010.04592 (2020).
  26. Local graph sparsification for scalable clustering. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of data. 721–732.
  27. Hidden technical debt in machine learning systems. Advances in neural information processing systems 28 (2015).
  28. Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE Access 5 (2017), 3909–3943.
  29. Surprise sampling: Improving and extending the local case-control sampling. (2021).
  30. Daniel A Spielman and Nikhil Srivastava. 2008. Graph sparsification by effective resistances. In Proceedings of the fortieth annual ACM symposium on Theory of computing. 563–568.
  31. NetworKit: A tool suite for large-scale complex network analysis. Network Science 4, 4 (2016), 508–530.
  32. Daniel Ting and Eric Brochu. 2018. Optimal subsampling with influence functions. Advances in neural information processing systems 31 (2018).
  33. Guihong Wan and Harsha Kokel. 2021. Graph sparsification via meta-learning. DLG@ AAAI (2021).
  34. HaiYing Wang. 2020. Logistic regression for massive data with rare events. In International Conference on Machine Learning. PMLR, 9829–9836.
  35. Nonuniform Negative Sampling and Log Odds Correction with Rare Events Data. Advances in Neural Information Processing Systems 34 (2021), 19847–19859.
  36. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. 515–524.
  37. Mind: A large-scale dataset for news recommendation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 3597–3606.
  38. Noise contrastive estimation for one-class collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 135–144.
  39. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 974–983.
  40. Optimizing top-n collaborative filtering via dynamic negative item sampling. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. 785–788.
  41. Robust graph representation learning via neural sparsification. In International Conference on Machine Learning. PMLR, 11458–11468.
  42. Learning with local and global consistency. Advances in neural information processing systems 16 (2003).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Xiaohui Chen (73 papers)
  2. Jiankai Sun (53 papers)
  3. Taiqing Wang (5 papers)
  4. Ruocheng Guo (62 papers)
  5. Li-Ping Liu (27 papers)
  6. Aonan Zhang (32 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now