Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SIGformer: Sign-aware Graph Transformer for Recommendation (2404.11982v3)

Published 18 Apr 2024 in cs.IR

Abstract: In recommender systems, most graph-based methods focus on positive user feedback, while overlooking the valuable negative feedback. Integrating both positive and negative feedback to form a signed graph can lead to a more comprehensive understanding of user preferences. However, the existing efforts to incorporate both types of feedback are sparse and face two main limitations: 1) They process positive and negative feedback separately, which fails to holistically leverage the collaborative information within the signed graph; 2) They rely on MLPs or GNNs for information extraction from negative feedback, which may not be effective. To overcome these limitations, we introduce SIGformer, a new method that employs the transformer architecture to sign-aware graph-based recommendation. SIGformer incorporates two innovative positional encodings that capture the spectral properties and path patterns of the signed graph, enabling the full exploitation of the entire graph. Our extensive experiments across five real-world datasets demonstrate the superiority of SIGformer over state-of-the-art methods. The code is available at https://github.com/StupidThree/SIGformer.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. Uri Alon and Eran Yahav. 2021. On the Bottleneck of Graph Neural Networks and its Practical Implications. In International Conference on Learning Representations.
  2. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  3. LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. In The Eleventh International Conference on Learning Representations.
  4. End-to-end object detection with transformers. In European conference on computer vision. Springer, 213–229.
  5. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 3438–3445.
  6. Structure-aware transformer for graph representation learning. In International Conference on Machine Learning. PMLR, 3469–3489.
  7. Pre-trained image processing transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12299–12310.
  8. AutoDebias: Learning to debias for recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 21–30.
  9. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems 41, 3 (2023), 1–39.
  10. NAGphormer: A tokenized graph transformer for node classification in large graphs. In The Eleventh International Conference on Learning Representations.
  11. Signgt: Signed attention-based graph transformer for graph representation learning. arXiv preprint arXiv:2310.11025 (2023).
  12. Fan RK Chung. 1997. Spectral graph theory. Vol. 92. American Mathematical Soc.
  13. Graph convolution network based recommender systems: Learning guarantee and item mixture powered strategy. Advances in Neural Information Processing Systems 35 (2022), 3900–3912.
  14. Signed graph convolutional networks. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 929–934.
  15. An improved sampler for bayesian personalized ranking by leveraging view data. In Companion Proceedings of the The Web Conference 2018. 13–14.
  16. Reinforced Negative Sampling for Recommendation with Exposure Data.. In IJCAI. Macao, 2230–2236.
  17. On the equivalence of decoupled graph convolution network and label propagation. In Proceedings of the Web Conference 2021. 3651–3662.
  18. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations.
  19. Vijay Prakash Dwivedi and Xavier Bresson. 2021. A Generalization of Transformer Networks to Graphs. AAAI Workshop on Deep Learning on Graphs: Methods and Applications (2021).
  20. Graph neural networks for social recommendation. In The world wide web conference. 417–426.
  21. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM international conference on information & knowledge management. 433–442.
  22. KuaiRec: A fully-observed dataset and insights for evaluating recommender systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 540–550.
  23. KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 3953–3957.
  24. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 639–648.
  25. Fritz Heider. 1946. Attitudes and cognitive organization. The Journal of psychology 21, 1 (1946), 107–112.
  26. On the Laplacian eigenvalues of signed graphs. Linear and Multilinear Algebra 51, 1 (2003), 21–30.
  27. Low rank modeling of signed networks. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 507–515.
  28. Signed bipartite graph neural networks. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 740–749.
  29. Signed graph attention networks. In Artificial Neural Networks and Machine Learning–ICANN 2019: Workshop and Special Sessions: 28th International Conference on Artificial Neural Networks, Munich, Germany, September 17–19, 2019, Proceedings 28. Springer, 566–577.
  30. Negative can be positive: Signed graph neural networks for recommendation. Information Processing & Management 60, 4 (2023), 103403.
  31. Signet: Scalable embeddings for signed networks. In Advances in Knowledge Discovery and Data Mining: 22nd Pacific-Asia Conference, PAKDD 2018, Melbourne, VIC, Australia, June 3-6, 2018, Proceedings, Part II 22. Springer, 157–169.
  32. Olivier Jeunen. 2019. Revisiting offline evaluation for implicit-feedback recommender systems. In Proceedings of the 13th ACM Conference on Recommender Systems. 596–600.
  33. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL-HLT. 4171–4186.
  34. Side: representation learning in signed directed networks. In Proceedings of the 2018 world wide web conference. 509–518.
  35. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.
  36. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems 34 (2021), 21618–21629.
  37. Graph Transformer for Recommendation. arXiv preprint arXiv:2306.02330 (2023).
  38. Distance encoding: Design provably more powerful neural networks for graph representation learning. Advances in Neural Information Processing Systems 33 (2020), 4465–4478.
  39. Signed laplacian graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 4444–4452.
  40. Learning signed network embedding via graph attention. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 4772–4779.
  41. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 world wide web conference. 689–698.
  42. PANE-GNN: Unifying Positive and Negative Edges in Graph Neural Networks for Recommendation. arXiv preprint arXiv:2306.04095 (2023).
  43. Julian John McAuley and Jure Leskovec. 2013. From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. In Proceedings of the 22nd international conference on World Wide Web. 897–908.
  44. Andriy Mnih and Russ R Salakhutdinov. 2007. Probabilistic matrix factorization. Advances in neural information processing systems 20 (2007).
  45. Understanding and Modeling Passive-Negative Feedback for Short-video Sequential Recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems. 540–550.
  46. Minju Park and Kyogu Lee. 2022. Exploiting Negative Preference in Content-based Music Recommendation with Contrastive Learning. In Proceedings of the 16th ACM Conference on Recommender Systems. 229–236.
  47. GRPE: Relative Positional Encoding for Graph Transformer. In ICLR2022 Machine Learning for Drug Discovery.
  48. SiReN: Sign-aware recommendation using graph neural networks. IEEE Transactions on Neural Networks and Learning Systems (2022).
  49. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE signal processing magazine 30, 3 (2013), 83–98.
  50. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441–1450.
  51. eTrust: Understanding trust evolution in an online world. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 253–261.
  52. Attention is all you need. Advances in neural information processing systems 30 (2017).
  53. Distributionally Robust Graph-based Recommendation System. arXiv preprint arXiv:2402.12994 (2024).
  54. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval. 165–174.
  55. Xiyuan Wang and Muhan Zhang. 2022. How powerful are spectral graph neural networks. In International Conference on Machine Learning. PMLR, 23341–23362.
  56. Understanding contrastive learning via distributionally robust optimization. Advances in Neural Information Processing Systems 36 (2024).
  57. BSL: Understanding and Improving Softmax Loss for Recommendation. arXiv preprint arXiv:2312.12882 (2023).
  58. Self-supervised graph learning for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 726–735.
  59. SSE-PT: Sequential recommendation via personalized transformer. In Proceedings of the 14th ACM Conference on Recommender Systems. 328–337.
  60. Simplifying and Empowering Transformers for Large-Graph Representations. arXiv preprint arXiv:2306.10759 (2023).
  61. Graph neural networks in recommender systems: a survey. Comput. Surveys 55, 5 (2022), 1–37.
  62. Self-supervised hypergraph transformer for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2100–2109.
  63. Deep feedback network for recommendation. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence. 2519–2525.
  64. How Powerful are Graph Neural Networks?. In International Conference on Learning Representations.
  65. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems 34 (2021), 28877–28888.
  66. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 974–983.
  67. XSimGCL: Towards extremely simple graph contrastive learning for recommendation. IEEE Transactions on Knowledge and Data Engineering (2023).
  68. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval. 1294–1303.
  69. Contrastive learning for signed bipartite graphs. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1629–1638.
  70. Recommendations with negative feedback via pairwise deep reinforcement learning. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1040–1048.
  71. Zhi-Dan Zhao and Ming-Sheng Shang. 2010. User-based collaborative-filtering recommendation algorithms on hadoop. In 2010 third international conference on knowledge discovery and data mining. IEEE, 478–481.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Sirui Chen (34 papers)
  2. Jiawei Chen (161 papers)
  3. Sheng Zhou (186 papers)
  4. Bohao Wang (11 papers)
  5. Shen Han (4 papers)
  6. Chanfei Su (2 papers)
  7. Yuqing Yuan (2 papers)
  8. Can Wang (156 papers)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now