Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

BiVRec: Bidirectional View-based Multimodal Sequential Recommendation (2402.17334v2)

Published 27 Feb 2024 in cs.IR and cs.AI

Abstract: The integration of multimodal information into sequential recommender systems has attracted significant attention in recent research. In the initial stages of multimodal sequential recommendation models, the mainstream paradigm was ID-dominant recommendations, wherein multimodal information was fused as side information. However, due to their limitations in terms of transferability and information intrusion, another paradigm emerged, wherein multimodal features were employed directly for recommendation, enabling recommendation across datasets. Nonetheless, it overlooked user ID information, resulting in low information utilization and high training costs. To this end, we propose an innovative framework, BivRec, that jointly trains the recommendation tasks in both ID and multimodal views, leveraging their synergistic relationship to enhance recommendation performance bidirectionally. To tackle the information heterogeneity issue, we first construct structured user interest representations and then learn the synergistic relationship between them. Specifically, BivRec comprises three modules: Multi-scale Interest Embedding, comprehensively modeling user interests by expanding user interaction sequences with multi-scale patching; Intra-View Interest Decomposition, constructing highly structured interest representations using carefully designed Gaussian attention and Cluster attention; and Cross-View Interest Learning, learning the synergistic relationship between the two recommendation views through coarse-grained overall semantic similarity and fine-grained interest allocation similarity BiVRec achieves state-of-the-art performance on five datasets and showcases various practical advantages.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Controllable multi-interest framework for recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2942–2951.
  2. Intent contrastive learning for sequential recommendation. In Proceedings of the ACM Web Conference 2022. 2172–2182.
  3. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191–198.
  4. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
  5. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  6. Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In 2016 IEEE 16th international conference on data mining (ICDM). IEEE, 191–200.
  7. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).
  8. Learning vector-quantized item representation for transferable sequential recommenders. In Proceedings of the ACM Web Conference 2023. 1162–1171.
  9. Towards Universal Sequence Representation Learning for Recommender Systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 585–593.
  10. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016).
  11. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: Long papers). 687–696.
  12. Fism: factored item similarity models for top-n recommender systems. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. 659–667.
  13. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM). IEEE, 197–206.
  14. Intention-aware sequential recommendation with structured intent transition. IEEE Transactions on Knowledge and Data Engineering 34, 11 (2021), 5403–5414.
  15. MMMLP: Multi-modal Multilayer Perceptron for Sequential Recommendations. In Proceedings of the ACM Web Conference 2023. 1109–1117.
  16. Explainable outfit recommendation with joint outfit matching and comment generation. IEEE Transactions on Knowledge and Data Engineering 32, 8 (2019), 1502–1516.
  17. Noninvasive self-attention for side information fusion in sequential recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4249–4256.
  18. NRPA: neural recommendation with personalized attention. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1233–1236.
  19. MEGCF: Multimodal entity graph collaborative filtering for personalized recommendation. ACM Transactions on Information Systems 41, 2 (2023), 1–27.
  20. Multimodal Recommender Systems: A Survey. arXiv preprint arXiv:2302.03883 (2023).
  21. Multi-Modal Contrastive Pre-training for Recommendation. In Proceedings of the 2022 International Conference on Multimedia Retrieval. 99–108.
  22. CrossCBR: Cross-view Contrastive Learning for Bundle Recommendation. arXiv preprint arXiv:2206.00242 (2022).
  23. Learning Hybrid Behavior Patterns for Multimedia Recommendation. In Proceedings of the 30th ACM International Conference on Multimedia. 376–384.
  24. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).
  25. Multimodal Meta-Learning for Cold-Start Sequential Recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 3421–3430.
  26. Contrastive learning for representation degeneration problem in sequential recommendation. In Proceedings of the fifteenth ACM international conference on web search and data mining. 813–823.
  27. CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention. arXiv preprint arXiv:2204.06519 (2022).
  28. Steffen Rendle. 2010. Factorization machines. In 2010 IEEE International conference on data mining. IEEE, 995–1000.
  29. Dynamic routing between capsules. Advances in neural information processing systems 30 (2017).
  30. Temporal aware multi-interest graph neural network for session-based recommendation. arXiv preprint arXiv:2112.15328 (2021).
  31. Temporal aware multi-interest graph neural network for session-based recommendation. In Asian Conference on Machine Learning. PMLR.
  32. Self-Supervised Multi-Modal Sequential Recommendation. arXiv preprint arXiv:2304.13277 (2023).
  33. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441–1450.
  34. Sparse-interest network for sequential recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 598–606.
  35. Neural discrete representation learning. Advances in neural information processing systems 30 (2017).
  36. Attention is all you need. Advances in neural information processing systems 30 (2017).
  37. TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedback. arXiv preprint arXiv:2206.06190 (2022).
  38. MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation. arXiv preprint arXiv:2308.11175 (2023).
  39. Selective fairness in recommendation via prompts. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2657–2662.
  40. Personalized Prompts for Sequential Recommendation. arXiv preprint arXiv:2205.09666 (2022).
  41. Contrastive learning for sequential recommendation. In 2022 IEEE 38th international conference on data engineering (ICDE). IEEE, 1259–1273.
  42. Decoupled side information fusion for sequential recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1611–1621.
  43. Groupvit: Semantic segmentation emerges from text supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18134–18144.
  44. Where to go next for recommender systems? id-vs. modality-based recommender models revisited. arXiv preprint arXiv:2303.13835 (2023).
  45. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329 (2014).
  46. Multimodal Pre-training Framework for Sequential Recommendation via Contrastive Learning. arXiv preprint arXiv:2303.11879 (2023).
  47. Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendation. In Proceedings of the ACM Web Conference 2022. 2216–2226.
  48. Feature-level Deeper Self-Attention Network for Sequential Recommendation.. In IJCAI. 4320–4326.
  49. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM international conference on information & knowledge management. 1893–1902.
  50. Xin Zhou. 2022. A Tale of Two Graphs: Freezing and Denoising Graph Structures for Multimodal Recommendation. arXiv preprint arXiv:2211.06924 (2022).
  51. Multi-representation adaptation network for cross-domain image classification. Neural Networks 119 (2019), 214–221.
  52. Using temporal data for making recommendations. arXiv preprint arXiv:1301.2320 (2013).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (9)
  1. Jiaxi Hu (12 papers)
  2. Jingtong Gao (14 papers)
  3. Xiangyu Zhao (193 papers)
  4. Yuehong Hu (4 papers)
  5. Yuxuan Liang (126 papers)
  6. Yiqi Wang (39 papers)
  7. Ming He (27 papers)
  8. Zitao Liu (76 papers)
  9. Hongzhi Yin (211 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now