Blockwise Feature Interaction in Recommendation Systems (2306.15881v1)
Abstract: Feature interactions can play a crucial role in recommendation systems as they capture complex relationships between user preferences and item characteristics. Existing methods such as Deep & Cross Network (DCNv2) may suffer from high computational requirements due to their cross-layer operations. In this paper, we propose a novel approach called blockwise feature interaction (BFI) to help alleviate this issue. By partitioning the feature interaction process into smaller blocks, we can significantly reduce both the memory footprint and the computational burden. Four variants (denoted by P, Q, T, S, respectively) of BFI have been developed and empirically compared. Our experimental results demonstrate that the proposed algorithms achieves close accuracy compared to the standard DCNv2, while greatly reducing the computational overhead and the number of parameters. This paper contributes to the development of efficient recommendation systems by providing a practical solution for improving feature interaction efficiency.
- Jason Brownlee. A gentle introduction to the rectified linear unit (relu). Machine learning mastery, 6, 2019.
- Pre-training tasks for embedding-based large-scale retrieval. In Proceedings of the 8th International Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia, 2020.
- Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys), pages 191–198, Boston, MA, 2016.
- MOBIUS: towards the next generation of query-ad matching in baidu’s sponsored search. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), pages 2509–2517, Anchorage, AK, 2019.
- GemNN: Gating-enhanced multi-task neural networks with feature interaction learning for CTR prediction. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 2166–2171, Virtual Event, Canada, 2021.
- Declutr: Deep contrastive learning for unsupervised textual representations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP, pages 879–895, Virtual Event, 2021.
- DeepFM: A factorization-machine based neural network for CTR prediction. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI), pages 1725–1731, Melbourne, Australia, 2017.
- Deep fragment embeddings for bidirectional image sentence mapping. In Advances in Neural Information Processing Systems (NIPS), pages 1889–1897, Montreal, Canada, 2014.
- General multi-label image classification with transformers. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 16478–16488, virtual, 2021.
- Video recommendation with multi-gate mixture of experts soft actor critic. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval (SIGIR), pages 1553–1556, Virtual Event, China, 2020.
- xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), pages 1754–1763, London, UK, 2018.
- Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005, 2022.
- Automatic differentiation in pytorch. 2017.
- Steffen Rendle. Factorization machines. In Proceedings of the 2010 IEEE International Conference on Data Mining (ICDM), pages 995–1000, Sydney, Australia, 2010.
- Knowledge-aware recommendations based on neuro-symbolic graph embeddings and first-order logical rules. In Proceedings of the Sixteenth ACM Conference on Recommender Systems (RecSys), pages 616–621, Seattle, WA, 2022.
- DCN V2: improved deep & cross network and practical lessons for web-scale learning to rank systems. In The Web Conference 2021 (WWW), pages 1785–1797, Virtual Event / Ljubljana, Slovenia, 2021.
- Hierarchical bilinear pooling for fine-grained visual recognition. In Proceedings of the 15th European Conference on Computer Vision (ECCV), Part XVI, pages 595–610, Munich, Germany, 2018.
- EGM: enhanced graph-based model for large-scale video advertisement search. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 4443–4451, Washington, DC, 2022.
- AIBox: CTR prediction model training on a single node. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM), pages 319–328, Beijing, China, 2019.