Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Securing Visually-Aware Recommender Systems: An Adversarial Image Reconstruction and Detection Framework (2306.07992v1)

Published 11 Jun 2023 in cs.CV, cs.AI, cs.CR, and cs.LG

Abstract: With rich visual data, such as images, becoming readily associated with items, visually-aware recommendation systems (VARS) have been widely used in different applications. Recent studies have shown that VARS are vulnerable to item-image adversarial attacks, which add human-imperceptible perturbations to the clean images associated with those items. Attacks on VARS pose new security challenges to a wide range of applications such as e-Commerce and social networks where VARS are widely used. How to secure VARS from such adversarial attacks becomes a critical problem. Currently, there is still a lack of systematic study on how to design secure defense strategies against visual attacks on VARS. In this paper, we attempt to fill this gap by proposing an adversarial image reconstruction and detection framework to secure VARS. Our proposed method can simultaneously (1) secure VARS from adversarial attacks characterized by local perturbations by image reconstruction based on global vision transformers; and (2) accurately detect adversarial examples using a novel contrastive learning approach. Meanwhile, our framework is designed to be used as both a filter and a detector so that they can be jointly trained to improve the flexibility of our defense strategy to a variety of attacks and VARS models. We have conducted extensive experimental studies with two popular attack methods (FGSM and PGD). Our experimental results on two real-world datasets show that our defense strategy against visual attacks is effective and outperforms existing methods on different attacks. Moreover, our method can detect adversarial examples with high accuracy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE transactions on knowledge and data engineering 17, 6 (2005), 734–749.
  2. V-Elliot: Design, Evaluate and Tune Visual Recommender Systems. In Fifteenth ACM Conference on Recommender Systems. 768–771.
  3. A Study of Defensive Methods to Protect Visual Recommendation Against Adversarial Manipulation of Images. In The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21). 10.
  4. Classification features for attack detection in collaborative recommender systems. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. 542–547.
  5. Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). IEEE, 39–57.
  6. Data poisoning attacks on neighborhood‐based recommender systems. Transactions on Emerging Telecommunications Technologies (2020).
  7. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597–1607.
  8. A Black-Box Attack Model for Visually-Aware Recommender Systems. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining (WSDM ’21). 94–102.
  9. A Study on the Relative Importance of Convolutional Neural Networks in Visually-Aware Recommender Systems. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 3961–3967.
  10. A survey on adversarial recommender systems: from attack/defense strategies to generative adversarial networks. ACM Computing Surveys (CSUR) 54, 2 (2021), 1–38.
  11. Taamr: Targeted adversarial attack against multimedia recommender systems. In 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W). IEEE, 1–8.
  12. Exploiting food choice biases for healthier recipe recommendation. In Proceedings of the 40th international acm sigir conference on research and development in information retrieval. 575–584.
  13. Influence function based data poisoning attacks to top-n recommender systems. In Proceedings of The Web Conference 2020. 3019–3025.
  14. Poisoning attacks to graph-based recommender systems. In Proceedings of the 34th Annual Computer Security Applications Conference. 381–392.
  15. Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Transactions on knowledge and data engineering 19, 3 (2007), 355–369.
  16. Dataset security for machine learning: Data poisoning, backdoor attacks, and defenses. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 2 (2022), 1563–1580.
  17. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).
  18. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  19. Ruining He and Julian McAuley. 2016a. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web. 507–517.
  20. Ruining He and Julian McAuley. 2016b. VBPR: visual bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.
  21. Trirank: Review-aware explainable recommendation by modeling aspects. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 1661–1670.
  22. Adversarial personalized ranking for recommendation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 355–364.
  23. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173–182.
  24. Collaborative fashion recommendation: A functional tensor factorization approach. In Proceedings of the 23rd ACM international conference on Multimedia. 129–138.
  25. Data Poisoning Attacks to Deep Learning Based Recommender Systems. In Network and Distributed System Security Symposium (NDSS) 2021.
  26. Robust Pre-Training by Adversarial Contrastive Learning.. In NeurIPS.
  27. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision. Springer, 694–711.
  28. Visually-aware fashion recommendation and design with generative image models. In 2017 IEEE International Conference on Data Mining (ICDM). IEEE, 207–216.
  29. Large scale metric learning from equivalence constraints. In 2012 IEEE conference on computer vision and pattern recognition. IEEE, 2288–2295.
  30. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.
  31. Adversarial examples in the physical world. In Artificial intelligence safety and security. Chapman and Hall/CRC, 99–112.
  32. Shyong K Lam and John Riedl. 2004. Shilling recommender systems for fun and profit. In Proceedings of the 13th international conference on World Wide Web. 393–402.
  33. Jong-Seok Lee and Dan Zhu. 2012. Shilling attack detection—a new approach for a trustworthy recommender system. INFORMS Journal on Computing 24, 1 (2012), 117–131.
  34. Data poisoning attacks on factorization-based collaborative filtering. Advances in neural information processing systems 29 (2016), 1885–1893.
  35. Certifiable robustness to discrete adversarial perturbations for factorization machines. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 419–428.
  36. Zhuoran Liu and Martha Larson. 2021. Adversarial Item Promotion: Vulnerabilities at the Core of Top-N Recommenders that Use Images to Address Cold Start. In Proceedings of The Web Conference 2021 (WWW ’21).
  37. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. 43–52.
  38. Robust collaborative filtering. In Proceedings of the 2007 ACM conference on Recommender systems. 49–56.
  39. Bhaskar Mehta and Wolfgang Nejdl. 2009. Unsupervised strategies for shilling detection and robust collaborative filtering. User Modeling and User-Adapted Interaction 19, 1 (2009), 65–97.
  40. Andriy Mnih and Russ R Salakhutdinov. 2007. Probabilistic matrix factorization. Advances in neural information processing systems 20 (2007), 1257–1264.
  41. Toward trustworthy recommender systems: An analysis of attack models and algorithm robustness. ACM Transactions on Internet Technology (TOIT) 7, 4 (2007), 23–es.
  42. Poisoning GNN-based recommender systems with generative surrogate-based attacks. ACM Transactions on Information Systems 41, 3 (2023), 1–24.
  43. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).
  44. Pinnersage: Multi-modal user embedding framework for recommendations at pinterest. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2311–2320.
  45. Sayak Paul and Pin-Yu Chen. 2021. Vision transformers are robust learners. arXiv preprint arXiv:2105.07581 (2021).
  46. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (Montreal, Quebec, Canada) (UAI ’09). AUAI Press, Arlington, Virginia, USA, 452–461.
  47. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web. 285–295.
  48. PicTouRe-A Picture-Based Tourism Recommender. In Fourteenth ACM Conference on Recommender Systems. 597–599.
  49. Poisonrec: an adaptive data poisoning framework for attacking black-box recommender systems. In 2020 IEEE 36th International Conference on Data Engineering (ICDE). IEEE, 157–168.
  50. Bottleneck transformers for visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 16519–16529.
  51. Adversarial Training Towards Robust Multimedia Recommender System. IEEE Transactions on Knowledge and Data Engineering 32, 5 (2020), 855–867.
  52. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).
  53. Triple Adversarial Learning for Influence based Poisoning Attack in Recommender Systems. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1830–1840.
  54. Fight Fire with Fire: Towards Robust Recommender Systems via Adversarial Poisoning Training. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1074–1083.
  55. Graph neural networks in recommender systems: a survey. Comput. Surveys 55, 5 (2022), 1–37.
  56. HySAD: A semi-supervised hybrid shilling attack detector for trustworthy product recommendation. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 985–993.
  57. Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3733–3742.
  58. Mitigating Adversarial Effects Through Randomization. In International Conference on Learning Representations.
  59. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 501–509.
  60. Take this personally: Pollution attacks on personalized services. In 22nd USENIX Security Symposium (USENIX Security 13). 671–686.
  61. On the Vulnerability of Graph Learning-based Collaborative Filtering. ACM Transactions on Information Systems 41, 4 (2023), 1–28.
  62. Estimating user behavior toward detecting anomalous ratings in rating systems. Knowledge-Based Systems 111 (2016), 144–158.
  63. Adversarial collaborative neural network for robust recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1065–1068.
  64. Adversarial Examples: Attacks and Defenses for Deep Learning. IEEE Transactions on Neural Networks and Learning Systems 30, 9 (2019), 2805–2824.
  65. Visual discovery at pinterest. In Proceedings of the 26th International Conference on World Wide Web Companion. 515–524.
  66. Defense against adversarial attacks by reconstructing images. IEEE Transactions on Image Processing 30 (2021), 6117–6129.
  67. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR) 52, 1 (2019), 5.
  68. Gcn-based user representation learning for unifying robust recommendation and fraudster detection. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval. 689–698.
  69. Reverse Attack: Black-box Attacks on Collaborative Recommendation. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 51–68.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Minglei Yin (5 papers)
  2. Bin Liu (441 papers)
  3. Neil Zhenqiang Gong (117 papers)
  4. Xin Li (980 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now