Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

NeuroHash: A Hyperdimensional Neuro-Symbolic Framework for Spatially-Aware Image Hashing and Retrieval (2404.11025v3)

Published 17 Apr 2024 in cs.CV

Abstract: Customizable image retrieval from large datasets remains a critical challenge, particularly when preserving spatial relationships within images. Traditional hashing methods, primarily based on deep learning, often fail to capture spatial information adequately and lack transparency. In this paper, we introduce NeuroHash, a novel neuro-symbolic framework leveraging Hyperdimensional Computing (HDC) to enable highly customizable, spatially-aware image retrieval. NeuroHash combines pre-trained deep neural network models with HDC-based symbolic models, allowing for flexible manipulation of hash values to support conditional image retrieval. Our method includes a self-supervised context-aware HDC encoder and novel loss terms for optimizing lower-dimensional bipolar hashing using multilinear hyperplanes. We evaluate NeuroHash on two benchmark datasets, demonstrating superior performance compared to state-of-the-art hashing methods, as measured by mAP@5K scores and our newly introduced metric, mAP@5Kr, which assesses spatial alignment. The results highlight NeuroHash's ability to achieve competitive performance while offering significant advantages in flexibility and customization, paving the way for more advanced and versatile image retrieval systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. H. Cao, L. Huang, J. Nie, and Z. Wei, “Unsupervised deep hashing with fine-grained similarity-preserving contrastive learning for image retrieval,” IEEE Transactions on Circuits and Systems for Video Technology, 2023.
  2. Z. Cao, M. Long, J. Wang, and P. S. Yu, “Hashnet: Deep learning to hash by continuation,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 5608–5617.
  3. C.-F. R. Chen, Q. Fan, and R. Panda, “Crossvit: Cross-attention multi-scale vision transformer for image classification,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 357–366.
  4. Y. Chen, Z. Lai, Y. Ding, K. Lin, and W. K. Wong, “Deep supervised hashing with anchor graph,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9796–9804.
  5. B. Dai, R. Guo, S. Kumar, N. He, and L. Song, “Stochastic generative hashing,” in International Conference on Machine Learning.   PMLR, 2017, pp. 913–922.
  6. S. Duan, Y. Liu, S. Ren, and X. Xu, “Lehdc: Learning-based hyperdimensional computing classifier,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, 2022, pp. 1111–1116.
  7. A. Dutta, S. Gupta, B. Khaleghi, R. Chandrasekaran, W. Xu, and T. Rosing, “Hdnn-pim: Efficient in memory design of hyperdimensional computing with feature extraction,” in Proceedings of the Great Lakes Symposium on VLSI 2022, 2022, pp. 281–286.
  8. A. Gionis, P. Indyk, R. Motwani et al., “Similarity search in high dimensions via hashing,” in Vldb, vol. 99, no. 6, 1999, pp. 518–529.
  9. Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin, “Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 12, pp. 2916–2929, 2012.
  10. M. Hersche, G. Karunaratne, G. Cherubini, L. Benini, A. Sebastian, and A. Rahimi, “Constrained few-shot class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9057–9067.
  11. Z. Huang and S. Liu, “Perceptual hashing with visual content understanding for reduced-reference screen content image quality assessment,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 7, pp. 2808–2823, 2020.
  12. M. Imani, D. Kong, A. Rahimi, and T. Rosing, “Voicehd: Hyperdimensional computing for efficient speech recognition,” in 2017 IEEE international conference on rebooting computing (ICRC).   IEEE, 2017, pp. 1–8.
  13. P. Kanerva, “Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional vectors,” Cognitive Computation, 2009.
  14. J. Kang, M. Zhou, A. Bhansali, W. Xu, A. Thomas, and T. Rosing, “Relhd: A graph-based learning on fefet with hyperdimensional computing,” in 2022 IEEE 40th International Conference on Computer Design (ICCD).   IEEE, 2022, pp. 553–560.
  15. A. Krizhevsky, G. Hinton et al., “Learning multiple layers of features from tiny images,” 2009.
  16. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems, vol. 25, 2012.
  17. X. Li, J. Yu, Y. Wang, J.-Y. Chen, P.-X. Chang, and Z. Li, “Dahp: Deep attention-guided hashing with pairwise labels,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 3, pp. 933–946, 2021.
  18. K. Lin, J. Lu, C.-S. Chen, J. Zhou, and M.-T. Sun, “Unsupervised deep learning of compact binary descriptors,” IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 6, pp. 1501–1514, 2018.
  19. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13.   Springer, 2014, pp. 740–755.
  20. S. Liu and Z. Huang, “Efficient image hashing with geometric invariant vector distance for copy detection,” ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), vol. 15, no. 4, pp. 1–22, 2019.
  21. W. Liu, C. Mu, S. Kumar, and S.-F. Chang, “Discrete graph hashing,” Advances in neural information processing systems, vol. 27, 2014.
  22. W. Liu, J. Wang, S. Kumar, and S.-F. Chang, “Hashing with graphs,” 2011.
  23. X. Liu, X. Fan, C. Deng, Z. Li, H. Su, and D. Tao, “Multilinear hyperplane hashing,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 5119–5127.
  24. Z. Ma, W. Ju, X. Luo, C. Chen, X.-S. Hua, and G. Lu, “Improved deep unsupervised hashing via prototypical learning,” in Proceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 659–667.
  25. I. Nunes, M. Heddes, T. Givargis, A. Nicolau, and A. Veidenbaum, “Graphhd: Efficient graph classification using hyperdimensional computing,” in 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE).   IEEE, 2022, pp. 1485–1490.
  26. M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, R. Howes, P.-Y. Huang, H. Xu, V. Sharma, S.-W. Li, W. Galuba, M. Rabbat, M. Assran, N. Ballas, G. Synnaeve, I. Misra, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski, “Dinov2: Learning robust visual features without supervision,” 2023.
  27. Z. Qiu, Q. Su, Z. Ou, J. Yu, and C. Chen, “Unsupervised hashing with contrastive information bottleneck,” arXiv preprint arXiv:2105.06138, 2021.
  28. F. Shen, C. Shen, W. Liu, and H. Tao Shen, “Supervised discrete hashing,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 37–45.
  29. F. Shen, Y. Xu, L. Liu, Y. Yang, Z. Huang, and H. T. Shen, “Unsupervised deep hashing with similarity-adaptive and discrete optimization,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 12, pp. 3034–3044, 2018.
  30. Y. Shen, L. Liu, and L. Shao, “Unsupervised binary representation learning with deep variational networks,” International Journal of Computer Vision, vol. 127, no. 11-12, pp. 1614–1628, 2019.
  31. Y. Shen, J. Qin, J. Chen, M. Yu, L. Liu, F. Zhu, F. Shen, and L. Shao, “Auto-encoding twin-bottleneck hashing,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2818–2827.
  32. J. Song, T. He, L. Gao, X. Xu, A. Hanjalic, and H. T. Shen, “Binary generative adversarial networks for image retrieval,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018.
  33. S. Su, C. Zhang, K. Han, and Y. Tian, “Greedy hash: Towards fast optimization for accurate hash coding in cnn,” Advances in neural information processing systems, vol. 31, 2018.
  34. J. Wang, Z. Zeng, B. Chen, T. Dai, and S.-T. Xia, “Contrastive quantization with code memory for unsupervised image retrieval,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, 2022, pp. 2468–2476.
  35. Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick, “Detectron2,” https://github.com/facebookresearch/detectron2, 2019.
  36. R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, “Supervised hashing for image retrieval via image representation learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 28, no. 1, 2014.
  37. E. Yang, C. Deng, T. Liu, W. Liu, and D. Tao, “Semantic structure-based unsupervised deep hashing,” in Proceedings of the 27th international joint conference on artificial intelligence, 2018, pp. 1064–1070.
  38. E. Yang, T. Liu, C. Deng, W. Liu, and D. Tao, “Distillhash: Unsupervised deep hashing by distilling data pairs,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 2946–2955.
  39. J. Yu, Y. Shen, M. Wang, H. Zhang, and P. H. Torr, “Learning to hash naturally sorts,” arXiv preprint arXiv:2201.13322, 2022.
  40. L. Yuan, T. Wang, X. Zhang, F. E. Tay, Z. Jie, W. Liu, and J. Feng, “Central similarity quantization for efficient image and video retrieval,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 3083–3092.
  41. X. Zheng, Y. Zhang, and X. Lu, “Deep balanced discrete hashing for image retrieval,” Neurocomputing, vol. 403, pp. 224–236, 2020.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets