Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Frequency Domain Modality-invariant Feature Learning for Visible-infrared Person Re-Identification (2401.01839v2)

Published 3 Jan 2024 in cs.CV

Abstract: Visible-infrared person re-identification (VI-ReID) is challenging due to the significant cross-modality discrepancies between visible and infrared images. While existing methods have focused on designing complex network architectures or using metric learning constraints to learn modality-invariant features, they often overlook which specific component of the image causes the modality discrepancy problem. In this paper, we first reveal that the difference in the amplitude component of visible and infrared images is the primary factor that causes the modality discrepancy and further propose a novel Frequency Domain modality-invariant feature learning framework (FDMNet) to reduce modality discrepancy from the frequency domain perspective. Our framework introduces two novel modules, namely the Instance-Adaptive Amplitude Filter (IAF) module and the Phrase-Preserving Normalization (PPNorm) module, to enhance the modality-invariant amplitude component and suppress the modality-specific component at both the image- and feature-levels. Extensive experimental results on two standard benchmarks, SYSU-MM01 and RegDB, demonstrate the superior performance of our FDMNet against state-of-the-art methods.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Frequency domain image translation: More photo-realistic, better identity-preserving. In CVPR, pages 13930–13940, 2021.
  2. Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain. In ICCV, pages 458–467, 2021a.
  3. Neural feature search for rgb-infrared person re-identification. In CVPR, pages 587–597, 2021b.
  4. Fast non-local neural networks with spectral residual learning. In ACM MM, pages 2142–2151, 2019.
  5. Hi-cmd: Hierarchical cross-modality disentanglement for visible-infrared person re-identification. In CVPR, pages 10257–10266, 2020.
  6. Cross-modality person re-identification with generative adversarial training. In IJCAI, page 2, 2018.
  7. Shape-erased feature learning for visible-infrared person re-identification. In CVPR, pages 22752–22761, 2023.
  8. Fftw: An adaptive software architecture for the fft. In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1381–1384. IEEE, 1998.
  9. Cm-nas: Cross-modality neural architecture search for visible-infrared person re-identification. In ICCV, pages 11823–11832, 2021.
  10. Generative adversarial nets. NeurIps, 27, 2014.
  11. Cross-modality person re-identification via modality confusion and center aggregation. In ICCV, pages 16403–16412, 2021.
  12. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.
  13. Transreid: Transformer-based object re-identification. In ICCV, pages 15013–15022, 2021.
  14. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017.
  15. Modality-adaptive mixup and invariant decomposition for rgb-infrared person re-identification. In AAAI, pages 1034–1042, 2022.
  16. Cross-modality transformer for visible-infrared person re-identification. In ECCV, pages 480–496, 2022.
  17. Focal frequency loss for image reconstruction and synthesis. In ICCV, pages 13919–13929, 2021.
  18. Decompose, adjust, compose: Effective normalization by playing with frequency for domain generalization. arXiv preprint arXiv:2303.02328, 2023.
  19. Infrared-visible cross-modal person re-identification with an x modality. In AAAI, pages 4610–4617, 2020.
  20. Harmonious attention network for person re-identification. In CVPR, pages 2285–2294, 2018.
  21. Diverse part discovery: Occluded person re-identification with part-aware transformer. In CVPR, pages 2898–2907, 2021.
  22. Visible-infrared person re-identification with modality-specific memory network. TIP, 31:7165–7178, 2022.
  23. Parameter sharing exploration and hetero-center triplet loss for visible-thermal person re-identification. IEEE Transactions on Multimedia, 2020.
  24. Learning memory-augmented unidirectional metrics for cross-modality person re-identification. In CVPR, pages 19366–19375, 2022.
  25. Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors, 17(3):605, 2017.
  26. The importance of phase in signals. Proceedings of the IEEE, 69(5):529–541, 1981.
  27. A demonstration of the visual importance and flexibility of spatial-frequency amplitude and phase. Perception, 11(3):337–346, 1982.
  28. Global filter networks for image classification. NeurIps, 34:980–993, 2021.
  29. Deep perceptual mapping for cross-modal face recognition. International Journal of Computer Vision, pages 426–438, 2017.
  30. Not all pixels are matched: Dense contrastive learning for cross-modality person re-identification. In ACM MM, pages 5333–5341, 2022.
  31. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In ECCV, pages 480–496, 2018.
  32. Rgb-infrared cross-modality person re-identification via joint pixel and feature alignment. In ICCV, pages 3623–3632, 2019a.
  33. Cross-modality paired-images generation for rgb-infrared person re-identification. In AAAI, pages 12144–12151, 2020a.
  34. High-frequency component helps explain the generalization of convolutional neural networks. In CVPR, pages 8684–8694, 2020b.
  35. Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In CVPR, pages 618–626, 2019b.
  36. Co-attentive lifting for infrared-visible person re-identification. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1028–1037, 2020.
  37. Syncretic modality collaborative learning for visible infrared person re-identification. In ICCV, pages 225–234, 2021.
  38. Rgb-infrared cross-modality person re-identification. In ICCV, pages 5380–5389, 2017.
  39. Discover cross-modality nuances for visible-infrared person re-identification. In CVPR, pages 4330–4339, 2021.
  40. Learning in the frequency domain. In CVPR, pages 1740–1749, 2020.
  41. A fourier-based framework for domain generalization. In CVPR, pages 14383–14392, 2021.
  42. Hierarchical discriminative learning for visible thermal person re-identification. In AAAI, 2018a.
  43. Visible thermal person re-identification via dual-constrained top-ranking. In IJCAI, page 2, 2018b.
  44. Cross-modality person re-identification via modality-aware collaborative ensemble learning. TIP, 29:9387–9399, 2020a.
  45. Dynamic dual-attentive aggregation learning for visible-infrared person re-identification. In ECCV, pages 229–247. Springer, 2020b.
  46. Visible-infrared person re-identification via homogeneous augmented tri-modal learning. IEEE Transactions on Information Forensics and Security, 16:728–739, 2020c.
  47. Channel augmented joint learning for visible-infrared recognition. In ICCV, pages 13567–13576, 2021a.
  48. Deep learning for person re-identification: A survey and outlook. IEEE transactions on pattern analysis and machine intelligence, 44(6):2872–2893, 2021b.
  49. Fmcnet: Feature-level modality compensation for visible-infrared person re-identification. In CVPR, pages 7349–7358, 2022.
  50. Mrcn: A novel modality restitution and compensation network for visible-infrared person re-identification. arXiv preprint arXiv:2303.14626, 2023.
  51. Person re-identification: Past, present and future. arXiv preprint arXiv:1610.02984, 2016.
  52. Re-ranking person re-identification with k-reciprocal encoding. In CVPR, pages 1318–1327, 2017.
  53. Random erasing data augmentation. In AAAI, pages 13001–13008, 2020.
  54. Dual cross-attention learning for fine-grained visual categorization and object re-identification. In CVPR, pages 4692–4702, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Yulin Li (35 papers)
  2. Tianzhu Zhang (61 papers)
  3. Yongdong Zhang (119 papers)
Citations (2)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets