Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Frequency Domain Nuances Mining for Visible-Infrared Person Re-identification (2401.02162v2)

Published 4 Jan 2024 in cs.CV

Abstract: The key of visible-infrared person re-identification (VIReID) lies in how to minimize the modality discrepancy between visible and infrared images. Existing methods mainly exploit the spatial information while ignoring the discriminative frequency information. To address this issue, this paper aims to reduce the modality discrepancy from the frequency domain perspective. Specifically, we propose a novel Frequency Domain Nuances Mining (FDNM) method to explore the cross-modality frequency domain information, which mainly includes an amplitude guided phase (AGP) module and an amplitude nuances mining (ANM) module. These two modules are mutually beneficial to jointly explore frequency domain visible-infrared nuances, thereby effectively reducing the modality discrepancy in the frequency domain. Besides, we propose a center-guided nuances mining loss to encourage the ANM module to preserve discriminative identity information while discovering diverse cross-modality nuances. Extensive experiments show that the proposed FDNM has significant advantages in improving the performance of VIReID. Specifically, our method outperforms the second-best method by 5.2\% in Rank-1 accuracy and 5.8\% in mAP on the SYSU-MM01 dataset under the indoor search mode, respectively. Besides, we also validate the effectiveness and generalization of our method on the challenging visible-infrared face recognition task. \textcolor{magenta}{The code will be available.}

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Learning mappings for face synthesis from near infrared to visual light images. In CVPR, pages 156–163, 2009.
  2. Neural feature search for rgb-infrared person re-identification. In CVPR, pages 587–597, 2021.
  3. Cross-modality person re-identification with memory-based contrastive embedding. In AAAI, pages 425–432, 2023.
  4. Hi-cmd: Hierarchical cross-modality disentanglement for visible-infrared person re-identification. In CVPR, pages 10257–10266, 2020.
  5. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255, 2009.
  6. Cross-spectral face hallucination via disentangling independent factors. In CVPR, 2020.
  7. Visible-infrared person re-identification via semantic alignment and affinity inference. In ICCV, pages 11270–11279, 2023.
  8. Shape-erased feature learning for visible-infrared person re-identification. In CVPR, pages 22752–22761, 2023.
  9. Cm-nas: Cross-modality neural architecture search for visible-infrared person re-identification. In ICCV, pages 11823–11832, 2021.
  10. Mso: Multi-feature space joint optimization network for rgb-infrared person re-identification. In ACM MM, pages 5257–5265, 2021.
  11. Cross-modality person re-identification via modality confusion and center aggregation. In CVPR, pages 16403–16412, 2021.
  12. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.
  13. Learning invariant deep representation for nir-vis face recognition. In AAAI, page 2000–2006, 2017.
  14. In defense of the triplet loss for person re-identification. ArXiv, 2017.
  15. The buaa-visnir face database instructions. School Comput. Sci. Eng., Beihang Univ., Beijing, China, Tech. Rep. IRIP-TR-12-FR-001, 3(3):8, 2012.
  16. Deep fourier-based exposure correction network with spatial-frequency interaction. In ECCV, pages 163–180, 2022.
  17. Partmix: Regularization strategy to learn part discovery for visible-infrared person re-identification. In CVPR, pages 18621–18632, 2023.
  18. Van Der Maaten Laurens and Geoffrey Hinton. Visualizing data using t-sne. In JMLR, pages 2579–2605, 2008.
  19. Decompose, adjust, compose: Effective normalization by playing with frequency for domain generalization. In CVPR, pages 11776–11785, 2023.
  20. Infrared-visible cross-modal person re-identification with an x modality. In AAAI, pages 4610–4617, 2020.
  21. Fsi: Frequency and spatial interactive learning for image restoration in under-display cameras. In ICCV, pages 12537–12546, 2023.
  22. Learning memory-augmented unidirectional metrics for cross-modality person re-identification. In CVPR, pages 19366–19375, 2022.
  23. Learning progressive modality-shared transformers for effective visible-infrared person re-identification. In AAAI, pages 1835–1843, 2023.
  24. Cross-modality person re-identification with shared-specific feature transfer. In CVPR, pages 13379–13389, 2020.
  25. Bag of tricks and a strong baseline for deep person re-identification. In CVPR Workshops, pages 1487–1495, 2019.
  26. Modality-aware style adaptation for rgb-infrared person re-identification. In IJCAI, pages 19–27, 2021.
  27. Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors, 17(3):605, 2017.
  28. Learning by aligning: Visible-infrared person re-identification using cross-modal correspondences. In ICCV, pages 12046–12055, 2021.
  29. Dual gaussian-based variational subspace disentanglement for visible-infrared person re-identification. In ACM MM, pages 2149–2158, 2020.
  30. Dual pseudo-labels interactive self-training for semi-supervised visible-infrared person re-identification. In ICCV, pages 11218–11228, 2023.
  31. Not all pixels are matched: Dense contrastive learning for cross-modality person re-identification. In ACM MM, page 5333–5341, 2022.
  32. Exploring invariant representation for visible-infrared person re-identification. ArXiv, 2023.
  33. Instance normalization: The missing ingredient for fast stylization. ArXiv, 2016.
  34. Spatial-frequency mutual learning for face super-resolution. In CVPR, pages 22356–22366, 2023.
  35. Rgb-infrared cross-modality person re-identification via joint pixel and feature alignment. In ICCV, pages 3623–3632, 2019a.
  36. Cross-modality paired-images generation for rgb-infrared person re-identification. In AAAI, pages 12144–12151, 2020.
  37. Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In CVPR, pages 618–626, 2019b.
  38. Co-attentive lifting for infrared-visible person re-identification. In ACM MM, pages 1028–1037, 2020.
  39. Syncretic modality collaborative learning for visible infrared person re-identification. In ICCV, pages 225–234, 2021.
  40. Rgb-infrared cross-modality person re-identification. In ICCV, pages 5380–5389, 2017.
  41. Learning concordant attention via target-aware alignment for visible-infrared person re-identification. In ICCV, pages 11122–11131, 2023.
  42. Discover cross-modality nuances for visible-infrared person re-identification. In CVPR, pages 4330–4339, 2021.
  43. A light cnn for deep face representation with noisy labels. TIFS, 13(11):2884–2896, 2018a.
  44. Coupled deep learning for heterogeneous face recognition. In AAAI, page 1679–1686, 2018b.
  45. Towards grand unified representation learning for unsupervised visible-infrared person re-identification. In ICCV, pages 11069–11079, 2023.
  46. Learning with twin noisy labels for visible-infrared person re-identification. In CVPR, pages 14308–14317, 2022.
  47. Generalized lightness adaptation with channel selective normalization. In ICCV, pages 10668–10679, 2023.
  48. Hierarchical discriminative learning for visible thermal person re-identification. In AAAI, pages 7501–7508, 2018a.
  49. Visible thermal person re-identification via dual-constrained top-ranking. In IJCAI, pages 1092–1099, 2018b.
  50. Modality-aware collaborative learning for visible thermal person re-identification. In ACM MM, pages 347–355, 2019.
  51. Dynamic dual-attentive aggregation learning for visible-infrared person re-identification. In ECCV, pages 229–247, 2020.
  52. Channel augmented joint learning for visible-infrared recognition. In ICCV, pages 13567–13576, 2021a.
  53. Deep learning for person re-identification: A survey and outlook. TPAMI, 44(6):2872–2893, 2021b.
  54. Frequency and spatial dual guidance for image dehazing. In ECCV, pages 181–198, 2022.
  55. Toplight: Lightweight neural networks with task-oriented pretraining for visible-infrared recognition. In CVPR, pages 3541–3550, 2023a.
  56. Modality unifying network for visible-infrared person re-identification. In ICCV, pages 11185–11195, 2023b.
  57. Fmcnet: Feature-level modality compensation for visible-infrared person re-identification. In CVPR, pages 7349–7358, 2022a.
  58. Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification. In CVPR, pages 2153–2162, 2023.
  59. Towards a unified middle modality learning for visible-infrared person re-identification. In ACM MM, pages 788–796, 2021.
  60. Modality synergy complement learning with cascaded aggregation for visible-infrared person re-identification. In ECCV, pages 462–479, 2022b.
  61. Mrcn: A novel modality restitution and compensation network for visible-infrared person re-identification. In AAAI, 37(3):3498–3506, 2023.
  62. Random erasing data augmentation. In AAAI, pages 13001–13008, 2020.
  63. Spatial-frequency domain information integration for pan-sharpening. In ECCV, pages 274–291, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Yukang Zhang (7 papers)
  2. Yang Lu (158 papers)
  3. Yan Yan (242 papers)
  4. Hanzi Wang (66 papers)
  5. Xuelong Li (268 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now