Test-time Similarity Modification for Person Re-identification toward Temporal Distribution Shift (2403.14114v1)
Abstract: Person re-identification (re-id), which aims to retrieve images of the same person in a given image from a database, is one of the most practical image recognition applications. In the real world, however, the environments that the images are taken from change over time. This causes a distribution shift between training and testing and degrades the performance of re-id. To maintain re-id performance, models should continue adapting to the test environment's temporal changes. Test-time adaptation (TTA), which aims to adapt models to the test environment with only unlabeled test data, is a promising way to handle this problem because TTA can adapt models instantly in the test environment. However, the previous TTA methods are designed for classification and cannot be directly applied to re-id. This is because the set of people's identities in the dataset differs between training and testing in re-id, whereas the set of classes is fixed in the current TTA methods designed for classification. To improve re-id performance in changing test environments, we propose TEst-time similarity Modification for Person re-identification (TEMP), a novel TTA method for re-id. TEMP is the first fully TTA method for re-id, which does not require any modification to pre-training. Inspired by TTA methods that refine the prediction uncertainty in classification, we aim to refine the uncertainty in re-id. However, the uncertainty cannot be computed in the same way as classification in re-id since it is an open-set task, which does not share person labels between training and testing. Hence, we propose re-id entropy, an alternative uncertainty measure for re-id computed based on the similarity between the feature vectors. Experiments show that the re-id entropy can measure the uncertainty on re-id and TEMP improves the performance of re-id in online settings where the distribution changes over time.
- A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012.
- K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” in International Conference on Learning Representations (ICLR), 2015.
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2016.
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” in International Conference on Learning Representations (ICLR), 2021. [Online]. Available: https://openreview.net/forum?id=YicbFdNTTy
- E. Ahmed, M. Jones, and T. K. Marks, “An Improved Deep Learning Architecture for Person Re-Identification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
- W. Deng, L. Zheng, Q. Ye, G. Kang, Y. Yang, and J. Jiao, “Image-Image Domain Adaptation With Preserved Self-Similarity and Domain-Dissimilarity for Person Re-Identification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
- J. Wang, X. Zhu, S. Gong, and W. Li, “Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-Identification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
- C. Yang, F. Qi, and H. Jia, “Survey on Unsupervised Techniques for Person Re-Identification,” in 2nd International Conference on Computing and Data Science (CDS), 2021.
- D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell, “Tent: Fully Test-Time Adaptation by Entropy Minimization,” in International Conference on Learning Representations (ICLR), 2021.
- A. Zhou and S. Levine, “Bayesian Adaptation for Covariate Shift,” Advances in Neural Information Processing Systems, 2021.
- S. Niu, J. Wu, Y. Zhang, Y. Chen, S. Zheng, P. Zhao, and M. Tan, “Efficient test-time model adaptation without forgetting,” in Proceedings of the 39th International Conference on Machine Learning (ICML), 2022.
- K. Han, C. Si, Y. Huang, L. Wang, and T. Tan, “Generalizable Person Re-identification via Self-Supervised Batch Norm Test-Time Adaption,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
- L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, and Q. Tian, “Scalable Person Re-Identification: A Benchmark,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015.
- L. Wei, S. Zhang, W. Gao, and Q. Tian, “Person Transfer GAN to Bridge Domain Gap for Person Re-Identification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
- X. Sun and L. Zheng, “Dissecting Person Re-Identification From the Viewpoint of Viewpoint,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
- Y. Lin, X. Dong, L. Zheng, Y. Yan, and Y. Yang, “A bottom-up clustering approach to unsupervised person re-identification,” in Proceedings of the AAAI conference on artificial intelligence, 2019.
- Y. Ge, F. Zhu, D. Chen, R. Zhao, and h. Li, “Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID,” in Advances in Neural Information Processing Systems, 2020.
- Y. Ge, D. Chen, and H. Li, “Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification,” in International Conference on Learning Representations (ICLR), 2020.
- M. Zhang, K. Liu, Y. Li, S. Guo, H. Duan, Y. Long, and Y. Jin, “Unsupervised domain adaptation for person re-identification via heterogeneous graph alignment,” in Proceedings of the AAAI conference on artificial intelligence, 2021.
- Z. Zhong, L. Zheng, Z. Luo, S. Li, and Y. Yang, “Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-Identification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
- Z. Huang, Z. Zhang, C. Lan, W. Zeng, P. Chu, Q. You, J. Wang, Z. Liu, and Z.-J. Zha, “Lifelong Unsupervised Domain Adaptive Person Re-Identification With Coordinated Anti-Forgetting and Adaptation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proceedings of the 32nd International Conference on Machine Learning (ICML), 2015.
- P. Benz, C. Zhang, A. Karjauv, and I. S. Kweon, “Revisiting Batch Normalization for Improving Corruption Robustness,” in Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), 2021.
- B. Zhao, C. Chen, and S.-T. Xia, “DELTA: DEGRADATION-FREE FULLY TEST-TIME ADAPTATION,” in International Conference on Learning Representations (ICLR), 2023. [Online]. Available: https://openreview.net/forum?id=eGm22rqG93
- X. LI, Y. Grandvalet, and F. Davoine, “Explicit Inductive Bias for Transfer Learning with Convolutional Networks,” in Proceedings of the 35th International Conference on Machine Learning (ICML), 2018.
- Q. Qian, L. Shang, B. Sun, J. Hu, H. Li, and R. Jin, “SoftTriple Loss: Deep Metric Learning Without Triplet Sampling,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR), 2015.
- Y. Ge, T. Xiao, and Z. Zhang, “OpenUnReID,” GitHub Repository, 2020, https://github.com/open-mmlab/OpenUnReID.
- X. Lin, P. Ren, C.-H. Yeh, L. Yao, A. Song, and X. Chang, “Unsupervised person re-identification: A systematic survey of challenges and solutions,” arXiv preprint arXiv:2109.06057, 2021.
- L. McInnes, J. Healy, N. Saul, and L. Grossberger, “UMAP: Uniform Manifold Approximation and Projection,” The Journal of Open Source Software, vol. 3, no. 29, p. 861, 2018.