Harmonious Attention Network for Person Re-Identification (1802.08122v1)

Abstract: Existing person re-identification (re-id) methods either assume the availability of well-aligned person bounding box images as model input or rely on constrained attention selection mechanisms to calibrate misaligned images. They are therefore sub-optimal for re-id matching in arbitrarily aligned person images potentially with large human pose variations and unconstrained auto-detection errors. In this work, we show the advantages of jointly learning attention selection and feature representation in a Convolutional Neural Network (CNN) by maximising the complementary information of different levels of visual attention subject to re-id discriminative learning constraints. Specifically, we formulate a novel Harmonious Attention CNN (HA-CNN) model for joint learning of soft pixel attention and hard regional attention along with simultaneous optimisation of feature representations, dedicated to optimise person re-id in uncontrolled (misaligned) images. Extensive comparative evaluations validate the superiority of this new HA-CNN model for person re-id over a wide variety of state-of-the-art methods on three large-scale benchmarks including CUHK03, Market-1501, and DukeMTMC-ReID.

• The paper introduces a Harmonious Attention Network (HA-CNN) that jointly learns soft pixel and hard regional attention to mitigate misalignment in re-identification.
• It employs cross-attention interaction between local and global branches to enhance feature representation despite varied poses and detection errors.
• Experimental results on Market-1501, DukeMTMC-ReID, and CUHK03 demonstrate state-of-the-art accuracy with improved computational efficiency.

Harmonious Attention Network for Person Re-Identification

The paper "Harmonious Attention Network for Person Re-Identification" by Wei Li, Xiatian Zhu, and Shaogang Gong proposes an advanced Convolutional Neural Network (CNN) architecture to address the intricate challenges in person re-identification (re-id). Person re-id involves matching pedestrians across non-overlapping camera views, which is critical for automated surveillance systems.

Problem Statement and Contributions

Re-id systems face challenges due to variations in human poses and auto-detection errors, which result in misaligned images. Traditional methods fail to adequately handle these variations or rely on constrained attention mechanisms that are not optimal. This research proposes the Harmonious Attention Convolutional Neural Network (HA-CNN) to jointly learn attention selection and feature representation, maximizing the complementary information from multi-granularity attention levels.

The key contributions of this work are:

1. Joint Learning of Multi-Granularity Attention: The paper introduces a novel approach where the HA-CNN jointly learns soft pixel attention and hard regional attention to optimize re-id in misaligned images.
2. Harmonious Attention Module: The authors design a HA-CNN to learn both soft and hard attention, leveraging a lightweight network architecture that ensures efficient and effective learning.
3. Cross-Attention Interaction Learning: This novel learning scheme enhances compatibility between different types of attention and feature representation under re-id constraints.

Methodology

Harmonious Attention Learning

The HA-CNN consists of two main branches: a local branch for learning features from local regions and a global branch for global feature learning. The network uses Inception units for both branches and shares weights across certain layers to reduce parameters.

1. Soft Spatial-Channel Attention: The HA module jointly learns spatial and channel attention maps in a factorized manner:
   • Spatial Attention focuses on pixel-level importance.
   • Channel Attention models inter-channel importance.
2. Hard Regional Attention: This component locates latent discriminative regions in an input image using a transformation matrix. It enables the network to learn attention at multiple levels progressively.
3. Cross-Attention Interaction Learning: This component enriches feature learning by allowing interaction between the attended features of the global and local branches.

Experiments and Results

The HA-CNN was evaluated on three large-scale re-id datasets: CUHK03, Market-1501, and DukeMTMC-ReID. The network outperformed state-of-the-art methods significantly:

• Market-1501: HA-CNN surpassed all competitors, with Rank-1 accuracy of 91.2% and mAP of 75.7% in the single-query setting.
• DukeMTMC-ReID: Achieved Rank-1 accuracy of 80.5% and mAP of 63.8%.
• CUHK03: Outperformed the best alternatives in both manually labeled and detected settings.

Key Observations

1. Effectiveness of Multi-Level Attention: The results affirmed that combining soft spatial and channel attentions leads to performance gains. The integration of hard regional attention further enhances this capability.
2. Cross-Attention Interaction Learning: This scheme proved to be beneficial in optimizing the harmony and compatibility of attended features across branches, thus improving re-id performance.
3. Model Efficiency: Despite a smaller parameter size and reduced computational complexity, HA-CNN outperformed models like ResNet50 that depend on ImageNet pre-training and extensive data augmentation.

Implications and Future Directions

The HA-CNN represents a robust solution for re-id by effectively managing misalignment and diverse appearance variations. Its lightweight design makes it suitable for real-time deployment in large-scale surveillance systems. Future work could explore:

1. Extending the HA-CNN to Other Domains: Applying this multi-level attention mechanism to other computer vision tasks such as object detection and action recognition.
2. Enhancing Feature Discrimination: Investigating more sophisticated attention interaction mechanisms and feature fusion strategies for further improvements in performance.
3. Scalability and Efficiency: Developing more efficient training algorithms and examining the model's scalability for even larger datasets and real-world applications.

In conclusion, the proposed HA-CNN significantly advances the capability of person re-identification systems through its innovative attention mechanisms and efficient network architecture, achieving remarkable accuracy and robustness in challenging scenarios.