Dynamic Head: Unifying Object Detection Heads with Attentions (2106.08322v1)

Abstract: The complex nature of combining localization and classification in object detection has resulted in the flourished development of methods. Previous works tried to improve the performance in various object detection heads but failed to present a unified view. In this paper, we present a novel dynamic head framework to unify object detection heads with attentions. By coherently combining multiple self-attention mechanisms between feature levels for scale-awareness, among spatial locations for spatial-awareness, and within output channels for task-awareness, the proposed approach significantly improves the representation ability of object detection heads without any computational overhead. Further experiments demonstrate that the effectiveness and efficiency of the proposed dynamic head on the COCO benchmark. With a standard ResNeXt-101-DCN backbone, we largely improve the performance over popular object detectors and achieve a new state-of-the-art at 54.0 AP. Furthermore, with latest transformer backbone and extra data, we can push current best COCO result to a new record at 60.6 AP. The code will be released at https://github.com/microsoft/DynamicHead.

Dynamic Head: Unifying Object Detection Heads with Attentions

The paper "Dynamic Head: Unifying Object Detection Heads with Attentions" introduces a novel framework for improving object detection by integrating multiple attention mechanisms. The proposed dynamic head aims to unify scale-awareness, spatial-awareness, and task-awareness within object detection tasks, significantly enhancing the detection capabilities without additional computational costs.

Overview

Object detection has traditionally relied on separately addressing the challenges of localization and classification. Many methods have attempted enhancements in specific object detection head components but lacked a unified approach. This paper presents a dynamic head framework integrating multiple self-attention mechanisms across feature levels, spatial locations, and output channels.

Methodology

The proposed dynamic head functions by applying attention mechanisms on different dimensions of input feature tensors, structured as $level \times space \times channel$. This structured attention application allows for:

1. Scale-awareness: Attention across feature levels adjusts for objects of varying scales.
2. Spatial-awareness: Attention across spatial dimensions addresses spatial transformations in the image, helping distinguish object geometry and location.
3. Task-awareness: Attention across channels supports different detection tasks like classification and various object representations, enhancing specialization.

By handling these aspects separately but coherently, the dynamic head effectively improves representation learning within detection models. These mechanisms are stackable, allowing for deep yet efficient channel, spatial, and scale-aware processing chains.

Experimental Results

The dynamic head demonstrates notable performance improvements on the COCO benchmark. With a ResNeXt-101-DCN backbone, it achieves a state-of-the-art $54.0\% AP$ in standard settings, scaling to $60.6\% AP$ when using a recent transformer backbone and additional data. It integrates efficiently with existing architectures like Faster R-CNN, RetinaNet, and ATSS, providing consistent performance gains across approaches—with overall increases of 1.2% to 3.2% AP.

Comparison and Significance

Compared to other attention-based approaches like deformable convolutions, non-local networks, and transformers, the dynamic head uniquely and comprehensively models attention across all three critical dimensions of a detection task. This modular yet unified approach offers robust performance improvements, offering significant gains in terms of both effectiveness and computational efficiency.

Implications and Future Work

The results imply a strong potential for refining object detection frameworks through attention-driven architectures. The findings encourage exploration of more complex attention modeling while maintaining computational efficiency, potentially integrating further modalities or attention mechanisms without compromising processing speed or model complexity.

Future developments might focus on easing the training of full attention models, ensuring efficient computation, and further extending the scope of attention to cover new perspectives or detection requirements. Such advancements could enhance the adaptability and precision of AI models deployed in diverse environments.

Overall, this paper contributes substantially to the understanding of attention mechanisms in object detection heads, offering a scalable and efficient solution for improving object detection performance.