NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection (1904.07392v1)

Abstract: Current state-of-the-art convolutional architectures for object detection are manually designed. Here we aim to learn a better architecture of feature pyramid network for object detection. We adopt Neural Architecture Search and discover a new feature pyramid architecture in a novel scalable search space covering all cross-scale connections. The discovered architecture, named NAS-FPN, consists of a combination of top-down and bottom-up connections to fuse features across scales. NAS-FPN, combined with various backbone models in the RetinaNet framework, achieves better accuracy and latency tradeoff compared to state-of-the-art object detection models. NAS-FPN improves mobile detection accuracy by 2 AP compared to state-of-the-art SSDLite with MobileNetV2 model in [32] and achieves 48.3 AP which surpasses Mask R-CNN [10] detection accuracy with less computation time.

An Overview of NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

The paper "NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection," authored by Golnaz Ghiasi, Tsung-Yi Lin, Ruoming Pang, and Quoc V. Le, introduces a novel approach to designing feature pyramid networks (FPN) for object detection using Neural Architecture Search (NAS). This research establishes a new framework to dynamically discover scalable feature pyramid architectures optimized for object detection.

The core objective of this paper is to improve feature representation across multiple scales, which is a significant challenge in object detection. Traditionally, FPNs are manually designed and may not provide the optimal architecture required for various detection tasks. The authors address this limitation by leveraging NAS to discover a superior pyramid architecture, termed NAS-FPN, which balances the tradeoff between accuracy and latency more effectively than existing state-of-the-art models.

Key Contributions

1. Neural Architecture Search (NAS) for FPN: The paper introduces a systematic method using NAS to explore a comprehensive search space encompassing all possible cross-scale connections. This automated approach aims to discover a flexible, high-performance feature pyramid architecture that supersedes manually designed counterparts.
2. Scalability and Flexibility: NAS-FPN's modular architecture allows for replication and stacking, making it scalable. The flexibility of NAS-FPN is demonstrated by its compatibility with various backbone models, such as MobileNet, ResNet, and AmoebaNet.
3. Performance Metrics: The experimental results indicate significant improvements in Average Precision (AP) across different backbones and image sizes. For instance, using NAS-FPN combined with MobileNetV2, the model achieves a 2 AP increase over the state-of-the-art SSDLite with MobileNetV2. Additionally, NAS-FPN, when paired with AmoebaNet, achieves 48.3 AP, surpassing the detection accuracy of Mask R-CNN with reduced inference time.

Methodology

The methodology revolves around the use of NAS to discover optimal architectures for feature pyramid networks. The primary steps involve:

1. Architecture Search Space: The search space is designed to include a variety of cross-scale connections, thereby allowing the model to combine high and low-resolution features effectively. During the search, an RNN controller predicts the architecture, which is trained on a proxy task—a reduced version of the object detection problem to speed up the process.
2. Merging Cells: The FPN is constructed using "merging cells" that merge features at different scales. Each merging cell takes two feature layers as input and combines them using operations like summation and global pooling, followed by convolution and batch normalization.
3. Training and Evaluation: The discovered NAS-FPN architectures undergo rigorous training and evaluation using the COCO dataset. The results are compared with multiple baselines, showcasing the efficiency and superiority of NAS-FPN in both high-accuracy and fast-inference scenarios.

Experimental Results

The paper provides extensive experimentation to validate the efficacy of NAS-FPN. The primary benchmarks include:

• Backbone Variations: NAS-FPN outperforms traditional FPNs across different backbone architectures. For example, NAS-FPN with MobileNetV2 achieves 36.6 AP on a 640x640 image size, whereas using AmoebaNet increases the AP to 48.0.
• Stacking and Feature Dimension: The architecture's scalability is demonstrated by stacking pyramid networks and adjusting feature dimensions. Increasing the number of pyramid networks consistently improves performance, indicating the robustness of the discovered architecture.
• Comparison with State-of-the-Art Models: NAS-FPN showcases superior performance in both precision and computational efficiency compared to models like Mask R-CNN and YOLOv3. This is evidenced by significant AP improvements with relatively lower inference times and the ability to achieve high performance even on mobile devices.

Implications and Future Directions

The implications of this research are multifaceted. Practically, NAS-FPN can be adopted in real-world applications requiring efficient and accurate object detection. Theoretically, it sets a precedent for using NAS in discovering domain-specific network architectures, potentially extending to other areas in computer vision.

Future developments could explore the integration of NAS-FPN with other automated machine learning techniques to further optimize different stages of the detection pipeline. Additionally, expanding the search space to incorporate more complex operations and leveraging advanced hardware accelerators could yield even more performant architectures.

In conclusion, NAS-FPN represents a significant advancement in the automated design of feature pyramid networks for object detection, providing a scalable, flexible, and highly performant solution that can cater to diverse real-world requirements.