Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation (2008.05676v2)

Abstract: Despite the previous success of object analysis, detecting and segmenting a large number of object categories with a long-tailed data distribution remains a challenging problem and is less investigated. For a large-vocabulary classifier, the chance of obtaining noisy logits is much higher, which can easily lead to a wrong recognition. In this paper, we exploit prior knowledge of the relations among object categories to cluster fine-grained classes into coarser parent classes, and construct a classification tree that is responsible for parsing an object instance into a fine-grained category via its parent class. In the classification tree, as the number of parent class nodes are significantly less, their logits are less noisy and can be utilized to suppress the wrong/noisy logits existed in the fine-grained class nodes. As the way to construct the parent class is not unique, we further build multiple trees to form a classification forest where each tree contributes its vote to the fine-grained classification. To alleviate the imbalanced learning caused by the long-tail phenomena, we propose a simple yet effective resampling method, NMS Resampling, to re-balance the data distribution. Our method, termed as Forest R-CNN, can serve as a plug-and-play module being applied to most object recognition models for recognizing more than 1000 categories. Extensive experiments are performed on the large vocabulary dataset LVIS. Compared with the Mask R-CNN baseline, the Forest R-CNN significantly boosts the performance with 11.5% and 3.9% AP improvements on the rare categories and overall categories, respectively. Moreover, we achieve state-of-the-art results on the LVIS dataset. Code is available at https://github.com/JialianW/Forest_RCNN.

• The paper introduces Forest R-CNN, which employs a classification forest to effectively mitigate noisy classifier logits in large-vocabulary object detection.
• It tackles long-tailed data imbalance with NMS Resampling, an adaptive strategy that rebalances instance proposals during training based on category frequency.
• Validation on LVIS datasets shows significant improvements, achieving an 11.5% AP increase for rare categories and surpassing state-of-the-art results.

Forest R-CNN: Enhancements in Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation

The paper "Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation" addresses a significant challenge in the field of object detection and instance segmentation: effectively managing a broad range of object categories that exhibit a long-tailed data distribution. Despite advancements in object recognition technologies, handling large-vocabulary object detection remains an intricate issue due to the increased prevalence of noisy classifier logits and inherent class imbalance attributed to long-tailed distributions. This paper makes substantive contributions by introducing a novel model, Forest R-CNN, which significantly mitigates these challenges through innovative methodologies.

Main Contributions

1. Classification Forest for Noisy Logit Mitigation:
   • The authors propose the concept of a classification forest to address the high occurrence of noisy logits in large-vocabulary classifiers. Unlike conventional single-layered approaches, the classification forest consists of multiple hierarchical classification trees which parse fine-grained categories through parent class nodes. Logits in parent class nodes, being fewer, are less noisy and are utilized to recalibrate and suppress incorrect logits at the fine-grained level.
   • Each classification tree in the forest leverages different types of prior knowledge, such as lexical and visual relations, allowing for comprehensive classification by amalgamating the "votes" from each tree to determine the final category labels.
2. NMS Resampling for Imbalanced Data Distribution:
   • The imbalance in data distribution is tackled through a novel resampling strategy called NMS Resampling. This approach adaptively adjusts the Non-Maximum Suppression (NMS) threshold based on category frequency, retaining more proposals for lesser-represented tail classes while limiting those for overly-represented head classes.
   • This technique does not alter the image-level data but rather focuses on rebalancing at the instance-level during the training phase, effectively avoiding issues like overfitting and excessive computation-time overhead.
3. Experimental Validation on LVIS Dataset:
   • Extensive evaluations on the LVIS (Large Vocabulary Instance Segmentation) datasets v0.5 and v1.0 showcase the efficacy of the Forest R-CNN. Comparisons with a baseline Mask R-CNN demonstrate significant improvements with an 11.5% increase in Average Precision (AP) for rare categories and a 3.9% boost for overall categories.
   • The method surpasses state-of-the-art results in most configurations, especially shining in recognizing rare categories which are typically underrepresented.

Implications and Future Prospects

Forest R-CNN sets a new benchmark not only for handling a large vocabulary of objects but also for integrating structural and hierarchical insights into the classification process. The use of multiple classification trees employing different knowledge types suggests an intriguing potential for further cross-disciplinary research, potentially incorporating even more varied sources of semantic relations.

Furthermore, the adaptive NMS Resampling approach introduces an efficient means of rebalancing datasets without extensive computational costs, opening future avenues for its application in other scenarios facing similar class imbalance issues.

Given the demonstrated success of the Forest R-CNN in object detection and instance segmentation, future efforts might look into extending similar methodologies across other domains of computer vision, such as video analysis or even multi-modal learning where hierarchical and imbalanced datasets are prevalent. Additionally, exploring automated ways to generate and validate the prior knowledge required for forest classification structures might yield even broader applicability and versatility in various machine learning domains.