Mining the Benefits of Two-stage and One-stage HOI Detection (2108.05077v2)

Abstract: Two-stage methods have dominated Human-Object Interaction (HOI) detection for several years. Recently, one-stage HOI detection methods have become popular. In this paper, we aim to explore the essential pros and cons of two-stage and one-stage methods. With this as the goal, we find that conventional two-stage methods mainly suffer from positioning positive interactive human-object pairs, while one-stage methods are challenging to make an appropriate trade-off on multi-task learning, i.e., object detection, and interaction classification. Therefore, a core problem is how to take the essence and discard the dregs from the conventional two types of methods. To this end, we propose a novel one-stage framework with disentangling human-object detection and interaction classification in a cascade manner. In detail, we first design a human-object pair generator based on a state-of-the-art one-stage HOI detector by removing the interaction classification module or head and then design a relatively isolated interaction classifier to classify each human-object pair. Two cascade decoders in our proposed framework can focus on one specific task, detection or interaction classification. In terms of the specific implementation, we adopt a transformer-based HOI detector as our base model. The newly introduced disentangling paradigm outperforms existing methods by a large margin, with a significant relative mAP gain of 9.32% on HICO-Det. The source codes are available at https://github.com/YueLiao/CDN.

• The paper introduces the Cascade Disentangling Network (CDN) that separates human-object detection from interaction classification to enhance HOI detection accuracy.
• It compares two-stage and one-stage methods, showing that blending their strengths overcomes individual inefficiencies in detecting complex interactions.
• Empirical results on the HICO-Det benchmark indicate a 9.32% mAP gain, highlighting the approach's potential for efficient real-time applications.

Insights into Human-Object Interaction Detection: Two-Stage and One-Stage Approaches

The paper presents a comparative investigation into two prevalent paradigms of Human-Object Interaction (HOI) detection methods: two-stage and one-stage frameworks. These methodologies have been foundational in advancing the machine understanding of complex human-object interaction scenes from static images. The authors propose a novel approach named Cascade Disentangling Network (CDN), which combines the strengths of both paradigms while mitigating their individual weaknesses.

Comparative Analysis of Two-Stage and One-Stage Methods

Two-stage methods for HOI detection traditionally involve a sequential process: first, detecting human and object instances, followed by pairing these detections for interaction classification. The main advantage of the two-stage approach lies in its ability to separate the detection and classification processes, allowing each to be optimized independently. However, this separation also results in inefficiencies, particularly due to the increased computational cost associated with evaluating a large number of negative samples and the suboptimal integration of interactive context for accurate classification.

One-stage approaches address some of these inefficiencies by adopting an end-to-end paradigm that simultaneously detects and classifies interactions. While one-stage methods are adept at pinpointing interactive pairs efficiently and with reduced computational overhead, their performance is often hampered by the necessity of balancing dual tasks — object detection and interaction classification — within a single framework.

Cascade Disentangling Network (CDN) Approach

The proposed CDN framework aims to leverage the advantages of both paradigms. It adopts a one-stage framework with a cascade structure to disentangle human-object detection from interaction classification. The CDN consists of two main components:

1. Human-Object Pair Decoder (HO-PD): Utilizes a state-of-the-art one-stage HOI detector, tweaking its structure by removing the interaction classification head to focus solely on detecting human-object pairs. The HO-PD outputs are subsequently used to guide interaction classification in the next stage.
2. Interaction Decoder: A separate module that classifies interactions based on the outputs from the HO-PD. This allows the interaction classification to focus on its specific task without being encumbered by detection-related processes.

This disentangled approach enables CDN to achieve superior performance, as evidenced by a substantial 9.32% improvement in mean average precision (mAP) on the HICO-Det benchmark compared to existing methods.

Implications and Future Directions

The implications of CDN are twofold. Practically, this approach presents a more efficient method for HOI detection in terms of computational resources and accuracy, particularly for complex scenes with high interaction diversity. Theoretically, it suggests a potential rethinking of how multi-task learning paradigms are approached in machine learning, particularly for tasks that traditionally require tightly coupled processes.

Future work in the domain of HOI detection could explore further refinements of the cascade mechanism to optimize the information flow between detection and classification tasks. Moreover, extending this approach to real-time systems implementations could significantly enhance applications in surveillance, autonomous systems, and advanced human-machine interaction frameworks.

In summary, the paper critically examines the strengths and limitations of current HOI detection methods and pioneers an innovative approach that adeptly synergizes the two-stage and one-stage paradigms, setting a new benchmark in the field.