Learning Background-Aware Correlation Filters for Visual Tracking (1703.04590v2)

Abstract: Correlation Filters (CFs) have recently demonstrated excellent performance in terms of rapidly tracking objects under challenging photometric and geometric variations. The strength of the approach comes from its ability to efficiently learn - "on the fly" - how the object is changing over time. A fundamental drawback to CFs, however, is that the background of the object is not be modelled over time which can result in suboptimal results. In this paper we propose a Background-Aware CF that can model how both the foreground and background of the object varies over time. Our approach, like conventional CFs, is extremely computationally efficient - and extensive experiments over multiple tracking benchmarks demonstrate the superior accuracy and real-time performance of our method compared to the state-of-the-art trackers including those based on a deep learning paradigm.

• The paper presents BACF, which integrates real negative background patches into correlation filters to reduce drift in cluttered scenes.
• It employs an efficient ADMM-based optimization using multi-channel HOG features that enables real-time tracking at around 35.3 FPS.
• The method consistently outperforms state-of-the-art trackers on benchmarks like OTB and VOT, balancing accuracy with computational efficiency.

Learning Background-Aware Correlation Filters for Visual Tracking

The paper "Learning Background-Aware Correlation Filters for Visual Tracking" by Hamed Kiani Galoogahi et al., presents a method to enhance the performance of Correlation Filters (CFs) for visual tracking by incorporating background-aware learning. The central innovation proposed is the Background-Aware Correlation Filter (BACF) which leverages hand-crafted features, particularly Histogram of Oriented Gradients (HOG), to efficiently model both foreground and background dynamics over time.

Correlation Filters are well-regarded for their ability to rapidly adapt to changes through online learning, a characteristic that makes them suitable for tracking objects under varied and dynamic conditions. Despite their computational efficiency and robustness, traditional CFs struggle to incorporate background information during training, leading to suboptimal performance, particularly in cluttered environments. The BACF method addresses this gap by integrating background patches into the learning process, thereby enhancing the filter's discriminative power against background noise.

Technical Contributions

The paper outlines several key contributions:

1. Background-Aware Training: Unlike conventional CF-based trackers that train using circularly shifted patches, the BACF method extracts real negative examples from the background. This approach enhances the filter's ability to distinguish between the target and cluttered backgrounds, minimizing tracking drift resulting from similar visual cues in the target and background.
2. Efficient Optimization Framework: The authors propose an Alternating Direction Method of Multipliers (ADMM)-based approach to learn the BACF from multi-channel HOG features. This method operates with a computational complexity of $\mathcal{O}(L K T \log(T))$, where $T$ is the size of the vectorized frame, $K$ is the number of feature channels, and $L$ is the number of ADMM iterations. This ensures that the BACF method remains computationally efficient, capable of real-time performance.
3. Online Adaptation Strategy: To maintain the tracking robustness over time, the BACF employs an online model adaptation strategy using the Sherman-Morrison lemma for managing changes in target and background appearances. This ongoing adaptation ensures the tracker maintains its efficacy despite significant photometric and geometric changes.

Experimental Analysis

The BACF method was extensively evaluated against 24 state-of-the-art trackers on several benchmarks including OTB50, OTB100, Temple-Color128 (TC128), and VOT2015. Notably:

• OTB100: The BACF achieved an AUC of 62.98, outperforming SRDCF (60.13) and Staple (58.03).
• Speed: The BACF achieved a real-time tracking speed of approximately 35.3 FPS on a CPU, significantly outpacing both SRDCF (3.8 FPS) and DeepSRDCF (0.4 FPS).
• Attributes: The BACF demonstrated superior performance across various challenging attributes such as background clutter, occlusion, and scale variation, further validating its robustness and adaptability.

Comparative Performance

In comparison to deep feature-based CF trackers like HCF, DeepSRDCF, and CCOT, the BACF showcased a notable balance between tracking accuracy and speed. While deep feature-based methods offered marginal improvements in accuracy, their computational expense rendered them impractical for real-time applications. The BACF's average success rate was comparable to the best deep trackers but was achieved with nearly 170 times the computational efficiency.

Furthermore, comparisons against purely deep learning-based trackers such as MDNet and SiamFC illustrated the BACF's competitive accuracy coupled with far superior real-time performance, demonstrating the practical advantages of leveraging efficient hand-crafted features together with a robust filter learning strategy.

Implications and Future Directions

The BACF method presents significant implications for the development of visual tracking systems, particularly in scenarios where computational resources are limited, or real-time performance is critical. By integrating background awareness into the correlation filter framework, the BACF addresses a longstanding limitation, leading to more robust and efficient tracking.

Future research could explore the integration of more advanced feature extraction methods or hybrid models that combine the strengths of both deep learning and efficient hand-crafted features. Additionally, extending the BACF framework to handle more complex scenes and multi-object tracking scenarios could further enhance its applicability and robustness.

Conclusion

The paper delivers a compelling approach to improving correlation filter-based visual tracking via background-aware learning. The BACF method attains a commendable balance between computational efficiency and tracking accuracy, showcasing superior real-time performance over numerous challenging benchmarks. These advancements mark a significant step forward in the design of robust and efficient visual tracking systems.