Parallel Tracking and Verifying: A Framework for Real-Time and High Accuracy Visual Tracking (1708.00153v1)

Abstract: Being intensively studied, visual tracking has seen great recent advances in either speed (e.g., with correlation filters) or accuracy (e.g., with deep features). Real-time and high accuracy tracking algorithms, however, remain scarce. In this paper we study the problem from a new perspective and present a novel parallel tracking and verifying (PTAV) framework, by taking advantage of the ubiquity of multi-thread techniques and borrowing from the success of parallel tracking and mapping in visual SLAM. Our PTAV framework typically consists of two components, a tracker T and a verifier V, working in parallel on two separate threads. The tracker T aims to provide a super real-time tracking inference and is expected to perform well most of the time; by contrast, the verifier V checks the tracking results and corrects T when needed. The key innovation is that, V does not work on every frame but only upon the requests from T; on the other end, T may adjust the tracking according to the feedback from V. With such collaboration, PTAV enjoys both the high efficiency provided by T and the strong discriminative power by V. In our extensive experiments on popular benchmarks including OTB2013, OTB2015, TC128 and UAV20L, PTAV achieves the best tracking accuracy among all real-time trackers, and in fact performs even better than many deep learning based solutions. Moreover, as a general framework, PTAV is very flexible and has great rooms for improvement and generalization.

• The paper presents the novel Parallel Tracking and Verifying (PTAV) framework, which uses parallel tracker and verifier components to achieve high accuracy in real-time visual object tracking.
• PTAV uses parallel threads for a fast fDSST tracker and a Siamese network verifier, which strategically validates tracking only when requested by the tracker to optimize computation.
• Evaluated on benchmarks, PTAV achieves superior accuracy among real-time trackers, scoring 89.4% DPR and 82.7% OSR on OTB2013, validating its high-performance capabilities.

Parallel Tracking and Verifying: Enhancing Visual Tracking through a Collaborative Framework

This paper presents a novel framework termed Parallel Tracking and Verifying (PTAV) which significantly improves real-time visual object tracking by integrating a parallel process of tracking and verification. The PTAV framework is designed to enhance both the speed and accuracy of visual tracking systems, addressing the challenge that most algorithms either excel at one or the other.

Core Features and Methodology

At its core, PTAV exploits the advantages of multi-thread processing. It consists of two fundamental components—a tracker (denoted as $\TRK$) tasked with executing super real-time tracking, and a verifier ($\VRF$) designed to ensure tracking accuracy. These components operate on separate threads, synchronizing through an innovative cooperative mechanism where the verifier only functions upon request from the tracker.

1. Tracker Design: For the tracker, the framework employs the fast discriminative scale space tracking (fDSST) as its base. This choice leverages the efficiency of correlation filters, facilitating real-time processing. The framework handles short-duration frame processing, ensuring that computational resources are optimally allocated.
2. Verifier Design: As a counterbalance to the tracker, the verifier uses Siamese networks to validate and correct tracking output as needed. Verification is not continuous but strategic, initiated upon indication from the tracker, thereby reducing unnecessary computational load. The verifier essentially checks a subset of frames to validate tracking continuity and accuracy, enabling corrective measures when necessary.

Experimental Results and Comparative Analysis

The PTAV framework is tested against several benchmark datasets, including OTB2013, OTB2015, TC128, and UAV20L, which comprise diverse and challenging scenarios for object tracking. The results indicate PTAV's superiority concerning tracking accuracy among all real-time tracking algorithms evaluated. Notably, PTAV achieves an 89.4% distance precision rate (DPR) and an 82.7% overlap success rate (OSR) on OTB2013, outperforming many contemporary deep learning methods, which are often computationally heavy and not real-time.

Furthermore, the framework's structure allows it to dynamically adjust the verifying interval based on tracking reliability, showcasing flexibility in handling varied scenarios, including fast-moving objects, occlusions, and illumination changes.

Implications and Future Directions

The development of the PTAV framework has several implications for the computer vision community. Practically, it offers a viable solution for applications requiring both real-time processing and reliable tracking accuracy, such as in robotics and surveillance. Theoretically, it introduces an innovative dual-process architecture that could inspire future exploration in adaptive, resource-efficient tracking solutions.

Looking forward, the flexibility inherent in the PTAV framework indicates considerable room for further optimization and adaptation. Subsequent research could explore integrating more advanced AI-driven tracking algorithms or examining alternative parallel computing strategies to enhance performance further.

Overall, PTAV represents a significant step towards resolving the enduring trade-off between tracking speed and accuracy, offering a robust template for future developments in visual tracking technologies.