Learning to Track at 100 FPS with Deep Regression Networks (1604.01802v2)

Abstract: Machine learning techniques are often used in computer vision due to their ability to leverage large amounts of training data to improve performance. Unfortunately, most generic object trackers are still trained from scratch online and do not benefit from the large number of videos that are readily available for offline training. We propose a method for offline training of neural networks that can track novel objects at test-time at 100 fps. Our tracker is significantly faster than previous methods that use neural networks for tracking, which are typically very slow to run and not practical for real-time applications. Our tracker uses a simple feed-forward network with no online training required. The tracker learns a generic relationship between object motion and appearance and can be used to track novel objects that do not appear in the training set. We test our network on a standard tracking benchmark to demonstrate our tracker's state-of-the-art performance. Further, our performance improves as we add more videos to our offline training set. To the best of our knowledge, our tracker is the first neural-network tracker that learns to track generic objects at 100 fps.

• The paper introduces GOTURN, a deep regression network trained offline to track objects in real time at 100 FPS.
• GOTURN’s simple feed-forward architecture eliminates costly online training while delivering state-of-the-art accuracy.
• Benchmarks like VOT 2014 confirm its robust performance, highlighting its potential for real-world applications such as autonomous driving.

Learning to Track at 100 FPS with Deep Regression Networks

The paper "Learning to Track at 100 FPS with Deep Regression Networks" authored by David Held, Sebastian Thrun, and Silvio Savarese from Stanford University introduces a novel approach to generic object tracking. The paper presents GOTURN (Generic Object Tracking Using Regression Networks), an offline-trained neural network capable of real-time object tracking at 100 frames per second (fps).

Overview

Traditional tracking methods often require online training from scratch, which significantly limits their performance as they cannot utilize large datasets collected offline. The prior attempts to use neural networks in this domain have been constrained due to their slow runtime, rendering them impractical for real-time applications requiring high-speed performance. GOTURN addresses these issues by leveraging offline training, thereby bypassing the need for resource-intensive online training during the test phase and achieving high-speed tracking capabilities.

Key Contributions

1. Offline Training: GOTURN diverges from traditional methods by using a neural network trained entirely offline. This allows the network to learn generic relationships between object motion and appearance changes from large offline datasets, improving tracking robustness and accuracy for novel objects during the test phase without online fine-tuning.
2. Speed: GOTURN achieves unprecedented tracking speeds of 100 fps, significantly outperforming previous neural-network-based trackers that operate at 0.8 fps to 15 fps. This enhancement is crucial for real-time applications such as autonomous driving and robotics.
3. Simplicity and Performance: The tracker employs a simple feed-forward network, making a direct regression to the object's bounding box location in subsequent frames. It outperforms state-of-the-art trackers on standard benchmarks, evident from its competitive results in the VOT 2014 Tracking Challenge.

Methodology

The network architecture integrates a two-stream model where one stream processes the image of the target object from the previous frame, and the other stream processes the current frame's search region. The network, pre-trained on ImageNet, uses convolutional layers to extract features and fully connected layers to learn spatial transformations and regress directly to the object's bounding box.

Training Data: The training set comprises video sequences from ALOV300++ and still images with labeled bounding boxes from ImageNet. Importantly, the network is trained with random cropping that emulates realistic motion, utilizing a Laplace distribution to prefer small motions over larger, abrupt changes, enhancing the network's stability.

Numerical Results

The GOTURN tracker demonstrates substantial improvements:

• Accuracy and Robustness: Results on the VOT 2014 dataset show that GOTURN ranks highest in overall performance (average of accuracy and robustness ranks). Specifically, it maintains robust performance even without online adaptation, outperforming all other trackers in the challenge.
• Speed: GOTURN operates at real-time speeds (100 fps) on a GTX 680 GPU and even faster (165 fps) on a Titan X GPU. This speed advantage is critical for applications demanding quick and reliable object tracking.

Implications and Future Research

Practical Implications: GOTURN’s ability to track at high speeds without compromising accuracy is invaluable for applications in autonomous systems and robotics. For instance, in autonomous vehicles, real-time tracking of dynamic obstacles is essential for navigation and safety.

Future Research Directions:

• Scalability: Given the substantial performance improvements with the current dataset, further research could involve evaluating the scalability of GOTURN with even larger and more diverse datasets.
• Adaptability: Exploring ways to incorporate limited online adaptability could further enhance robustness without significantly impacting the runtime speed.
• Application-Specific Enhancements: Tailoring the network to specific environments or object classes (e.g., pedestrian tracking in urban environments) can potentially optimize performance for targeted applications.

In conclusion, this paper effectively challenges the status quo of generic object tracking by delivering a high-performance, real-time tracker that leverages the strengths of deep learning and offline training. GOTURN's contribution to both theoretical understanding and practical applications in the field of computer vision is significant, laying groundwork for future advancements in high-speed object tracking.