Incremental Learning of Object Detectors without Catastrophic Forgetting (1708.06977v1)

Abstract: Despite their success for object detection, convolutional neural networks are ill-equipped for incremental learning, i.e., adapting the original model trained on a set of classes to additionally detect objects of new classes, in the absence of the initial training data. They suffer from "catastrophic forgetting" - an abrupt degradation of performance on the original set of classes, when the training objective is adapted to the new classes. We present a method to address this issue, and learn object detectors incrementally, when neither the original training data nor annotations for the original classes in the new training set are available. The core of our proposed solution is a loss function to balance the interplay between predictions on the new classes and a new distillation loss which minimizes the discrepancy between responses for old classes from the original and the updated networks. This incremental learning can be performed multiple times, for a new set of classes in each step, with a moderate drop in performance compared to the baseline network trained on the ensemble of data. We present object detection results on the PASCAL VOC 2007 and COCO datasets, along with a detailed empirical analysis of the approach.

• The paper presents a dual-network framework integrating cross-entropy and distillation losses to mitigate catastrophic forgetting.
• It employs a frozen original detector alongside an adaptable network to incrementally add new classes without prior training data.
• Experimental results on PASCAL VOC and COCO show minimal performance loss with sequential class additions, validating the approach.

Incremental Learning of Object Detectors without Catastrophic Forgetting

This paper addresses a significant challenge in the domain of convolutional neural networks (CNNs), specifically focusing on the problem of catastrophic forgetting during incremental learning of object detectors. The authors propose an innovative approach to enable a neural network to learn to detect new object classes while retaining the performance of previously learned classes, all without access to the initial training data.

Core Contributions

The paper's primary contribution is a methodological framework that facilitates incremental learning for object detection tasks. This framework is centered around a novel loss function that combines two critical components: a standard cross-entropy loss for new classes and a distillation loss. The distillation loss plays a pivotal role as it minimizes the discrepancy between the predictions for old classes from the original and updated networks. This strategic incorporation effectively mitigates the risk of catastrophic forgetting.

Methodology

The authors utilize a dual-network strategy in their framework.

• Network A: A frozen copy of the original detector, used to compute the distillation loss and select non-background proposals.
• Network B: An adaption of Network A, extended to detect new classes by adjusting the number of outputs in the network's final layer.

The elegance of this approach lies in its ability to perform multiple incremental learning iterations, consistently maintaining moderate performance drops.

Experimental Analysis

The authors provide a thorough empirical evaluation using benchmark datasets such as PASCAL VOC 2007 and COCO. The results demonstrate that the proposed method can incrementally add new classes and achieve performance close to that of a baseline model trained on all classes simultaneously. Notable outcomes include:

• The framework effectively retained an average precision (mAP) comparable to jointly trained models, with only marginal reductions for newly added classes.
• The distillation-focused methodology outperformed alternative solutions, such as those relying solely on frozen layers or cross-entropy variants.
• Sequential addition of classes was shown to be feasible with controlled degradation in performance.

Implications and Future Directions

The research presents valuable implications for developing scalable AI systems capable of learning continuously evolving datasets. The authors successfully address catastrophic forgetting, a longstanding issue in neural network architectures, thereby paving the way for more robust incremental learning paradigms.

Future research might expand upon this work by adapting the method for use with RPN-based proposals in Faster R-CNN architectures, requiring further refinement of the distillation process.

Conclusion

This paper makes a vital contribution to the field of computer vision by presenting a feasible solution to incremental learning challenges in CNN-based object detectors. The proposed strategy offers a balance of innovation and practicality, reinforcing its applicability in dynamic learning environments where data evolves over time. By ensuring networks can integrate new knowledge without sacrificing existing capabilities, this work holds promise for advancing the adaptability of artificial intelligence systems.