An End-to-End Framework For Universal Lesion Detection With Missing Annotations

Abstract: Fully annotated large-scale medical image datasets are highly valuable. However, because labeling medical images is tedious and requires specialized knowledge, the large-scale datasets available often have missing annotation issues. For instance, DeepLesion, a large-scale CT image dataset with labels for various kinds of lesions, is reported to have a missing annotation rate of 50\%. Directly training a lesion detector on it would suffer from false negative supervision caused by unannotated lesions. To address this issue, previous works have used sophisticated multi-stage strategies to switch between lesion mining and detector training. In this work, we present a novel end-to-end framework for mining unlabeled lesions while simultaneously training the detector. Our framework follows the teacher-student paradigm. In each iteration, the teacher model infers the input data and creates a set of predictions. High-confidence predictions are combined with partially-labeled ground truth for training the student model. On the DeepLesion dataset, using the original partially labeled training set, our model can outperform all other more complicated methods and surpass the previous best method by 2.3\% on average sensitivity and 2.7\% on average precision, achieving state-of-the-art universal lesion detection results.