Weak Novel Categories without Tears: A Survey on Weak-Shot Learning (2110.02651v3)

Abstract: Deep learning is a data-hungry approach, which requires massive training data. However, it is time-consuming and labor-intensive to collect abundant fully-annotated training data for all categories. Assuming the existence of base categories with adequate fully-annotated training samples, different paradigms requiring fewer training samples or weaker annotations for novel categories have attracted growing research interest. Among them, zero-shot (resp., few-shot) learning explores using zero (resp., a few) training samples for novel categories, which lowers the quantity requirement for novel categories. Instead, weak-shot learning lowers the quality requirement for novel categories. Specifically, sufficient training samples are collected for novel categories but they only have weak annotations. In different tasks, weak annotations are presented in different forms (e.g., noisy labels for image classification, image labels for object detection, bounding boxes for segmentation), similar to the definitions in weakly supervised learning. Therefore, weak-shot learning can also be treated as weakly supervised learning with auxiliary fully supervised categories. In this paper, we discuss the existing weak-shot learning methodologies in different tasks and summarize the codes at https://github.com/bcmi/Awesome-Weak-Shot-Learning.

• The paper distinguishes weak-shot learning from zero-shot and few-shot methods by emphasizing reduced annotation quality rather than quantity.
• It categorizes methodologies into transferring category-invariant targets, mapping weak to full annotations, and decomposing tasks to enhance novel category learning.
• The survey demonstrates practical applications in image classification, object detection, and segmentation, highlighting future research directions to overcome annotation challenges.

Insightful Overview of "Weak Novel Categories without Tears: A Survey on Weak-Shot Learning"

The paper "Weak Novel Categories without Tears: A Survey on Weak-Shot Learning" addresses the inherent challenges posed by deep learning's dependence on vast quantities of annotated data. It posits weak-shot learning as a paradigm that reduces the demand for fully-annotated data, opting instead for weaker annotations for novel categories. This approach becomes especially relevant in tasks where acquiring large, annotated datasets is impractical due to the emergence of numerous categories, some of which might be fine-grained or entirely new.

Key Contributions

The paper effectively delineates weak-shot learning from the more traditional zero-shot and few-shot learning frameworks. While zero-shot learning utilizes zero samples and relies on semantic representations, and few-shot learning utilizes a limited number of samples per category, weak-shot learning assumes a different stance by significantly reducing the quality of annotations rather than their quantity. This results in weakly-annotated data, such as noisy labels in image classification or image labels rather than bounding boxes in object detection, which must be compensated through knowledge transfer from categories with full annotations.

Methodologies

The authors categorize existing weak-shot learning methodologies into three distinct strategies:

1. Transfer of Category-Invariant Targets: This involves leveraging targets that are invariant across categories, such as similarity in classification tasks or objectness in detection tasks, and applying them to novel categories. By learning these targets on base categories, they can be extended to novel categories to aid in tasks like feature denoising or proposal generation.
2. Transfer of the Mapping from Weak Annotations to Full Annotations: This approach harnesses the potential of base categories where both weak and full annotations are available, allowing the learning of a mapping that can be transferred to novel categories. Such mappings might involve weight transformations or refined predictions from weak annotations.
3. Task Decomposition: This strategy involves decomposing a task into sub-tasks that can be addressed with weakly-supervised and fully-supervised learning in parallel. Semantic similarities learned from base categories are adapted for use in novel categories, improving performance in the absence of strong annotations.

Practical Applications

The survey extends its coverage to various applications within weak-shot learning, specifically focusing on image classification, object detection, and segmentation tasks—both semantic and instance. Each application taps into the methodologies discussed, demonstrating both the versatility and the challenges of the weak-shot paradigm. For example, in weak-shot object detection, the survey highlights how methods like knowledge transfer of objectness and iterative refinement are crucial to bridging the gap left by weak annotations.

Implications and Future Directions

Weak-shot learning stands out by addressing the practical bottleneck of acquiring extensive training datasets. This paradigm shift can significantly impact fields sensitive to data annotation constraints, such as ecology or medical imaging. The paper suggests several potential avenues for future research, including enhancing knowledge transfer techniques and developing more robust models to handle the inherent uncertainties introduced by weak annotations. The emphasis on cross-category knowledge transfer presents both a challenge and an opportunity for further theoretical advancement in the discipline.

Overall, the survey provides a comprehensive examination of current weak-shot learning approaches while encouraging further exploration into this emergent topic within the machine learning landscape.