Recent Advances in Open Set Recognition: A Survey (1811.08581v4)

Abstract: In real-world recognition/classification tasks, limited by various objective factors, it is usually difficult to collect training samples to exhaust all classes when training a recognizer or classifier. A more realistic scenario is open set recognition (OSR), where incomplete knowledge of the world exists at training time, and unknown classes can be submitted to an algorithm during testing, requiring the classifiers to not only accurately classify the seen classes, but also effectively deal with the unseen ones. This paper provides a comprehensive survey of existing open set recognition techniques covering various aspects ranging from related definitions, representations of models, datasets, evaluation criteria, and algorithm comparisons. Furthermore, we briefly analyze the relationships between OSR and its related tasks including zero-shot, one-shot (few-shot) recognition/learning techniques, classification with reject option, and so forth. Additionally, we also overview the open world recognition which can be seen as a natural extension of OSR. Importantly, we highlight the limitations of existing approaches and point out some promising subsequent research directions in this field.

• The paper presents a comprehensive review of OSR techniques that enable classifiers to distinguish between seen and unseen classes using methods like Extreme Value Theory.
• It evaluates traditional machine learning, deep neural networks, and generative models, providing empirical insights with performance metrics such as F-measure and AUROC.
• The survey outlines promising research directions including hybrid models and semantic integration to enhance the detection and management of unknown classes.

Recent Advances in Open Set Recognition: A Survey

The paper "Recent Advances in Open Set Recognition: A Survey" by Geng et al. presents a comprehensive review of methodologies designed to tackle the challenges presented by Open Set Recognition (OSR). Open Set Recognition addresses the realistic scenario where classifiers encounter classes during testing that were not present during training. With its focus on enabling recognition systems to handle both seen and unseen classes, this paper serves to untangle various existing approaches while also delineating potential future directions.

Key Contributions and Methodologies

The paper is organized into several major sections beginning with foundational concepts and definitions relevant to OSR. A significant portion is devoted to outlining existing OSR techniques categorized into traditional machine learning models and deep learning methodologies.

1. Traditional ML-Based Methods: These methods often modify existing schemes, such as Support Vector Machines and Nearest Neighbor classifiers, to incorporate policies for rejecting unknown classes. A notable approach is the use of Extreme Value Theory (EVT) for tail distribution modeling to enhance rejection performance in real-world applications. Models like PI-SVM and W-SVM utilize EVT to differentiate between known known classes (KKCs) and unknown unknown classes (UUCs).
2. Deep Neural Network Approaches: Innovations in this category typically center around adapting neural networks that inherently operate under a closed set assumption. Methods like OpenMax modify the output layers of neural networks to encapsulate open space considerations, allowing for the identification and rejection of UUCs.
3. Generative Models: This section details instance generation-based methods, focusing on the application of adversarial learning techniques to create synthetic examples of UUCs. These methods attempt to reveal the structure of open space by generating novel instances that aid in classifier training.
4. Non-Instance Generation Methods: Approaches like the collective decision-based model utilize Dirichlet processes to adaptively handle UUCs by modeling the joint distribution of unknowns across testing sets.

Numerical Results and Claims

The survey critically evaluates the effectiveness of each discussed method via empirical evaluations on benchmark datasets, assessing their performance in terms of metrics such as the F-measure and AUROC. The results underscore the importance of threshold selection and proper model calibration, particularly highlighting the utility of data adaptation techniques such as Bayesian nonparametrics.

Implications and Future Directions

A key insight from the paper is the realization that OSR is not merely an extension of closed set recognition but rather a paradigm shift that involves a fundamentally different problem space. The authors propose several promising avenues for future research:

• The exploration of hybrid models that combine the strengths of both generative and discriminative paradigms.
• The integration of semantic/attribute information to enrich the representational capacity of models and enhance UUC handling.
• Investigating more robust decision-making processes that account for batch decision strategies and correlate unknown samples for possible class discovery.

Additionally, the paper suggests further investigation into the concept of Generalized Open Set Recognition, where models can utilize side-information beyond feature-level data. This broader approach could potentially integrate universum classes, thereby strengthening OSR strategies.

Conclusion

"Recent Advances in Open Set Recognition: A Survey" by Geng et al. offers a valuable framework for understanding the multifaceted challenges of OSR along with the state-of-the-art solutions. By paving the way forward with concretely suggested research directions, the paper contributes a vital resource to the ongoing development of adaptive, resilient recognition systems crucial for real-world applications. As the landscape of AI continues to evolve, addressing the open set recognition challenge remains a critical endeavor.