Boosting Open Set Recognition Performance through Modulated Representation Learning (2505.18137v1)

Abstract: The open set recognition (OSR) problem aims to identify test samples from novel semantic classes that are not part of the training classes, a task that is crucial in many practical scenarios. However, existing OSR methods use a constant scaling factor (the temperature) to the logits before applying a loss function, which hinders the model from exploring both ends of the spectrum in representation learning -- from instance-level to semantic-level features. In this paper, we address this problem by enabling temperature-modulated representation learning using our novel negative cosine scheduling scheme. Our scheduling lets the model form a coarse decision boundary at the beginning of training by focusing on fewer neighbors, and gradually prioritizes more neighbors to smooth out rough edges. This gradual task switching leads to a richer and more generalizable representation space. While other OSR methods benefit by including regularization or auxiliary negative samples, such as with mix-up, thereby adding a significant computational overhead, our scheme can be folded into any existing OSR method with no overhead. We implement the proposed scheme on top of a number of baselines, using both cross-entropy and contrastive loss functions as well as a few other OSR methods, and find that our scheme boosts both the OSR performance and the closed set performance in most cases, especially on the tougher semantic shift benchmarks.

• The paper introduces NegCosSch, a dynamic temperature scheduling scheme that transitions from instance-specific to semantic feature learning, boosting recognition performance.
• It integrates seamlessly with existing OSR methods without additional computational overhead, improving both closed and open set classification.
• Empirical results on semantic shift benchmarks demonstrate enhanced representation compactness, indicating scalability to larger, evolving datasets.

Enhancing Open Set Recognition via Modulated Representation Learning

In the domain of deep learning, identifying test samples from unseen semantic classes, known as the Open Set Recognition (OSR) problem, poses significant challenges. Unlike closed set scenarios where models are trained and tested on the same classes, OSR requires models to generalize well beyond the training class boundaries. Existing OSR methods often employ a static approach to temperature scaling—adjusting the logits with a constant scaling factor before applying loss functions. This constant approach limits the exploration of representation features from the instance-level to the semantic-level. Addressing this limitation, the paper introduces a novel temperature-modulated learning strategy—negative cosine scheduling (NegCosSch)—that dynamically adjusts the learning temperature throughout training, fostering a more versatile representation space.

The proposed NegCosSch scheduling innovatively addresses the transition from focusing on instance-specific features to semantic structures by allowing a gradual adjustment of the temperature. It starts with a lower temperature to establish a coarse decision landscape, augmenting the instance discrimination capability. Gradually, the temperature rises during training, enabling deeper semantic understanding as more class neighbors are prioritized. This fluid modulation improves representation compactness and separation, beneficial for both closed set classification and OSR performance.

A significant advantage of NegCosSch lies in its seamless integration with existing methods, free from computational overhead—a common limitation when incorporating regularization or auxiliary sample generation in other OSR approaches. The authors demonstrate the efficacy of NegCosSch across multiple baseline setups using cross-entropy and contrastive loss functions, with notable improvements observed particularly in semantic shift benchmarks.

Key contributions from the paper include:

• A novel temperature scheduling scheme, NegCosSch, which can be incorporated in current OSR methods effortlessly, enhancing both open and closed set recognition performance.
• The demonstration of representation enhancement, crucial for distinguishing samples from unknown classes, thus offering potential improvements in OSR benchmarks and in more complex scenarios such as semantic shift benchmarks (SSBs).
• The insights provided on the effectiveness of NegCosSch with increasing training class count suggest potential for scaling and extending applicability to broader datasets.

The implications of this work span both practical and theoretical realms within AI. Practically, this method provides a tool for enhancing safety and reliability in machine learning systems by improving the detection of novel patterns—a crucial aspect in dynamic environments or applications with emerging categories. Theoretically, it opens avenues for exploring temperature modulation effects on feature learning and its impacts on model robustness and generalization.

Future exploration and applications in AI could involve refining temperature dynamics tailored for specific OSR challenges or extending methodologies to address open world recognition problems where categories continuously evolve. Furthermore, studies could assess speculative combinations of NegCosSch with advanced augmentation methods to explore unseen open set identification scenarios, pushing boundaries of current OSR capabilities.