Coarse-to-Fine Open-set Classification

• CFC is a framework that decomposes open-set recognition into a coarse stage for OOD detection and a fine stage for detailed classification.
• It leverages metric learning, LLM-based semantic labeling, and data augmentation to differentiate between known and unknown classes effectively.
• Empirical studies across vision, graph, and text domains show significant gains in AUROC and classification accuracy, validating its practical impact.

Coarse-to-Fine Open-set Classification (CFC) refers to a family of algorithmic frameworks that address open-set recognition by explicitly decomposing the recognition workflow into hierarchically organized modules: a coarse stage for unknown (OOD) detection and label suggestion, and a fine stage for in-distribution (ID) classification and refined OOD subclass assignment. Recent work establishes both the conceptual generality and empirical superiority of CFC across domains including vision and graph-based structured data, with variants leveraging metric learning, LLMs, and data augmentation for robust performance in high-stakes open-world deployments (Brignac et al., 2024, Ma et al., 18 Dec 2025).

1. Formal Problem Definition

Open-set classification extends closed-set prediction by requiring a classifier $f$ not only to assign samples to known classes $\{1,\dots,C\}$, but also to detect out-of-distribution (OOD) samples and allocate them to one or more unknown labels. In the canonical graph node classification setting, the training set $T = \{(x_i, y_i)\}_{i=1}^n$ is restricted to $y_i \in \{1,\ldots,C\}$, but test samples may belong to unknown classes not seen at train time. CFC formalizes two subproblems:

• OOD detection: learn $f_{C+1}: \mathbb{R}^d \to \{1,\ldots,C,C+1\}$ (where $C+1$ is the generic OOD flag).
• OOD subclassification: learn $$f_{C+u}: \mathbb{R}^d \to \{1,\ldots,C,C+1,\ldots,C+u\}$$, recovering the true (unknown) OOD subclass label when possible (Ma et al., 18 Dec 2025).

2. Hierarchical CFC Architectures

CFC frameworks instantiate a coarse-to-fine pipeline in which specialized modules perform sequential tasks:

1. Coarse Unknown Detection: Identifies whether samples are in-distribution or OOD using metric learning (e.g., embedding separation with triplet loss (Brignac et al., 2024)) or, in graph/text domains, zero-shot LLM-aided rejection via targeted prompts (Ma et al., 18 Dec 2025).
2. Fine Closed-set Classification: ID-assigned samples undergo standard C-way softmax classification using supervised deep nets (e.g., ResNet-18 (Brignac et al., 2024) or GCNs (Ma et al., 18 Dec 2025)).
3. Refined OOD Subclassification: For samples flagged as OOD, the system aims to assign semantically meaningful subclass labels using LLM-generated candidate labels and post-processed embeddings. Semantic label post-processing includes TF-IDF similarity merging, filtering out rare labels, and final subclass assignment by LLM prompt annotation (Ma et al., 18 Dec 2025).

Parallel networks are often used in vision (embedding and classifier nets trained separately), while graph/text pipelines incorporate LLMs as coarse detectors and label generators, coupled with GNN-based denoising, manifold mixup augmentation, and subsequent GNN retraining across augmented (ID+OOD) pools.

3. Key Mathematical Formulations

CFC architectures employ several central mathematical operators:

• Triplet Loss (Metric Learning): For vision, the coarse detector models embedding separation over known–known (KK) and known–unknown (KU) pairs via

$$\mathcal{L}_u(\theta) = \sum_{i=1}^N [\|g_\theta(x_i^a) - g_\theta(x_i^{KK})\|^2 - \|g_\theta(x_i^a) - g_\theta(x_i^{KU})\|^2 + \beta]_+$$

with margin hyperparameter $\beta$ (Brignac et al., 2024).

• Label Propagation (Graph Denoising):

$$Y^{l(k)} = D^{-1}A Y^{l(k-1)},\quad \text{where}\; D = \text{deg}(A)$$

with ID rows reset at each step (Ma et al., 18 Dec 2025).

• Semantic OOD Manifold Mixup: For graph embeddings, new OOD samples are synthesized by mixing uncertain ID and OOD vectors:

$$\tilde{x}_i = \alpha h_i^k + (1-\alpha) h_c^k,\quad \tilde{y}_i = C+1,\quad \alpha \in (0,1)$$

for improved decision boundary regularization (Ma et al., 18 Dec 2025).

• GNN Forward Pass:

$$Z = \text{softmax}(\hat{A}\text{ReLU}(\hat{A} X W^{(0)}) W^{(1)})$$

$\hat{A}$ denotes normalized adjacency (Ma et al., 18 Dec 2025).

• Threshold/Decision Rule: OOD assignment is performed based on distance or predicted softmax, e.g., flag OOD if $s(x) \leq \tau$ (metric detector) (Brignac et al., 2024), or $\max p_i < \tau$ (probabilistic detector) (Ma et al., 18 Dec 2025).

4. Experimental Protocols and Results

CFC frameworks are validated on image and graph datasets:

• Vision: Datasets include MNIST, SVHN, CIFAR-10, CIFAR+10/50, and Tiny-ImageNet. Metrics consist of AUROC for known/unknown discrimination, and CCR at calibrated TPR levels. Cas-DC achieves AUROC up to $0.9930$ on MNIST and $0.9475$ on CIFAR+50, consistently outperforming prior state-of-the-art methods (Brignac et al., 2024).
• Graph/Text: CFC attains overall accuracy gains of $+10\%$ over the best baselines on Cora, Citeseer, WikiCS, DBLP, and Amazon-Computer datasets. OOD classification accuracy for subclass labeling reaches $70\%$ on Cora and Citeseer (Ma et al., 18 Dec 2025).
• Text Domain: Gains are observed on News Category and Twitter Topic datasets, with CFC achieving OOD detection accuracy up to $82.04\%$ and subclass identification up to $66\%$ per class on News Category (Ma et al., 18 Dec 2025).

Ablation studies demonstrate robustness to LLM choice, the size of LLM-labeled OOD pools, and manifold mixup parameter $\alpha$ (optimal at $\alpha\approx0.5-0.7$). The addition of semantic OOD, label propagation denoising, and mixup augmentation leads to incremental accuracy gains ranging from $3\%$ to $8\%$ for OOD detection.

5. Comparative Analysis and Methodological Distinctions

CFC outperforms existing baselines including Outlier Exposure, generative models (PROSER, G$^2$Pxy), energy-based OOD, and thresholded GCN variants. In the vision domain, Cas-DC demonstrates mean AUROC improvements of $+1{-}20$ points over competitors across datasets. In the graph domain, CFC’s interpretability and semantic labeling capabilities distinguish it from baselines that treat all OOD as a single undifferentiated class (Ma et al., 18 Dec 2025). Notably, CFC does not require synthetic or auxiliary OOD samples; it instead leverages natural semantic instances and LLM-generated candidate labels, enhancing practical utility for deployment-critical scenarios.

6. Limitations and Future Directions

CFC methodologies rely on the underlying knowledge coverage of LLMs (especially in semantic OOD detection and label suggestion). Poor domain alignment or inadequate textual descriptions for graph nodes may degrade detection performance. Computational cost for repeated LLM querying on large test sets is marked as a bottleneck. Prospective directions include fine-tuning smaller domain-specific LLMs, integrating retrieval-augmented generation for label space expansion, extension to graph-level open-set and non-textual features, and exploration of contrastive augmentation methods beyond linear mixing. The interpretability afforded by semantic subclass labels and expanded OOD subspaces is highlighted as a functional advantage, suggesting further research on integrating human-interpretable semantics in open-set classifiers (Ma et al., 18 Dec 2025).

7. Summary Table: Framework Components

Stage	Vision CFC (Cas-DC)	Graph/Text CFC
Coarse Detector	Metric: embedding distance + τ	LLM prompt: Easy/Hard-Reject
Fine Classifier	C-way softmax network	GCN trained with ID/semantic OOD + mixup
OOD Subclassification	Not implemented	LLM post-processing + semantic label assignment

CFC frameworks systematically advance open-set classification by leveraging coarse-to-fine specialization, modular use of LLMs/generative augmentation, and empirically justified denoising strategies. The paradigm achieves state-of-the-art performance in OOD detection and subclassification across modalities, with potential for further methodological innovation and expanded real-world adoption (Brignac et al., 2024, Ma et al., 18 Dec 2025).