OpenHAIV: Open-World Learning Framework

• OpenHAIV is a unified open-world learning framework that integrates OOD detection, new class discovery, and incremental continual fine-tuning for autonomous knowledge acquisition.
• It employs a modular architecture combining plug-and-play detectors, unsupervised clustering, and continual fine-tuning to adapt dynamically to mixed and unlabeled data streams.
• Empirical evaluations on the OpenEarthSensing benchmark highlight its effectiveness in managing unknown class discoveries and mitigating catastrophic forgetting.

Searching arXiv for OpenHAIV and related frameworks mentioned in the source material. OpenHAIV is a framework for practical open-world learning that integrates out-of-distribution (OOD) detection, new class discovery (NCD), and incremental continual fine-tuning (ICF) into a unified pipeline for autonomous knowledge acquisition and updating in open-world environments. It is designed for settings in which a model operates over time under uncertainty in both data composition and task sequence, and it is validated on the OpenEarthSensing benchmark, a large remote sensing testbed with scene-level and object-level recognition across diverse scales and modalities (Xiang et al., 10 Aug 2025).

1. Problem setting and formalization

Open-world learning in OpenHAIV is defined against two adjacent but insufficient paradigms. Conventional class-incremental learning (CIL) assumes a predetermined number of phases and labeled class partitions, while open-set recognition (OSR) focuses on rejecting samples outside the known set at test time. OpenHAIV adopts a more realistic assumption set: the number of phases is not predetermined, the number of unknown classes is unknown at arrival, and each new phase may contain a mixture of previously known classes and truly unknown classes. In this formulation, open-world learning requires not only rejection of unknowns, but also their acquisition, organization, and integration into the model’s evolving label space (Xiang et al., 10 Aug 2025).

The framework formalizes a mixed unlabeled session $i$ as

$$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$

where $\mathcal{K}_i \subseteq C_{\text{known}}$ are samples of known classes that may reappear, and $\mathcal{U}_i \subseteq U$ are samples from previously unknown classes. For comparison, the CIL reference formulation uses disjoint class subsets

$$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$

with labeled session data

$$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$

This distinction is central: OpenHAIV is intended for mixed, unlabeled arrivals rather than closed-world labeled increments.

The OOD subproblem is treated generically. Given a trained model $M$, let $P_{\text{ID}}(x)$ denote the training in-distribution and $P_{\text{test}}(x)$ the test distribution. An input is OOD if $P_{\text{test}}(x)$ significantly deviates from $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$0. OpenHAIV defines a method-agnostic decision rule in which an inference function $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$1 outputs logits $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$2 or features $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$3, an OOD score $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$4 is computed post-hoc or during training, and a calibrated threshold $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$5 yields

$$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$6

where $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$7 indicates OOD, with the convention reversible depending on the score. This suggests that the framework is organized around interoperable score-and-threshold interfaces rather than a single detector-specific semantics.

2. Modular system architecture

OpenHAIV comprises four core capabilities organized into modular system components. The OWL module performs end-to-end orchestration of autonomous knowledge evolution. The OOD module detects unknowns in the incoming stream with plug-and-play detectors. The NCD module discovers novel categories via unsupervised clustering over learned representations. The CIL/FSCIL modules provide continual fine-tuning engines that integrate newly discovered classes using class-incremental learning or few-shot CIL methods, including LwF, EWC, iCaRL, replay-based, and distillation-based strategies (Xiang et al., 10 Aug 2025).

Supporting infrastructure includes data processing, model construction, a trainer and evaluator, and extensibility mechanisms. Data processing covers loading, phase partitioning, and augmentation, specifically stitching, negative sample construction, rotation, cropping, and format conversion. Model construction includes backbone and heads, losses, and algorithmic hooks for OOD, NCD, and ICF. The trainer and evaluator supply configurable optimizers and schedulers together with standardized evaluation protocols on OpenEarthSensing and natural image datasets. Each module exposes well-defined interfaces, and a hook-based mechanism enables custom operations injection during training.

The end-to-end pipeline begins with supervised training on base classes. For each phase $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$8, the framework receives an unlabeled batch $$D_i = \{ x_j \mid x_j \in (\mathcal{K}_i \cup \mathcal{U}_i) \}_{j=1}^N,$$9, computes $\mathcal{K}_i \subseteq C_{\text{known}}$0 using a chosen OOD detector, and partitions the batch into $\mathcal{K}_i \subseteq C_{\text{known}}$1 and $\mathcal{K}_i \subseteq C_{\text{known}}$2 through threshold $\mathcal{K}_i \subseteq C_{\text{known}}$3. Features

$\mathcal{K}_i \subseteq C_{\text{known}}$4

are extracted from the OOD subset using the shared feature extractor $\mathcal{K}_i \subseteq C_{\text{known}}$5, clustered to produce pseudo-labels $\mathcal{K}_i \subseteq C_{\text{known}}$6, and pruned by cluster quality. The classifier $\mathcal{K}_i \subseteq C_{\text{known}}$7 is then expanded with heads for newly discovered classes, and the model is trained with a composite objective balancing classification on pseudo-labels, distillation to prevent forgetting, and optional regularization from OOD and NCD filters. Replay memory can be updated under a budget constraint. Evaluation reports accuracy on all encountered classes so far and OOD metrics per setup.

This architecture encodes a specific claim about practical open-world learning: rejection, discovery, and assimilation are treated as sequentially coupled modules rather than isolated tasks. A plausible implication is that system performance depends not only on the quality of each component in isolation, but also on thresholding and error propagation across interfaces.

3. OOD detection subsystem

OpenHAIV integrates more than 10 OOD detection algorithms and standardizes their evaluation on OpenEarthSensing across five OOD settings: Standard, Resampling bias, Aerial modality shift, MSRGB modality shift, and IR modality shift (Xiang et al., 10 Aug 2025). The framework includes classic post-hoc detectors and training-time regularization methods. Representative scoring rules are explicitly specified.

For maximum softmax probability (MSP), if logits are $\mathcal{K}_i \subseteq C_{\text{known}}$8 and

$\mathcal{K}_i \subseteq C_{\text{known}}$9

the ID-confidence score is

$\mathcal{U}_i \subseteq U$0

with decision $\mathcal{U}_i \subseteq U$1. For energy-based OOD,

$\mathcal{U}_i \subseteq U$2

and a common decision is $\mathcal{U}_i \subseteq U$3, where high energy indicates OOD. ODIN is described through temperature scaling and a small perturbation $\mathcal{U}_i \subseteq U$4 along the gradient of the loss, using an MSP-like score with tuned $\mathcal{U}_i \subseteq U$5 and $\mathcal{U}_i \subseteq U$6. For Mahalanobis distance scoring, with class-conditional Gaussian features $\mathcal{U}_i \subseteq U$7 and estimates $\mathcal{U}_i \subseteq U$8,

$\mathcal{U}_i \subseteq U$9

and

$$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$0

OpenHAIV also provides canonical implementations and calibration procedures for ViM, MLS, VOS, LogitNorm, and FBDB.

Calibration is treated as part of the detector interface. Threshold $$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$1 or energy threshold $$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$2 can be set on a held-out ID validation split or through a desired operating point such as target FPR@95. Temperature $$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$3 is tuned to maximize AUROC or minimize FPR@95 on validation. Metrics include AUROC,

$$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$4

AUPR, and FPR@95. The framework emphasizes the downstream role of the OOD gate: thresholding trades recall of unknowns against precision of forwarded samples, and the exposed $$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$5 sweeps allow end-to-end sensitivity analysis.

A common misconception is that strong OOD detection is sufficient for open-world learning. OpenHAIV explicitly rejects that view: OOD detection separates known from unknown, but does not by itself update the model’s knowledge. In the framework, its function is to gate the discovery stage while minimizing contamination of NCD by in-distribution samples.

4. New class discovery

OpenHAIV’s NCD engine identifies novel categories within OOD samples by unsupervised clustering on backbone features. By default, it extracts a feature set $$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$6 from the OOD subset $$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$7 and supports K-Means together with density-based variants such as DBSCAN and HDBSCAN via a modular API (Xiang et al., 10 Aug 2025). The canonical K-Means objective is

$$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$8

where $$C = \{C_1, C_2, \ldots, C_T\}, \qquad C_i \cap C_j = \varnothing \ \text{for}\ i \neq j,$$9 are cluster centroids and $$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$0 assigns sample $$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$1 to a cluster.

The number of clusters can be estimated heuristically, for example with elbow or silhouette methods, or set by the user. In open-world practice, the framework allows oversegmentation followed by pruning according to cluster quality. Pseudo-labels $$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$2 are derived from cluster memberships, and cluster purity can be assessed using intra-cluster variance

$$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$3

together with simple size filters such as discarding too-small clusters. Confidence scores may optionally combine OOD scores with local neighborhood consistency, and low-quality clusters can be deferred for re-discovery in later phases.

Representation refinement is optional rather than integral. The framework keeps discovery unsupervised and lightweight, but allows self-supervised refinement through configurable plugins. One provided choice is a contrastive loss over pseudo-labels,

$$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$4

where $$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$5 shares the pseudo-label of $$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$6, $$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$7 is cosine similarity, and $$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$8 is a temperature.

The emphasis on pruning and deferred re-discovery reflects a practical constraint of open-world settings: NCD errors are not isolated. They directly shape the pseudo-labels consumed by continual fine-tuning, so cluster quality functions as a control point for noise propagation.

5. Incremental continual fine-tuning

After NCD, OpenHAIV integrates discovered classes with continual learning strategies using a shared feature extractor $$X_t = \{(x_1,y_1), (x_2,y_2), \ldots, (x_N,y_N)\}, \qquad y_n \in C_t.$$9 and a classifier $M$0 whose heads expand as classes grow. The aggregate training objective is

$M$1

where $M$2 is the supervised or pseudo-supervised classification loss on current session data, including pseudo-labels for discovered classes (Xiang et al., 10 Aug 2025).

Knowledge distillation follows the LwF style. With teacher logits $M$3 and student logits $M$4,

$M$5

where $M$6 denotes softmax at temperature $M$7. Elastic Weight Consolidation is specified as

$M$8

where $M$9 are current parameters, $P_{\text{ID}}(x)$0 are old parameters, and $P_{\text{ID}}(x)$1 are Fisher information estimates. Optional $P_{\text{ID}}(x)$2 and $P_{\text{ID}}(x)$3 regularizers penalize contradictions between OOD signals and NCD assignments, such as suppressing learning from low-quality clusters.

Anti-forgetting strategies include distillation, regularization methods such as EWC, SI, and MAS, and replay-based methods with iCaRL-style exemplars. Exemplar selection via herding minimizes the distance between the true class mean and the selected subset mean:

$P_{\text{ID}}(x)$4

The framework supports a fixed memory budget $P_{\text{ID}}(x)$5 with per-class quotas and balanced sampling. Update schedules align with the end of each phase, though online mini-updates can be enabled for streaming conditions.

OpenHAIV also exposes hooks for pseudo-label noise filtering and class imbalance handling. Noise can be controlled through cluster quality and confidence thresholds, while imbalance can be addressed through reweighting, balanced sampling, or adapter losses; long-tailed strategies such as logit adjustment and effective number weighting can be integrated through the hook system. This suggests that ICF is conceived less as a single algorithm than as a structured compatibility layer between discovery outputs and continual-learning mechanisms.

6. Benchmarks, metrics, and empirical findings

The primary benchmark is OpenEarthSensing (OES), which contains 157,674 images, 10 coarse categories, and 189 fine-grained subcategories, and supports scene-level and object-level recognition under diverse scales and modalities (Xiang et al., 10 Aug 2025). Open-world simulation on OES uses a base session with 94 classes and a subsequent unlabeled session containing the remaining 95 classes as a mixture of known and unknown samples, aiming to learn all 189 classes. The CIL setup uses two orders—random, with 10 sessions times 20 classes, and coarse-to-fine—with a ResNet-18 backbone and eight representative methods: Finetune, EWC, iCaRL, LwF, WA, BiC, GEM, and SSRE. The OOD setup trains on 94 in-distribution classes and evaluates five OOD settings for both unimodal CNN and VLM-based detectors. The FSCIL setup uses a base of 109 classes and 10-way $P_{\text{ID}}(x)$6-shot increments with $P_{\text{ID}}(x)$7 using ResNet-18 and the methods Alice, FACT, and SAVC.

OOD metrics are AUROC, FPR@95, and AUPR when applicable; continual-learning metrics are top-1 accuracy per session, average accuracy, and last-session accuracy. The framework also supports open-world metrics such as OSCR via extensions.

Setting	Method combination or method	Reported result
OWL integration	MSP + iCaRL	ID Acc = 91.17; OOD Acc = 55.01; Session 1 = 91.27; Session 2 = 50.29; Avg = 70.78
OWL integration	MLS + iCaRL	ID Acc = 90.60; OOD Acc = 63.85; Session 2 = 51.33; Avg = 71.30
OWL integration	VIM + LwF	ID Acc = 93.50; OOD Acc = 59.99; Session 2 = 43.49; Avg = 67.38
Unimodal OOD, Standard Far-OOD AUROC	VIM / FBDB / MSP	98.75 / 98.17 / 93.91
Unimodal OOD, Aerial Near-OOD AUROC	MSP / VIM	54.38 / 50.69
Unimodal OOD, MSRGB Far-OOD	MDS / VIM	85.58 / 81.75
VLM-based OOD	DPM	Standard Far-OOD AUROC = 99.24; MSRGB Far-OOD = 93.56; IR Far-OOD = 75.10
FSCIL	SAVC	50-shot: Last = 71.71, Avg = 79.55; 10-shot: Last = 72.23, Avg = 80.07; 5-shot: Last = 66.61, Avg = 75.17; 1-shot: Last = 59.63, Avg = 76.77

Several empirical patterns are emphasized. Base-session performance is strong, but phase-2 performance drops significantly in OWL integration experiments because of noise in discovered classes, which underscores the importance of robust OOD filtering and NCD quality control. Under covariate shift, OOD performance decreases substantially; for example, Aerial Near-OOD AUROC is low for both MSP and VIM. Traditional CIL methods exhibit severe catastrophic forgetting on OES, and replay-based and architecture-expanding methods that perform well on standard benchmarks are challenged by OES’s varied scales, resolutions, and distributions. In FSCIL, SAVC achieves the highest average accuracy across all reported shot settings, whereas FACT trails notably in the 5-shot setting with Avg = 49.15.

The framework also supports ablations over threshold $P_{\text{ID}}(x)$8, memory budget $P_{\text{ID}}(x)$9, and the $P_{\text{test}}(x)$0 weights in the composite objective. Empirical observations indicate that stricter OOD thresholds reduce contamination but may under-recall unknowns, which in turn affects NCD recall and downstream ICF benefits.

7. Relation to prior work, reproducibility, and open problems

OpenHAIV is positioned relative to several adjacent tool ecosystems and baseline families. On the OOD side, it includes MSP, ODIN, Mahalanobis, Energy-based, VIM, VOS, LogitNorm, FBDB, and VLM-based methods such as CoOp, LoCoOp, SCT, and DPM. Benchmarks like OpenOOD and Dassl are described as providing isolated OOD evaluation capability. On the continual-learning side, PyCIL and PILOT consolidate many CIL and FSCIL methods, but do not integrate OOD and NCD into an open-world loop. NCD baselines are characterized as conventional K-Means-based pipelines and generalized or incremental category discovery methods that primarily assume cleaner conditions or offline discovery (Xiang et al., 10 Aug 2025).

What distinguishes OpenHAIV, within this description, is the explicit coupling of OOD detection, NCD, and ICF under realistic assumptions of unknown phase count and mixed-phase arrivals. Its practicality is tied to modular interfaces, standardized end-to-end evaluation on OES, and plug-and-play extensibility for detectors, clustering routines, and continual learners. The implementation is modular in PyTorch, NumPy, Matplotlib, and Pandas, with hook-based extensibility for methods, networks, and datasets, and code and documentation are provided at https://haiv-lab.github.io/openhaiv.

The reported experimental defaults are concrete. For unimodal OOD, ResNet-50 is trained with cross-entropy on ID for 100 epochs with learning rate 0.05, and training-required OOD methods are fine-tuned for 30 epochs with learning rate 0.01. For VLM-based OOD, GeoRSCLIP with ViT-B/32 is fine-tuned for 20 epochs at learning rate 0.01. For CIL and FSCIL, a ResNet-18 backbone is used, with method-specific defaults such as LwF distillation temperature and EWC $P_{\text{test}}(x)$1 specified in configuration files. A typical workflow is to configure dataset and phase definitions, select and calibrate an OOD detector, choose an NCD strategy and quality filters, select an ICF method and define $P_{\text{test}}(x)$2, $P_{\text{test}}(x)$3, $P_{\text{test}}(x)$4, and $P_{\text{test}}(x)$5, optionally enable replay memory with budget $P_{\text{test}}(x)$6, and then train and evaluate per phase.

The principal limitations are also explicit. OpenHAIV observes significant performance drops in later OWL sessions caused by noisy pseudo-labels and contamination from imperfect OOD filtering. Conventional CIL strategies continue to exhibit catastrophic forgetting on OES. NCD faces cluster purity and cluster-count estimation challenges, particularly under domain and modality shifts, and scalability to many fine-grained classes with sparse samples per class may require stronger priors or multimodal cues. The stated future directions are robust NCD with improved quality control, noise-robust continual training, domain-aware OOD/NCD/ICF modules for shifts such as Aerial, MSRGB, and IR, and memory and replay optimization through dynamic budgets and diversity-aware exemplar selection. These directions indicate that OpenHAIV should be understood not as a solved formulation of open-world learning, but as a framework for studying the interactions among recognition, discovery, and continual adaptation under realistic uncertainty.