Separate Class Learning (SCL)

• Separate Class Learning is a framework that assigns unique, dedicated representations to each class to enhance inter-class separation and reduce interference.
• It leverages geometric strategies, such as ETF-based classifier design and optimization decoupling methods, to improve incremental and long-tailed learning performance.
• The approach adapts to different contexts by disambiguating similar acronyms and tailoring separation techniques for base, novel, and tail classes.

Searching arXiv for papers using “Separate Class Learning (SCL)” and closely related usages. Separate Class Learning (SCL) is a label used in multiple, only partially overlapping research contexts on arXiv rather than a single standardized formalism. In the literature provided here, the phrase is used most directly for methods that treat classes or class groups separately in representation learning, classifier design, or class-incremental optimization, with the common objective of increasing inter-class separability and reducing interference between classes or class groups. At the same time, several papers use the acronym “SCL” for different concepts—such as “Suppressed Consistency Loss,” “Semantic Contrastive Learning,” “Supervised Contrastive Learning,” “Streaming Continual Learning,” and “Clause Learning from Simple Models”—so the term requires contextual disambiguation (Hyun et al., 2020, Huang et al., 2021, Song et al., 2023, Giannini et al., 2 Mar 2026, Leidinger et al., 2022). Within the subset of works that explicitly connect SCL to separate or maximally separated class representations, the idea centers on assigning each class its own distinct representation, prototype, or classifier direction, often under geometric constraints such as simplex Equiangular Tight Frame (ETF) structure, or on decoupling class groups during learning to reduce cross-class bias (Khandelwal, 2023, Ahn et al., 2020).

1. Terminological scope and ambiguity

The phrase “Separate Class Learning” is not uniformly defined across the cited literature. One strand explicitly uses the term to denote the idea that each class should have its own clearly distinguished representation or classifier, “such that classes are as ‘separate’ as possible in representation space,” and identifies MASIL as an implementation of this idea for Few Shot Class Incremental Learning (FSCIL) (Khandelwal, 2023). Another strand uses closely aligned constructions without always naming them “Separate Class Learning,” for example by separating old and new class groups through a separated softmax or a split network topology in class-incremental learning (Ahn et al., 2020, Kim et al., 2021). A more recent long-tailed OOD detection paper explicitly uses “separate class learning” to describe an approach that “separately conduct head-specific and tail-specific class learning” (Feng et al., 21 Sep 2025).

By contrast, several papers use the acronym “SCL” for unrelated notions. In semi-supervised learning under imbalance, SCL denotes “Suppressed Consistency Loss” (Hyun et al., 2020). In unsupervised deep clustering, SCL denotes “Semantic Contrastive Learning” (Huang et al., 2021). In few-shot class-incremental learning and hierarchical text classification, SCL is used for “supervised contrastive learning” (Song et al., 2023, Lian et al., 2024), and in contrastive learning theory it likewise denotes “Supervised Contrastive Learning” (Jiang et al., 2022). In continual and streaming learning, SCL denotes “Streaming Continual Learning” (Giannini et al., 2 Mar 2026, Cossu et al., 2 Mar 2026). In automated reasoning, SCL denotes “Clause Learning from Simple Models” (Leidinger et al., 2022). Any encyclopedia treatment of Separate Class Learning therefore requires distinguishing the phrase from acronym collisions.

A plausible implication is that “Separate Class Learning” is best understood as a family resemblance term: methods differ in mechanism, but share the goal of preserving or enforcing class-wise structural separation.

2. Core idea: separate class representations

In the most direct formulation, Separate Class Learning is the idea that “every class” should be represented by “a distinct weight vector” aligned with a class-specific feature prototype and arranged so that different classes are maximally separated (Khandelwal, 2023). MASIL states this in explicitly geometric form: its goal is to learn a classifier whose class weight vectors are “maximally separable,” with within-class feature collapse and between-class simplex ETF structure (Khandelwal, 2023). Under this view, a class is not merely one label among many in a shared output space; it is a dedicated direction in embedding or classifier space.

MASIL describes this using the Neural Collapse picture. For class $k$, the collapsed class mean is $\mathbf{h}_k^\*$, and the classifier weight $\mathbf{w}_k^\*$ aligns with it as

$\mathbf{w}_k^\* = \sqrt{\frac{n_k \lambda_H}{\lambda_W}}\, \mathbf{h}_k^\*,$

while the collection of class means forms a simplex ETF (Khandelwal, 2023). In the same paper, maximal separation is described geometrically by unit-norm class weights with constant pairwise inner product,

$$\langle \mathbf{w}_i, \mathbf{w}_j \rangle = c,\ \forall i \neq j,$$

which places the class vectors at the vertices of a regular simplex (Khandelwal, 2023). The resulting interpretation of SCL is explicitly “geometric”: every class occupies its own direction, and all directions are equally separated.

This representation-centric notion of separation also appears in a different form in long-tailed OOD detection. There, separate class learning splits the in-distribution classes into head and tail groups and trains them differently because OOD confusion with head classes and OOD confusion with tail classes are different problems (Feng et al., 21 Sep 2025). The separation is therefore not only between individual classes, but also between class subsets with different statistical roles.

3. Geometric realizations in class-incremental learning

A major realization of Separate Class Learning appears in class-incremental and few-shot class-incremental settings, where catastrophic forgetting and old–new confusion make class separation operationally central.

MASIL and maximal separability

MASIL, proposed for FSCIL, addresses forgetting of old classes and over-fitting to novel classes by learning classifier weights that are “maximally separable between classes forming a simplex Equiangular Tight Frame” (Khandelwal, 2023). Its formulation begins from a deep linear network objective

$$\underset{\mathbf{W}, \mathbf{H}}{\min}\ \mathcal{L}(\mathbf{W}, \mathbf{H}) = \frac{1}{2N}\|\mathbf{W}\mathbf{H} - \mathbf{Y}\|_F^2 + \frac{\lambda_{\mathbf{W}}}{2}\|\mathbf{W}\|_F^2 + \frac{\lambda_{\mathbf{H}}}{2}\|\mathbf{H}\|_F^2,$$

and uses Neural Collapse to motivate simplex-aligned class means and classifier weights (Khandelwal, 2023). MASIL then adds a simplex loss

$$\underset{\mathbf{h}_i}{\min}\ \frac{1}{|\mathcal{D}_j|} \sum_{(x_i, y_i)\in \mathcal{D}_j} \|\mathbf{w}_{y_i}^\top \mathbf{h}_i - 1\|_F^2,$$

to align features with class-specific simplex directions (Khandelwal, 2023).

A distinctive feature is concept factorization. Base-session activations are decomposed with Nonnegative Matrix Factorization,

$$\underset{\mathbf{P} \ge 0,\, \mathbf{Q} \ge 0}{\min}\ \frac{1}{2}\|\mathbf{A} - \mathbf{P}\mathbf{Q}^\top\|_F^2,$$

producing a concept bank $\mathbf{Q}$ used to express novel-class features and initialize novel-class classifier weights (Khandelwal, 2023). The initial weight for a few-shot class is

$$\hat{\mathbf{w}}_{y_i} = \frac{1}{|\mathcal{D}_j|} \left(\sum_{(x_i,y_i)\in \mathcal{D}_j} \mathbf{P}(x_i)\right)\mathbf{Q}^\top.$$

This suggests a concept-basis view of SCL: classes are separated not only by final-layer geometry, but by distinct combinations of reusable semantic factors.

MASIL also stores collapsed base-class means

$\mathbf{h}_k^\*$0

and fine-tunes classifier weights using a three-term objective combining novel-sample fit, preservation of alignment with base class centers, and regularization toward the simplex initializer (Khandelwal, 2023). This preserves class separation without retaining base images.

SCL-PNC and parametric ETF expansion

A later class-incremental formulation, “Scalable Class-Incremental Learning based on Parametric Neural Collapse,” is explicitly named SCL-PNC and likewise treats class-wise separation as a geometric principle (Zhang et al., 26 Dec 2025). It defines a dynamic ETF classifier whose weight matrix $\mathbf{h}_k^\*$1 satisfies

$\mathbf{h}_k^\*$2

and parameterizes the class weights at task $\mathbf{h}_k^\*$3 as

$\mathbf{h}_k^\*$4

for the current number of classes $\mathbf{h}_k^\*$5 (Zhang et al., 26 Dec 2025). Unlike static ETF classifiers, this construction is recomputed as classes are added, which the paper presents as a response to class misalignment under evolving class distributions (Zhang et al., 26 Dec 2025).

SCL-PNC combines this classifier with an expandable backbone, an adapt-layer, and a knowledge distillation loss between expansion modules,

$\mathbf{h}_k^\*$6

together with a point-regression neural-collapse loss (Zhang et al., 26 Dec 2025). The paper frames this as scalable class-incremental learning in which separate task modules remain geometrically aligned under one global classifier.

A plausible implication is that one branch of Separate Class Learning research has converged on ETF-structured classifier geometry as a particularly clean way to formalize maximal class separation.

4. Output-layer and optimization decoupling strategies

A second family of methods realizes SCL not through explicit simplex geometry, but by decoupling class groups during optimization so that one group does not suppress another.

Separated Softmax for Incremental Learning

“SS-IL: Separated Softmax for Incremental Learning” analyzes score bias in class-incremental learning and argues that joint softmax over old and new classes is a main cause of bias toward new classes (Ahn et al., 2020). Its proposed “Separated Softmax” computes the cross-entropy differently for old and new examples: $\mathbf{h}_k^\*$7 New-class samples are trained with a softmax over new classes only, so gradients from new data do not directly push down old-class logits (Ahn et al., 2020). The method is paired with task-wise knowledge distillation,

$\mathbf{h}_k^\*$8

rather than global KD (Ahn et al., 2020).

This is a form of separate class learning at the loss level: class groups are separated during training, even though inference still uses a unified classifier.

Split-and-Bridge

“Split-and-Bridge” implements a similar idea inside a single neural network by explicitly separating parameters for old and new classes during part of training (Kim et al., 2021). The network is decomposed into shared lower layers $\mathbf{h}_k^\*$9 and two upper partitions $\mathbf{w}_k^\*$0 and $\mathbf{w}_k^\*$1. The split-phase objective is

$\mathbf{w}_k^\*$2

where the old branch receives KD and the new branch receives a localized CE over new classes only (Kim et al., 2021). After this separate training, the branches are reconnected and trained with standard KD+CE to learn cross-task discrimination (Kim et al., 2021).

The method also includes a sparsification objective,

$\mathbf{w}_k^\*$3

to encourage explicit separation between old and new subnetworks (Kim et al., 2021).

These methods suggest that SCL can be formulated as optimization decoupling rather than only as metric geometry: separate class learning can mean preventing mutually destructive gradients between class groups.

5. Supervised and semantic contrastive variants

Another major usage of “SCL” in the provided literature refers to contrastive learning, sometimes directly and sometimes only as an acronym. These works are not always about Separate Class Learning in the strict terminological sense, but several embody class-separation objectives closely enough to be relevant.

Supervised contrastive learning in FSCIL

In “Learning with Fantasy,” SCL refers to supervised contrastive learning during the base session of few-shot class-incremental learning (Song et al., 2023). The supervised contrastive loss is

$\mathbf{w}_k^\*$4

The paper argues that naïve SCL yields only a modest gain in last-session accuracy on miniImageNet, whereas its proposed Semantic-Aware Virtual Contrastive model introduces virtual classes to create “placeholders for unseen classes in the representation space” (Song et al., 2023). Its fantasy-space classification loss is

$\mathbf{w}_k^\*$5

and the fantasy-space contrastive loss is averaged over transformed views (Song et al., 2023). This line of work suggests that plain class-wise contrastive separation may be insufficient unless the representation space is structured to accommodate future classes.

Semantic Contrastive Learning

In “Deep Clustering by Semantic Contrastive Learning,” SCL denotes “Semantic Contrastive Learning” and addresses unsupervised discovery of semantic classes from unlabeled images (Huang et al., 2021). The key instance-level loss is

$\mathbf{w}_k^\*$6

where the negative set $\mathbf{w}_k^\*$7 contains keys from other clusters only (Huang et al., 2021). A cluster-consistency loss aligns distance-based cluster assignments with classifier outputs,

$\mathbf{w}_k^\*$8

and the total objective is

$\mathbf{w}_k^\*$9

Although this paper does not use the phrase Separate Class Learning, it explicitly frames its goal as learning “separate class decision boundaries” from unlabeled data (Huang et al., 2021). It therefore belongs to the broader conceptual neighborhood of SCL as class-separating representation learning.

Hierarchy-aware SCL

“Learning Label Hierarchy with Supervised Contrastive Learning” starts from the observation that standard supervised contrastive learning treats classes as independent and equally important (Lian et al., 2024). Its scaled instance–instance loss

$\mathbf{w}_k^\* = \sqrt{\frac{n_k \lambda_H}{\lambda_W}}\, \mathbf{h}_k^\*,$0

modulates the contribution of negatives using a class-similarity matrix derived from hierarchical label descriptions (Lian et al., 2024). An instance–center loss brings examples closer to learnable label parameters that can be used directly as a classifier (Lian et al., 2024). This is relevant to SCL because it shows a limitation of naïve “separate classes” thinking: when labels are hierarchically related, strict equal separation may be suboptimal.

Hard-negative SCL

In “Supervised Contrastive Learning with Hard Negative Samples,” SCL is formalized through a class-conditional negative sampling distribution,

$\mathbf{w}_k^\* = \sqrt{\frac{n_k \lambda_H}{\lambda_W}}\, \mathbf{h}_k^\*,$1

and its hard-negative extension H-SCL further tilts this distribution by a hardening function (Jiang et al., 2022). This paper clarifies how supervision enforces separation by explicitly preventing class collisions in the negative sampling process (Jiang et al., 2022).

Taken together, these papers show that contrastive SCL variants operationalize class separation by sampling design, prototype alignment, or cluster-aware negatives rather than by simplex geometry alone.

6. Long-tailed OOD detection and refined separate class learning

A more recent explicit use of the term appears in “Long-Tailed Out-of-Distribution Detection with Refined Separate Class Learning” (Feng et al., 21 Sep 2025). Here SCL addresses the fact that OOD confusion occurs differently for head classes and tail classes in long-tailed data. The method extends Outlier Class Learning,

$\mathbf{w}_k^\* = \sqrt{\frac{n_k \lambda_H}{\lambda_W}}\, \mathbf{h}_k^\*,$2

with two separate modules: tail-specific class learning and head-specific class learning (Feng et al., 21 Sep 2025).

The tail-specific loss A-TSCL is

$\mathbf{w}_k^\* = \sqrt{\frac{n_k \lambda_H}{\lambda_W}}\, \mathbf{h}_k^\*,$3

while the head-specific A-OHL loss anchors informative outliers to an outlier prototype and repels them from head-class samples (Feng et al., 21 Sep 2025). The crucial refinement is dynamic class-wise temperature adjustment,

$\mathbf{w}_k^\* = \sqrt{\frac{n_k \lambda_H}{\lambda_W}}\, \mathbf{h}_k^\*,$4

which varies by class frequency and training epoch (Feng et al., 21 Sep 2025). The overall objective is

$\mathbf{w}_k^\* = \sqrt{\frac{n_k \lambda_H}{\lambda_W}}\, \mathbf{h}_k^\*,$5

This formulation makes explicit that SCL can mean different learning rules for different class subsets, motivated not by incremental learning but by asymmetric statistical roles in OOD detection.

7. Misconceptions and disambiguation

A common misconception is that “SCL” denotes a single established method family. The supplied literature shows otherwise. In semi-supervised learning, SCL in (Hyun et al., 2020) stands for “Suppressed Consistency Loss,” not Separate Class Learning (Hyun et al., 2020). In Semantic Contrastive Learning and Supervised Contrastive Learning papers, SCL refers to contrastive objectives rather than to a unified theory of class separation (Huang et al., 2021, Song et al., 2023, Lian et al., 2024, Jiang et al., 2022). In streaming and continual learning, SCL means “Streaming Continual Learning,” a framework for unifying CL and streaming machine learning (Giannini et al., 2 Mar 2026, Cossu et al., 2 Mar 2026). In theorem proving, SCL means “Clause Learning from Simple Models” (Leidinger et al., 2022).

Another misconception is that Separate Class Learning always implies fully independent per-class models. The literature shows softer forms: separated softmax computes different normalizations for old and new groups while retaining a shared classifier (Ahn et al., 2020); Split-and-Bridge temporarily splits only upper layers and then restores a unified network (Kim et al., 2021); MASIL shares a single backbone and concept bank while separating classes geometrically (Khandelwal, 2023).

A further misconception is that stronger class separation is always achieved by treating all classes as equally unrelated. Hierarchical contrastive learning shows that when label hierarchies exist, using class similarities can produce a “more well-structured and discriminative feature space” than flat class independence (Lian et al., 2024). This suggests that “separation” in SCL need not mean uniform repulsion.

8. Significance and research directions

Across the papers on arXiv, Separate Class Learning is significant because it addresses recurrent failure modes of standard joint training: catastrophic forgetting, old–new class bias, class overlap in few-shot regimes, weak tail-class structure, and OOD confusion with statistically dominant or statistically weak classes (Khandelwal, 2023, Ahn et al., 2020, Kim et al., 2021, Feng et al., 21 Sep 2025). The specific mechanisms differ, but the shared insight is that naïvely optimizing all classes under one undifferentiated objective can produce structurally poor decision boundaries.

Several directions are already visible in the literature. One is increasingly explicit geometric structure, especially ETF-based classifier design and neural-collapse-inspired objectives (Khandelwal, 2023, Zhang et al., 26 Dec 2025). Another is class-group-aware optimization, as in separated softmax, split architectures, and head/tail-specific contrastive modules (Ahn et al., 2020, Kim et al., 2021, Feng et al., 21 Sep 2025). A third is the enrichment of class separation with semantic structure, such as concept factorization or label hierarchy information (Khandelwal, 2023, Lian et al., 2024). A plausible implication is that future SCL work may continue moving away from purely flat class separation toward structured separation that reflects geometry, frequency, semantic relatedness, and temporal evolution.

In that sense, Separate Class Learning is less a single algorithm than an organizing principle: classes, or meaningful subsets of classes, are not merely labels in a shared objective, but entities that may require explicit geometric, architectural, or optimization-level separation.