Free-Flow Class-Incremental Learning

• FFCIL is a class-incremental learning paradigm where novel classes arrive unpredictably, requiring immediate model updates without prior scheduling.
• It addresses challenges like replay imbalance and step-dependent loss-scale shifts by employing methods such as Class-Wise Mean objectives and Dynamic Intervention Weight Alignment.
• Research in FFCIL demonstrates that techniques including synthetic future data and fixed classifier designs improve stability in both exemplar-free and replay-based settings.

Searching arXiv for the cited papers to ground the article in current literature. Free-Flow Class-Incremental Learning (FFCIL) denotes a class-incremental learning regime in which new classes arrive as a realistic stream with a highly variable number of unseen classes at each update, rather than under a predefined schedule of equal-sized tasks (Xu et al., 3 Apr 2026). In this setting, the learner must update immediately when any non-empty set of novel classes appears, without assuming advance knowledge of step size. The term has become associated with a broader shift in continual-learning methodology: away from benchmark-convenient task partitions and toward algorithms that remain stable under irregular class arrivals, replay imbalance, and step-dependent loss-scale variation. Closely related lines of work address especially hard subcases, including exemplar-free class-incremental learning with frozen feature extractors and no rehearsal memory (Jodelet et al., 2024), as well as architectural designs that reduce the need for classifier expansion during long class streams (Pernici et al., 2020).

1. Formal definition and problem setting

In standard class-incremental learning (CIL), learning proceeds over a sequence of tasks $D_1,D_2,\dots,D_t$, each with a disjoint class set, and the classifier expands to cover the cumulative label space

$\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$

Training at step $t$ is typically performed using the current task together with a small replay memory of old exemplars, and evaluation is conducted on all classes seen so far, without task identity at test time (Xu et al., 3 Apr 2026).

FFCIL removes the regularity assumption that task schedules are fixed or nearly fixed. Let $\mathcal{C}_t$ denote the new class set introduced at step $t$. The defining constraints are

$$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$

while the increment size is allowed to vary arbitrarily: $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$ The central distinction is therefore not merely that FFCIL is “harder” than conventional CIL, but that it assumes a different data stream: in one update the system may receive one novel class, and in the next it may receive tens of new classes (Xu et al., 3 Apr 2026).

A related but narrower formulation appears in exemplar-free class-incremental learning (EFCIL), where replay memory is unavailable. In that setting, a unified model $\theta=\{\Phi,\mathcal{W}\}$ is learned over $T$ incremental steps, each step $t$ providing a dataset $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$0 with new classes $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$1, with

$\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$2

At step $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$3, the model $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$4 is trained solely on $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$5, and evaluation is done on all classes seen so far, without any task ID (Jodelet et al., 2024). This no-replay regime aligns closely with a hard FFCIL scenario, but FFCIL as formalized later is not restricted to exemplar-free methods.

2. Instabilities induced by free-flow increments

The free-flow assumption exposes several failure modes that are largely hidden by equal-task benchmarks. First, the effective class prior inside each mini-batch changes dramatically from step to step. Second, replayed old exemplars and newly arriving classes can be mixed in highly uneven proportions depending on the current increment size. Third, losses whose magnitudes depend on the number of active classes or valid negatives can vary substantially across updates. The result is step-dependent supervision imbalance and step-dependent gradient-scale shifts, which increase optimization inconsistency and catastrophic forgetting (Xu et al., 3 Apr 2026).

These effects are empirically pronounced. On CIFAR-100, Replay drops from $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$6 under equal tasks to $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$7 in the original free-flow setting, and iCaRL drops from $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$8 to $\mathcal{Y}=Y_1\cup Y_2\cup \cdots \cup Y_t.$9. BiC degrades from $t$0 to $t$1. On ImageNet, iCaRL falls from $t$2 to $t$3. On VTAB, TagFex drops from $t$4 to $t$5, and on ImageNet TagFex falls from $t$6 to $t$7 (Xu et al., 3 Apr 2026). The same paper reports confusion matrices showing that under free-flow arrivals the classifier output distribution becomes skewed and recent classes can be badly under-predicted; under an extreme CIFAR-100 schedule with 90 classes arriving initially and only 1–2 classes afterward, TagFex reportedly degrades to around $t$8 accuracy.

A common misconception is that FFCIL is only a scheduling variant of standard CIL. The formalization and the reported degradation indicate a stronger claim: many existing methods are implicitly tuned to the regularity of benchmark task splits rather than to irregular real streams (Xu et al., 3 Apr 2026).

3. Model-agnostic stabilization strategies

The principal optimization response proposed specifically for FFCIL is a model-agnostic framework built around the Class-Wise Mean (CWM) objective (Xu et al., 3 Apr 2026). Standard mini-batch cross-entropy can be written as a class-weighted average in which each class contribution is scaled by its within-batch frequency $t$9. Under free-flow increments this is unstable because class frequency changes with the current increment size. CWM removes the implicit frequency bias by averaging per class first and then averaging uniformly across classes present in the batch: $\mathcal{C}_t$0 This makes each class contribute equally regardless of how many samples of that class appear in the batch.

For distillation-based methods, the same work argues that vanilla knowledge distillation has a step-dependent gradient decomposition because the ratio $\mathcal{C}_t$1 versus $\mathcal{C}_t$2 varies with the number of arriving classes. The proposed remedy is twofold: apply CWM to distillation and, when replay is used, restrict distillation to replayed old-class samples. If no replay memory is used, the distillation term is simply the CWM version over the batch. The stated intuition is that distillation should preserve old classes, and under free-flow updates it is safer to apply it only where the old-class signal is stable (Xu et al., 3 Apr 2026).

For dynamic-expansion methods such as DER and MEMO, the auxiliary classifier loss is also reformulated with a CWM objective over step-relative labels. For methods such as TagFex that combine auxiliary cross-entropy, contrastive loss, and knowledge transfer, the framework additionally normalizes losses whose scale changes with step composition. The contrastive term is normalized by $\mathcal{C}_t$3, where $\mathcal{C}_t$4 is the effective number of valid negatives, and the knowledge-transfer term is normalized by $\mathcal{C}_t$5 when its scale depends on the number of new classes (Xu et al., 3 Apr 2026).

For weight-alignment methods, the paper introduces Dynamic Intervention Weight Alignment (DIWA). Standard Weight Alignment rescales newly learned classifier weights so that their average $\mathcal{C}_t$6 norm matches that of old-class weights. Under FFCIL, this can over-correct when only a few new classes arrive, because the estimate of $\mathcal{C}_t$7 is then noisy. DIWA modulates the intervention strength according to the current increment size: $\mathcal{C}_t$8

$\mathcal{C}_t$9

Small increments therefore receive gentler calibration, while large increments permit stronger alignment (Xu et al., 3 Apr 2026).

The framework is evaluated on Replay, iCaRL, WA, BiC, DER, MEMO, and TagFex. The reported pattern is consistent: FFCIL hurts all methods, and CWM, replay-only distillation, DIWA, and loss normalization recover much of the lost performance, with little or no computational overhead (Xu et al., 3 Apr 2026).

4. Exemplar-free and frozen-backbone formulations

A distinct but closely related research direction studies the no-replay regime, where the learner cannot revisit old training exemplars. “Future-Proofing Class Incremental Learning” introduces Future-Proof Class Incremental Learning (FPCIL) for this setting (Jodelet et al., 2024). The motivating observation is that frozen feature extractor methods in exemplar-free class-incremental learning perform well only if the extractor was trained on a sufficiently rich initial set. When the first incremental step contains too few classes, the learned representation is insufficiently transferable and later steps suffer.

FPCIL modifies only the first incremental step. Instead of training on $t$0 alone, it trains on

$t$1

where $t$2 is an auxiliary labeled dataset containing classes expected to appear in future steps. In the idealized case,

$t$3

After this first step, the classifier is restricted back to the base classes by dropping the weights associated with future classes, and the feature extractor $t$4 is frozen for the remainder of learning. The method is explicitly designed to be compatible with frozen-backbone exemplar-free methods such as FeTrIL (Jodelet et al., 2024).

The main novelty is that $t$5 is built from synthetic rather than real future-class images. A pre-trained text-to-image diffusion model $t$6 is conditioned on a prompt of the form

$t$7

where $t$8 is the class name and $t$9 is a simple textual definition obtained automatically with WordNet from the synset lemmas and synset definition. The purpose is to reduce ambiguity and homonym errors, such as “crane” as bird versus machine. For each future class, the method generates $$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$0 synthetic images, where $$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$1 equals the number of real images per class in the target dataset (Jodelet et al., 2024).

The default generator is Stable Diffusion v1.4 with guidance scale $$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$2 and 50 denoising steps. The same study also tests DALL-E2 and Stable Diffusion with guidance scale $$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$3, concluding that, for pre-training a feature extractor for future incremental learning, diversity matters more than image fidelity (Jodelet et al., 2024). This is a narrowly defined use of synthetic data: it is employed only at initialization to induce a more general representation before the backbone is frozen.

The practical claim is equally specific. Because only labels of future classes are needed, the approach is argued to be viable in deployments where the likely future label space is known even if actual future data are unavailable (Jodelet et al., 2024). A plausible implication is that FFCIL can sometimes be addressed not only by online stabilization during the stream, but also by improving the representational prior before the stream becomes difficult.

5. Pre-allocated classifiers and representation geometry

Another antecedent relevant to free-flow learning replaces the expanding classifier head with a fixed, pre-allocated output layer. “Class-incremental Learning with Pre-allocated Fixed Classifiers” proposes a Regular Polytope Classifier (RPC) in which output weights are fixed from the beginning and already participate in the loss throughout the learning stream (Pernici et al., 2020). Although this work predates the formalization of FFCIL, it is directly pertinent to settings in which classes arrive online and repeated classifier expansion is undesirable.

The central difference from standard expanding classifiers is structural. In the expanding case, new output nodes are created only when classes arrive. In the pre-allocated fixed classifier, all output nodes exist from the beginning and are not trained; instead, the feature extractor learns to align class features with predetermined output directions. This yields two stated benefits: future unseen classes can see negative samples since the beginning of learning, and features do not change their geometric configuration as novel classes are incorporated (Pernici et al., 2020).

The classifier geometry is derived from a regular polytope, specifically the $$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$4-Simplex. The construction given is

$$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$5

with

$$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$6

followed by centroid shifting and normalization. In embedding dimension $$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$7, the $$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$8-simplex classifier accommodates

$$|\mathcal{C}_t|\ge 1,\qquad \mathcal{C}_t\cap \mathcal{C}_s=\varnothing\ \ \forall t\neq s,$$9

classes, which the paper states is the maximum number of equidistant class vertices embeddable in $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$0 (Pernici et al., 2020).

The method is evaluated in an Experience Replay setting, with updates of the form

$$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$1

where $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$2 is a memory buffer and task identifiers are ignored in CIL. The reported conclusion is that the pre-allocated classifier is as effective as the expanding classifier while exhibiting stable geometric properties that the expanding alternative does not show (Pernici et al., 2020). The same paper also reports that pre-allocating many classes does not substantially affect final performance, which supports long class streams in principle.

In the context of FFCIL, this suggests an architectural complement to optimization-based stabilization: rather than only correcting losses after variable-size arrivals occur, one can also reduce the amount of classifier restructuring required when they occur.

6. Empirical findings, misconceptions, and research directions

The empirical record across these works supports three distinct but connected claims: variable-size class arrivals degrade standard CIL methods; targeted stabilization can recover much of the loss; and in no-replay settings the quality of the initial representation is decisive (Xu et al., 3 Apr 2026).

Setting	Baseline	Reported change
CIFAR-100, BiC	Equ.T $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$3 to FF.org $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$4	FF.ours $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$5
CIFAR-100, WA	Equ.T $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$6 to FF.org $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$7	FF.ours $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$8
CIFAR-100, TagFex	Equ.T $$\big||\mathcal{C}_t|-|\mathcal{C}_{t-1}|\big| \text{ is unbounded.}$$9 to FF.org $\theta=\{\Phi,\mathcal{W}\}$0	FF.ours $\theta=\{\Phi,\mathcal{W}\}$1
ImageNet, iCaRL	Equ.T $\theta=\{\Phi,\mathcal{W}\}$2 to FF.org $\theta=\{\Phi,\mathcal{W}\}$3	degradation under FFCIL
VTAB, TagFex	Equ.T $\theta=\{\Phi,\mathcal{W}\}$4 to FF.org $\theta=\{\Phi,\mathcal{W}\}$5	degradation under FFCIL

In the exemplar-free frozen-backbone regime, the gains from future-proofing are especially large when the first step is small (Jodelet et al., 2024).

Protocol	FeTrIL	FPCIL-Oracle
CIFAR100 B0 Inc10	$\theta=\{\Phi,\mathcal{W}\}$6	$\theta=\{\Phi,\mathcal{W}\}$7
CIFAR100 B0 Inc5	$\theta=\{\Phi,\mathcal{W}\}$8	$\theta=\{\Phi,\mathcal{W}\}$9
ImageNet-Subset B0 Inc10	$T$0	$T$1
ImageNet-Subset B0 Inc5	$T$2	$T$3

The 2024 study further reports that even FPCIL-Partial (0%), where none of the auxiliary classes are actually future classes, still improves FeTrIL to $T$4 on CIFAR100 B0 Inc10 and $T$5 on B0 Inc5; on ImageNet-Subset the corresponding figures are $T$6 and $T$7 (Jodelet et al., 2024). Performance degrades smoothly as the fraction of correctly predicted future classes drops from $T$8 to $T$9, which shows that perfect forecasting is not required, although better alignment with actual future classes yields better results.

Several misconceptions are therefore not supported by the current evidence. First, FFCIL is not synonymous with exemplar-free learning: the formalization in 2026 explicitly covers replay-based, distillation-based, weight-alignment, and dynamic-expansion methods (Xu et al., 3 Apr 2026). Second, more data alone is not the operative explanation for improvement in the exemplar-free future-proofing work: synthetic future-class data outperforms real auxiliary data from different classes, and FPCIL-Oracle outperforms ImageNet-Compl. X2, which contains twice as many real auxiliary classes (Jodelet et al., 2024). Third, future-proofing does not depend on maximum image fidelity: Stable Diffusion with guidance scale $t$0 performs better than guidance scale $t$1 and better than DALL-E2, leading to the conclusion that diversity matters more than image fidelity for representation learning in this setting (Jodelet et al., 2024).

The current literature also delineates the remaining scope of the problem. The 2026 framework primarily addresses optimization and calibration instability caused by variable class counts, rather than proposing an architecture specifically designed for FFCIL (Xu et al., 3 Apr 2026). The 2020 fixed-classifier work avoids classifier expansion and stabilizes feature geometry, but is evaluated on standard benchmarks with replay rather than on unrestricted free-flow streams (Pernici et al., 2020). The 2024 future-proofing method is highly effective in exemplar-free frozen-backbone settings, but it assumes that likely future labels are available in advance and that synthetic data can be generated for them (Jodelet et al., 2024). Taken together, these results suggest that FFCIL is best understood not as a single algorithmic template, but as a research program centered on variable-size class arrivals, stability under step heterogeneity, and anticipatory representation learning.